user3663622 user3663622 - 3 years ago 166
Scala Question

Using tuple as a key in scala

  • Question 1: Can I use tuple as a key of a map in Scala?

  • Question 2: If yes , how can I create a map with a tuple as key?

  • Question 3: I want to convert my scala map to RDD, how would I do in the following case? I am trying to do in this way

    var mapRDD = sc.parallelize(map.toList)

    Is this the right way to do ?

  • Question 4: For this particular code snippet, when I do a println on map, it has no values.

I have not included the whole code, basically mapAgainstValue contains userId as key and list of friends as values. I want to recreate a map RDD with the following transformation in the key.
What would be the reason for empty map?

var mapAgainstValue =>x.split("\t")).filter(x => x.length == 2).map(x => (x(0),x(1).split(",")))
var map:Map[String,List[String]] = Map()
var changedMap ={
line =>
var key ="";
for(userIds <- line._2){
if(line._1.toInt < userIds.toInt){
key =line._1.concat("-"+userIds);
else {
key = userIds.concat("-" + line._1);
map += (key -> line._2.toList)

Answer Source

Yes, you can use Tuple as a key in Map.

For example:

val userMap = Map(
    (1, 25) -> "shankar",
    (2, 35) -> "ramesh")

Then you can try print the output using foreach

val userMapRDD = sparkContext.parallelize(userMap.toSeq, 2)
  mapRDD.foreach(element => {

If you want to transform the mapRDD to something else. following code returns only age and name as tuple.

  val mappedRDD = {
    case ((empId: Int, age: Int), name: String) => {
      (age, name)
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download