sai sai - 1 month ago 14
Scala Question

word count in spark-scala for rdd(String,String,Long)

I am a new to Spark-scala, trying to solve simple word count(having multiple attributes as keys). can I get some inputs?
I have an Rdd(String,String,Long) like
(a,b,1)
(a,c,1)
(a,c,1)
(b,b,1)
(b,b,1)

desired result is an rdd like
(a,b,1)
(a,c,2)
(b,b,2)

Answer

Try:

rdd.map { 
  case (x, y, c) => ((x, y), c) 
}.reduceByKey(_ + _)