Anaadih.pradeep Anaadih.pradeep - 2 months ago 7
Scala Question

Group By error 33 : wrong number of parameter

I am kind of new to Spark and programming too and i am not able to understand how to deal with Rdd of type

rdd.RDD[(Int, Iterable[Double])] = ShuffledRDD[10] at groupByKey
. I am little bit interested in learning groupByKey in spark and i have a filtered RDD

scala> p.first
res11: (Int, Double) = (1,299.98)

I go the above result after applying GroupByKEy instead of reduceByKey now i have rdd of type (Int, Iterable[Double]) and i want to get result like (Int , sum (Double)).
I have tried this but got the error.

scala> val price =,b) => (a, sum(b)))
<console>:33: error: wrong number of parameters; expected = 1
val price =,b) => (a, sum(b)))

Please suggest and help me in this to understand it


g.mapValues(_.sum), which is short for { case (k, v) => (k, v.sum) }