neoguy neoguy - 9 days ago 5
Scala Question

reduceByKey results in Infinity value

I am trying to add values for same key.

val final= d1.join(d2).flatMap(line => Seq(line.swap._1)).reduceByKey((x, y) =>(x+y))


d1 and d2 are data streams. After flatMap I get Key value pair.

However, it is resulting in Infinity value in this line
reduceByKey((x, y) =>(x+y))


for example, if the pairs are (k1,1.0) (k1,1.0) the line
reduceByKey((x, y) =>(x+y))
results in (k1,Infinity)

Any suggestion?

Answer

The above code snippet is working. As @maasg righty hinted the problem was elsewhere. The error was caused by division by zero in previous code which I didnt post here. Thanks!