mixtou mixtou - 1 month ago 12
Java Question

Apache Spark: How can i access nested array of integers within Tuple2 object in Java?

I am new to spark and i am trying to perform a mapping of words to counts and errors for the implementation of Space Saving Algorithm. For every word i need to keeps its counts and its errors. My code the following. The problem is that i cannot access the elements in array of integers within tuple2.

JavaPairRDD<String, Integer[]> ones = words.mapToPair(new PairFunction<String, String, Integer[]>() {
@Override
public Tuple2<String, Integer[]> call(String s) {
Integer[] defaults = {1,0};
return new Tuple2<String, Integer[]>(s, defaults);
}
});

List<Tuple2<String,Integer[]>> output = ones.collect();

for (Tuple2<?, ?> tuple : output) {
Integer temp = (Integer) tuple._2()[0]; ///ERROR, HOW CAN I ACCESS ARRAY ELEMENT???
System.out.println(tuple._1() + ": " + tuple._2()+" "+temp[0]);
}

Answer

change this:

for (Tuple2<?, ?> tuple : output) {
    Integer temp = (Integer) tuple._2()[0];  ///ERROR, HOW CAN I ACCESS ARRAY ELEMENT???
    System.out.println(tuple._1() + ": " + tuple._2()+" "+temp[0]);
}

to

for (Tuple2<String, Integer[]> tuple : output) {
    Integer temp = tuple._2()[0];
    System.out.println(tuple._1() + ": " + tuple._2() + " " + temp[0]);
}