Suzy Tros Suzy Tros - 5 months ago 25
Java Question

reducer with iterator error

I have a reducer in hadoop with this code:

public void reduce(Text key, Iterator<Text> values, Context context) throws IOException, InterruptedException {
/*some code*/
String followers = "";
while(values.hasNext()){
followers = followers + values.next().toString() + ",";
}
/*some code*/
}


I want to make a list of followers for a certain node but when I run it I am getting this:

Error: java.lang.ArrayIndexOutOfBoundsException: 1


If values is type Iterable then I don't have a problem but why is this happening?

thanks in advance.

Answer

If you're extending org.apache.hadoop.mapreduce.Reducer (Java Docs) the method signature you've used for reduce() is incorrect. When you implement the reduce() method, you are overriding the implementation in the Reduce class.

You have:

reduce(Text key, Iterator<Text> values, Context context)

It should be:

reduce(Text key, Iterable<Text> values, Context context)

Note the difference between Iterator and Iterable. This probably means your reduce isnt actually being called.

You can add the @Override annotation to the method to help prevent this.

Comments