Ije Ije - 10 months ago 67
Java Question

java.lang.ArrayIndexOutOfBoundsException: MapReduce

Beginner level - can't get my head around this.

I wrote a MapReduce program that will result in a list of words and which line number they appear in, e.g.

  • and: 4

  • if: 1, 2, 3

etc. (see Sample Text below).

My code works perfectly well when there is no blank line as in line 2. But it throws an Exception 1 error for the below sample text.

From what I understand, this error indicates that there is an element in the array I am trying to access that doesn't exist - in this case there is just no element in the array for line 2. But how can I edit my code to ignore the blank line?

Here's the Mapper Code (with sample text):

import java.io.IOException;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

// SampleText:
1: if you prick us do we not bleed
3: if you tickle us do we not laugh
4: if you poison us do we not_ die and
5: ***if you wrong us shall we not revenge

public class IIndexMapper extends Mapper<LongWritable, Text, Text, Text>
private final static Text listing = new Text();
private Text wordText = new Text();

public void map(LongWritable key, Text value, Context context)
throws IOException, InterruptedException
//Single out line number by splitting each line by colon - first part being lineID
String[] line = value.toString().split(": ");
String lineID = line[0];

//Further split second part of the line by spaces
String textStr = line[1];

//Create an array of words contained in each line
String [] tokens = textStr.split(" ");
int count = tokens.length;

for (int i = 0; i < count; i++)
context.write(wordText, listing);


Any help is much appreciated!


Answer Source

I'd rewrite the above code something like this because we are accessing Elements in String Array with checking the existence.

String[] line = value.toString().split(": ");
if(line.length >= 2){


    //Create an array of words contained in each line
    String [] tokens = line[1].split(" ");
    int count = tokens.length;

    for (String token : tokens){
        context.write(wordText, listing);
} else {
    // increase the counter for tacking bad lines