Coco Coco - 1 month ago 12
Java Question

Java 8 - Most effective way to merge List<byte[]> to byte[]

I have a library that returns some binary data as list of binary arrays. Those byte[] need to be merged into an InputStream.

This is my current implementation:

public static InputStream foo(List<byte[]> binary) {
byte[] streamArray = null;
binary.forEach(bin -> {
org.apache.commons.lang.ArrayUtils.addAll(streamArray, bin);
});
return new ByteArrayInputStream(streamArray);
}


but this is quite cpu intense. Is there a better way?

Thanks for all the answers. I did a performance test. Those are my results:


  • Function: 'NicolasFilotto' => 63,39 ms average on 100 calls

  • Function: 'NicolasFilottoEstSize' => 60,17 ms average on 100 calls

  • Function: 'Saka1029_1' => 59,96 ms average on 100 calls

  • Function: 'Saka1029_2' => 0,79 ms average on 100 calls

  • Function: 'Coco' => 481,20 ms average on 10 calls



I'm not sure if 'Saka1029_2' is measured correctly...

this is the execute function:

private static double execute(Callable<InputStream> funct, int times) throws Exception {
List<Long> executions = new ArrayList<>(times);

for (int idx = 0; idx < times; idx++) {
BufferedReader br = null;
long startTime = System.currentTimeMillis();
InputStream is = funct.call();
br = new BufferedReader(new InputStreamReader(is));
String line = null;
while ((line = br.readLine()) != null) {}
executions.add(System.currentTimeMillis() - startTime);
}

return calculateAverage(executions);
}


note that I read every input stream

those are the used implementations:

public static class NicolasFilotto implements Callable<InputStream> {

private final List<byte[]> binary;

public NicolasFilotto(List<byte[]> binary) {
this.binary = binary;
}

@Override
public InputStream call() throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
for (byte[] bytes : binary) {
baos.write(bytes, 0, bytes.length);
}
return new ByteArrayInputStream(baos.toByteArray());
}

}

public static class NicolasFilottoEstSize implements Callable<InputStream> {

private final List<byte[]> binary;
private final int lineSize;

public NicolasFilottoEstSize(List<byte[]> binary, int lineSize) {
this.binary = binary;
this.lineSize = lineSize;
}

@Override
public InputStream call() throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream(binary.size() * lineSize);
for (byte[] bytes : binary) {
baos.write(bytes, 0, bytes.length);
}
return new ByteArrayInputStream(baos.toByteArray());
}

}

public static class Saka1029_1 implements Callable<InputStream> {

private final List<byte[]> binary;

public Saka1029_1(List<byte[]> binary) {
this.binary = binary;
}

@Override
public InputStream call() throws Exception {
byte[] all = new byte[binary.stream().mapToInt(a -> a.length).sum()];
int pos = 0;
for (byte[] bin : binary) {
int length = bin.length;
System.arraycopy(bin, 0, all, pos, length);
pos += length;
}
return new ByteArrayInputStream(all);
}

}

public static class Saka1029_2 implements Callable<InputStream> {

private final List<byte[]> binary;

public Saka1029_2(List<byte[]> binary) {
this.binary = binary;
}

@Override
public InputStream call() throws Exception {
int size = binary.size();
return new InputStream() {
int i = 0, j = 0;

@Override
public int read() throws IOException {
if (i >= size) return -1;
if (j >= binary.get(i).length) {
++i;
j = 0;
}
if (i >= size) return -1;
return binary.get(i)[j++];
}
};
}

}

public static class Coco implements Callable<InputStream> {

private final List<byte[]> binary;

public Coco(List<byte[]> binary) {
this.binary = binary;
}

@Override
public InputStream call() throws Exception {
byte[] streamArray = new byte[0];
for (byte[] bin : binary) {
streamArray = org.apache.commons.lang.ArrayUtils.addAll(streamArray, bin);
}
return new ByteArrayInputStream(streamArray);
}

}

Answer

Try this.

public static InputStream foo(List<byte[]> binary) {
    byte[] all = new byte[binary.stream().mapToInt(a -> a.length).sum()];
    int pos = 0;
    for (byte[] bin : binary) {
        int length = bin.length;
        System.arraycopy(bin, 0, all, pos, length);
        pos += length;
    }
    return new ByteArrayInputStream(all);
}

Or

public static InputStream foo(List<byte[]> binary) {
    int size = binary.size();
    return new InputStream() {
        int i = 0, j = 0;
        @Override
        public int read() throws IOException {
            if (i >= size) return -1;
            if (j >= binary.get(i).length) {
                ++i;
                j = 0;
            }
            if (i >= size) return -1;
            return binary.get(i)[j++];
        }
    };
}
Comments