adamsmith adamsmith - 1 month ago 18
Java Question

java GZIPInputStream and GZIPOutputStream not working as expected

I am trying to write a series of

Long
's to a
GZIPOutputStream
, hoping to decompress the numbers later.

The following program works fine when I try it with a small number of
Long
's, but it will throw an exception with many
Long
's, like (10240).

Please see and run this code:

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;

public class GzipTest {
private static final String filename = "data.gz";

public static void main(String... args) throws Exception {
test(32); // OK
test(10240); // Why won't this work? (unexpected read: 6)
}

public static void test(int max) throws Exception {
System.out.println("testing with " + max);
FileOutputStream fos = new FileOutputStream(filename);
GZIPOutputStream gos = new GZIPOutputStream(fos, 4096);
long num = 10241000L;
for (int i = 0; i < max; ++i) {
gos.write(long2bytes(num + i));
}
gos.close();
fos.close();

GZIPInputStream gis = new GZIPInputStream(new FileInputStream(filename), 4096);
byte[] buf = new byte[Long.BYTES];
for (int i = 0; i < max; ++i) {
int nread = gis.read(buf);
if (nread != Long.BYTES) {
throw new RuntimeException("unexpected read: " + nread);
}
long value = bytes2long(buf);
if (value != num + i) {
throw new RuntimeException("unexpected value: " + value);
}
}
gis.close();
}

static byte[] long2bytes(long num) {
ByteBuffer buf = ByteBuffer.allocate(Long.BYTES);
buf.putLong(num);
return buf.array();
}

static long bytes2long(byte[] bytes) {
return ByteBuffer.wrap(bytes).getLong();
}
}


The outputs:

testing with 32
testing with 10240
Exception in thread "main" java.lang.RuntimeException: unexpected read: 6
at GzipTest.test(GzipTest.java:31)
at GzipTest.main(GzipTest.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

Answer

Th code works without any problems if you make the changes proposed by Peter Lawrey:

 try (DataInputStream in = new DataInputStream(new GZIPInputStream(new FileInputStream(filename), 4096))) {
        byte[] buf = new byte[Long.BYTES];
        for (int i = 0; i < max; ++i) {
            in.readFully(buf);
            long value = bytes2long(buf);
            if (value != num + i) {
                throw new RuntimeException("unexpected value: " + value);
            }
        }
    }
Comments