Kevin Kevin - 4 years ago 85
Java Question

Download file in java when Content-Length is incorrectly set?

I have an android app that downloads and uses a file at runtime. The file is valid as I can download it via the browser and open it up, etc. However my app kept reporting that the file is corrupted.

After investigation I discovered the server (which I have no control over) is returning an incorrect "Content-Length:" (~180 vs ~120000). The header is the culprit as I confirmed the issue by downloading the file with curl - which also resulted in a truncated file.

After some research I concluded that my use of

BufferedInputStream
to append to a
ByteArrayBuffer
is autosizing the byte array to the url connections content length. To circumvent this, I tried to use
ByteArrayOutputStream
instead, however this solved nothing.

Anybody know of a way to download a file if the Content-Length is incorrectly set? A browser can.

Here's my latest attempt:

public static void downloadFileFromRemoteUrl(String urlString, String destination){
try {
URL url = new URL(urlString);
File file = new File(destination);
URLConnection urlConnection = url.openConnection();
InputStream inputStream = urlConnection.getInputStream();
byte[] buffer = new byte[1024];
int curLength = 0;
int newLength = 0;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
while((newLength = inputStream.read(buffer))>0)
{
curLength += newLength;
byteArrayOutputStream.write(buffer, 0, newLength);
}
FileOutputStream fos = new FileOutputStream(file);
fos.write(byteArrayOutputStream.toByteArray());
fos.close();
android.util.Log.d("DB UPDATE", "Done downloading database. Size: " + byteArrayOutputStream.toByteArray().length);
}
catch (MalformedURLException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
}

EJP EJP
Answer Source

After some research I concluded that my use of BufferedInputStream to append to a ByteArrayBuffer is autosizing the byte array to the url connections content length.

Nonsense. You are crediting those classes with paranormal powers. How could an output stream possibly become aware of the Content-length header? The URLConnection's input stream is being terminated at the content-length. Correctly.

To circumvent this, I tried to use ByteArrayOutputStream instead, however this solved nothing.

Of course not.

Anybody know of a way to download a file if the Content-Length is incorrectly set?

You could use a Socket and engage in HTTP yourself, which is less trivial than it sounds. But the problem is at the server and that's where it should be fixed. Complain. Or else @Zong Yu is correct and the page is HTML containing JavaScript, say.

NB You don't need to read the entire file into memory:

while((newLength = inputStream.read(buffer))>0)
{
    curLength += newLength;
    fos.write(buffer, 0, newLength);
}
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download