InamTaj InamTaj - 3 months ago 20
Java Question

JAVA HttpURLConnection I/O Not working

Hello My Respected Seniors :)

My Goal: Download a URL Resource, given a URL, by using Multi-Threading in Java, i.e. download a single file into multiple pieces (much like how IDM does) & at the end of download, combine all of them to 1 final file.

Technology Using: Java, RandomAccessFile, MultiThreading, InputStreams

Problem:
The file is downloaded fine with exact KB size, I've checked many times, but the final file is corrupted. For example, If I download an Image, it will be somewhat blurry, If I download an .exe, it downloads fine but when I run the .exe file, it says "media is damaged, retry download".

This is my Main code from which I call to thread class with parameters such as fileName, starting Range and ending Range for a connection as well as a JProgressBar for every thread which will update its own respectively.

public void InitiateDownload()
{
HttpURLConnection uc = (HttpURLConnection) url.openConnection();
uc.connect();
long fileSize = uc.getContentLengthLong();
System.out.println("File Size = "+ fileSize );
uc.disconnect();


chunkSize = (long) Math.ceil(fileSize/6);

startFrom = 0;
endRange = (startFrom + chunkSize) - 1;

Thread t1 = new MyThread(url, fileName, startFrom, endRange, progressBar_1);
t1.start();
//-----------------------------------------

startFrom += chunkSize;
endRange = endRange + chunkSize;
System.out.println("Part 2 :: Start = " + startFrom + "\tEnd To = " + endRange );

Thread t2 = new MyThread(url, fileName, startFrom, endRange, progressBar_2);
t2.start();
//-----------------------------------------
//..
//..
//..
//-----------------------------------------
startFrom += chunkSize;
long temp = endRange + chunkSize;
endRange = temp + (fileSize - temp); //add any remaining bits, that were rounded off in division

Thread t6 = new MyThread(url, fileName, startFrom, endRange, progressBar_6);
t6.start();
//-----------------------------------------
}


Here is run() function of MyThread class:

public void run() {

Thread.currentThread().setPriority(MAX_PRIORITY);

System.setProperty("http.proxyHost", "192.168.10.50");
System.setProperty("http.proxyPort", "8080");

HttpURLConnection uc = null;
try {

uc = (HttpURLConnection) url.openConnection();

uc.setRequestProperty("Range", "bytes="+startFrom+"-"+range);
uc.connect();
fileSize = uc.getContentLengthLong();

inStream = uc.getInputStream();

int[] buffer = new int[ (int) totalDownloadSize ];

file.seek(startFrom); //adjusted start of file


THIS IS WHERE I THINK THE PROBLEM IS,

run() continued...

for(int i = 0 ; i < totalDownloadSize; i++)
{
buffer[i] = inStream.read();
file.write(buffer[i]);

//Updating Progress bars
totalDownloaded = totalDownloaded + 1;
int downloaded = (int) (100 * ( totalDownloaded/ (float) totalDownloadSize)) ;
progressbar.setValue( downloaded );

}


System.err.println( Thread.currentThread().getName() + "'s download is Finished!");
uc.disconnect();
}
catch(IOException e) {
System.err.println("Exception in " + Thread.currentThread().getName() + "\t Exception = " + e );

}
finally {
try {
file.close();
if(inStream!=null)
inStream.close();

} catch (IOException e) {
e.printStackTrace();
}
}

}


}

Now the file is downloaded with complete size, but as I said, a little part of it is corrupt.
Now,
If I replace the
for loop
with following
while
loop, the problem is completely solved.

int bytesRead = 0;
byte[] buffer = new byte[ (int) totalDownloadSize ];

file.seek(startFrom); //adjusted start of file

while( (bytesRead = inStream.read(buffer) ) != -1 ) {

file.write(buffer, 0, bytesRead);

}


BUT I NEED
for
LOOP TO MEASURE HOW MUCH FILE EACH THREAD HAS DOWNLOADED & I WANT TO UPGRADE RESPECTIVE JPROGRESSBARs of THREADS.


Kindly help me out with the for loop logic.

OR

If you can advise on how can I upgrade Jprogressbars within
while
loop. I can't seem to find a way to quantify how much file a thread has downloaded...

I've spent alot of hours & I'm extremely tired now...

Answer

You can use the while loop that works, and then keep track of the total amount of bytes read like this:

int totalRead = 0;
while ((bytesRead = inStream.read(buffer)) != -1) {
    totalRead += bytesRead;
    file.write(buffer, 0, bytesRead);

    progressBar.setValue((int)(totalRead / (double) totalDownloadSize));
}

just remember that for (a; b; c) { ... } is equal to a; while (b) { c; ... }.