tipsywacky tipsywacky -4 years ago 71
Python Question

How to unfreeze requests download file in python?

I wrote a python script that downloads files from the internet. However everytime I run the script, it seems like my computer is frozen.

Codes:

response = requests.get(url, stream=True)

with open(local_filename, 'wb') as f:
for chunk in response.iter_content(chunk_size=2048):
if chunk:

f.write(chunk)

f.flush()


What can I do to this to load so my computer doesn't freeze?

Should I allocate a limited amount of ram? Or should I create a thread to do this?

Any advice would be appreciated. ty.

Answer Source

If I were you, I'd increase the chunk size and add some breathing room for the underlying I/O thread, so:

import time

response = requests.get(url, stream=True)

with open(local_filename, 'wb') as f:
    for chunk in response.iter_content(chunk_size=1024*1024):  # lets use 1 meg chunks
        if chunk:  
            f.write(chunk)
            f.flush()
        time.sleep(0.05)  # 50ms delay won't kill anyone

If that doesn't help, you have deeper issues on your system / in your code than this piece.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download