sundar_ima sundar_ima - 22 days ago 8
Python Question

Python copy larger file too slow

I am trying to copy a large file (> 1 GB) from hard disk to usb drive using

shutil.copy
. A simple script depicting what I am trying to do is:-

import shutil
src_file = "source\to\large\file"
dest = "destination\directory"
shutil.copy(src_file, dest)


It takes only 2-3 min on linux. But the same file copy on same file takes more that 10-15 min under Windows. Can somebody explain why and give some solution, preferably using python code?

Update 1

Saved the file as test.pySource file size is 1 GB. Destinantion directory is in USB drive. Calculated file copy time with ptime. Result is here:-

ptime.exe test.py

ptime 1.0 for Win32, Freeware - http://www.
Copyright(C) 2002, Jem Berkes <jberkes@pc-t

=== test.py ===

Execution time: 542.479 s


542.479 s == 9 min. I don't think
shutil.copy
should take 9 min for copying 1 GB file.

Update 2

Health of the USB is good as same script works well under Linux. Calculated time with same file under windows native xcopy.Here is the result.

ptime 1.0 for Win32, Freeware - http://www.pc-tools.net/
Copyright(C) 2002, Jem Berkes <jberkes@pc-tools.net>

=== xcopy F:\test.iso L:\usb\test.iso
1 File(s) copied

Execution time: 128.144 s


128.144 s == 2.13 min. I have 1.7 GB free space even after copying test file.

Answer

Your problem has nothing to do with Python. In fact, the Windows copy process is really poor compared to the Linux system.

You can improve this by using xcopy or robocopy according to this post: Is (Ubuntu) Linux file copying algorithm better than Windows 7?. But in this case, you have to make different calls for Linux and Windows...

import os
import shutil
import sys

source = "source\to\large\file"
target = "destination\directory"

if sys.platform == 'win32':
    os.system('xcopy "%s" "%s"' % (source, target))
else:
    shutil.copy(source, target)

See also: