paul_h paul_h - 18 days ago 5
Linux Question

Writing to a USB extremely HDD slow, but not with 'sponge'

This takes 2 mins 9 secs, and I don't know why:

time openssl rand 10000000 > foo


This takes 0.8 seconds:

time openssl rand 10000000 | sponge > foo


System: Ubuntu 16.04.1, HP Stream Mini 010, 4TB USB 3.0 Seagate 'Backup Plus' hard drive.

Why is it 161x faster with sponge?

More Testing...


  1. The same hard drive reformatted to HFS+ on a Mac, and the same two tests yields 1.09 secs and 0.9 secs respectively.

  2. Faster work Mac & Parallels & Ubuntu 14.04: 0.4 and 0.2 seconds respectively.


Answer

Usbmount defaults to a 'sync' mode which makes some categories of write slow:

https://www.google.com/search?q=usbmount+sync+slow

Editing the /etc/usbmount/usbmount.conf file and taking out the 'sync' property makes it as fast as you'd expect. Specifically the sponge and non-sponge tests having the same time.