I'm using Python 2.7 and NumPy 1.11.2, as well as the latest versions of dill ( I just did the
pip install dill
with open('dill','wb') as fp:
with open('pickle','wb') as fp:
This ought to be a comment, but I have not enough reputation... My guess is that this is due to the pickle protocol used.
On Python 2, the default protocol is 0 and highest supported protocol is 2. On Python 3, the default protocol is 3 and highest supported protocol is 4 (as of Python 3.6).
Each protocol version improves on the previous one, but protocol 0 is especially slow for largish objects. It should be avoided in most cases, except if you need to be able to read your pickles using extremely old versions of Python. Protocol 2 is already much better.
Now, I suppose dill uses pickle.HIGHEST_PROTOCOL by default, and if that is indeed the case, it would probably be the cause of a good deal of the speed difference. You could try using pickle.HIGHEST_PROTOCOL to see if you get similar performance using dill and standard pickle.
with open('dill', 'wb') as fp: dill.dump(B, fp, protocol=pickle.HIGHEST_PROTOCOL) with open('pickle', 'wb') as fp: pickle.dump(B, fp, protocol=pickle.HIGHEST_PROTOCOL)