stormfield stormfield - 7 days ago 4
Python Question

error with snappy while importing fastparquet in python

I have installed installed the following modules in my EC2 server which already has python (3.6) & anaconda installed :


  • snappy

  • pyarrow

  • s3fs

  • fastparquet



except fastparquet everything else works on importing. When I try to import fastparquet it throws the following error :

[username@ip8 ~]$ conda -V
conda 4.2.13
[username@ip-~]$ python
Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
import fastparquet
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/username/anaconda3/lib/python3.6/site-packages/fastparquet/__init__.py", line 15, in <module>
from .core import read_thrift
File "/home/username/anaconda3/lib/python3.6/site-packages/fastparquet/core.py", line 11, in <module>
from .compression import decompress_data
File "/home/username/anaconda3/lib/python3.6/site-packages/fastparquet/compression.py", line 43, in <module>
compressions['SNAPPY'] = snappy.compress
AttributeError: module 'snappy' has no attribute 'compress'


How do I go about fixing this ?

Answer

Unfortunately, there are multiple things in python-land called "snappy". I believe you may have the wrong one, in which case one of the following conda commands should solve this for you:

conda install python-snappy

or

conda install python-snappy -c conda-forge

where the latter is slightly more recent (releases the GIL which can be important in threaded applications).