emson emson - 28 days ago 12
Python Question

File not found error when pip installing python-lzo

I'm trying to install

python-lzo
on mac OS Sierra. I have used
brew
to install the c
lzo
library:

brew update
brew install lzo


I then try and install
python-lzo
using a virtual environment and
pip
:

$ virtualenv ./env
$ . ./env/bin/activate
$ pip install python-lzo


Which gives me the following output:

Collecting python-lzo
Using cached python-lzo-1.11.tar.gz
Building wheels for collected packages: python-lzo
Running setup.py bdist_wheel for python-lzo: started
Running setup.py bdist_wheel for python-lzo: finished with status 'error'
Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/tmp_ueOY4pip-wheel- --python-tag cp27:
/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
warnings.warn(msg)
running bdist_wheel
running build
running build_ext
building 'lzo' extension
creating build
creating build/temp.macosx-10.11-x86_64-2.7
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>
^
1 error generated.
error: command 'clang' failed with exit status 1

----------------------------------------
Running setup.py clean for python-lzo
Failed to build python-lzo
Installing collected packages: python-lzo
Running setup.py install for python-lzo: started
Running setup.py install for python-lzo: finished with status 'error'
Complete output from command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-wDRYjx/python-lzo/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-8RUWxo-record/install-record.txt --single-version-externally-managed --compile:
/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
warnings.warn(msg)
running install
running build
running build_ext
building 'lzo' extension
creating build
creating build/temp.macosx-10.11-x86_64-2.7
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
lzomodule.c:35:10: fatal error: 'lzo1x.h' file not found
#include <lzo1x.h>
^
1 error generated.
error: command 'clang' failed with exit status 1

----------------------------------------


The
lzo1x.h
file does exist here:
/usr/local/opt/lzo/include/lzo/
or with the
brew
command:

$ ls $(brew --prefix lzo)/include/lzo/


I have then tried to set the
CFLAGS
envronment variables, but still have the same error:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install


And in desperation:

$ export EXTRA_CFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_CXXFLAGS=-I/usr/local/opt/lzo/include/lzo && export EXTRA_LDFLAGS=-L/usr/local/opt/lzo/lib && pip install python-lzo


Still the same error message, any ideas?

-- Updated--

Also tried passing the variables directly into
pip
:

pip install --global-option='build_ext' --global-option='-I/usr/local/opt' --global-option='-I/usr/local/opt/lzo/include/lzo' --global-option='-L/usr/local/opt/lzo/lib' python-lzo


This time I get a slightly different error:

cmdoptions.check_install_build_global(options)
Collecting python-lzo
Using cached python-lzo-1.11.tar.gz
Skipping bdist_wheel for python-lzo, due to binaries being disabled for it.
Installing collected packages: python-lzo
Running setup.py install for python-lzo ... error
Complete output from command /Users/myuser/dir/devel/lzo_example/env/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-build-14vN3V/python-lzo/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext -I/usr/local/opt -I/usr/local/opt/lzo/include/lzo -L/usr/local/opt/lzo/lib install --record /var/folders/38/v_r4bjvx7dz626x16nh6jcn00000gn/T/pip-Gq5pjs-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/emson/Dropbox/devel/aws/coursenut_bigdata/lzo_example/env/bin/../include/site/python2.7/python-lzo:
/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:251: UserWarning: 'licence' distribution option is deprecated; use 'license'
warnings.warn(msg)
running build_ext
building 'lzo' extension
creating build
creating build/temp.macosx-10.11-x86_64-2.7
clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include/lzo -I/usr/local/opt/lzo/include/lzo -I/usr/local/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c lzomodule.c -o build/temp.macosx-10.11-x86_64-2.7/lzomodule.o
In file included from lzomodule.c:35:
/usr/local/opt/lzo/include/lzo/lzo1x.h:33:10: fatal error: 'lzo/lzoconf.h' file not found
#include <lzo/lzoconf.h>
^
1 error generated.
error: command 'clang' failed with exit status 1


Many thanks

-- SUCCESS ---

Big thanks @daphtdazz your solution (see below) worked!
I have modified it slightly to use the
brew --cellar
command to get the location of the
lzo
files:


export C_INCLUDE_PATH=$(brew --cellar lzo)/2.09/include/lzo:$(brew --cellar lzo)/2.09/include/
export LIBRARY_PATH=/usr/local/lib
pip install python-lzo

Answer

I got the same results as you, and eventually this worked for me:

export C_INCLUDE_PATH=/usr/local/Cellar/lzo/2.09/include/lzo:/usr/local/Cellar/lzo/2.09/include 
export LIBRARY_PATH=/usr/local/lib 
pip install python-lzo

(Obviously you may need to tweak those depending on what version of the lzo brew distribution you have installed.)

Which answered the question for me, but I don't really understand why those aren't configured correctly...

I have the Xcode developer tools installed though as well, and I think they've conflicted with brew and pip before, so maybe that's it. I'll keep having a look and if I work it out I'll update this answer.

Edit

Having a look at the setup.py file for python-lzo it seems there's a bit of weirdness going on, in particular I'm concerned that on Windows both include paths are being added, which suggests the lzo source distribution isn't very well arranged. But there doesn't seem to be a designed way of achieving this on macOS with a brew install, so I think the include part of it at least is best solved with the extra environment variable.