Jono Jono - 6 months ago 21
Python Question

Cannot import zmq in python (install issue)

I can't seem to install pyzmq on my macbook (OSX 10.9.1)

First call was to run:

sudo pip install pyzmq


There was an error that libzmq couldn't be found, and it appeared to try and compile the bundled version:

jono@air:~ $ sudo pip install pyzmq
Password:
Downloading/unpacking pyzmq
Downloading pyzmq-14.0.1.tar.gz (867kB): 867kB downloaded
Running setup.py egg_info for package pyzmq

no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'docs/gh-pages'
warning: no directories found matching 'bundled/uuid'
warning: no previously-included files found matching 'bundled/uuid/Makefile*'
warning: no previously-included files found matching 'bundled/zeromq/src/Makefile*'
warning: no previously-included files found matching 'bundled/zeromq/src/platform.hpp'
warning: no previously-included files found matching 'setup.cfg'
warning: no previously-included files found matching 'zmq/libzmq*'
warning: no previously-included files matching '__pycache__/*' found anywhere in distribution
warning: no previously-included files matching '.deps/*' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
warning: no previously-included files matching '.mailmap' found anywhere in distribution
Installing collected packages: pyzmq
Running setup.py install for pyzmq
Did not find libzmq via pkg-config:
Package libzmq was not found in the pkg-config search path.
Perhaps you should add the directory containing `libzmq.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libzmq' found

xcrun clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -c build/temp.macosx-10.8-x86_64-2.7/scratch/check_sys_un.c -o build/temp.macosx-10.8-x86_64-2.7/scratch/check_sys_un.o
xcrun clang -undefined dynamic_lookup build/temp.macosx-10.8-x86_64-2.7/scratch/check_sys_un.o -o build/temp.macosx-10.8-x86_64-2.7/scratch/check_sys_un
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir:
************************************************
xcrun clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.macosx-10.8-x86_64-2.7/scratch/vers.c -o build/temp.macosx-10.8-x86_64-2.7/scratch/vers.o
build/temp.macosx-10.8-x86_64-2.7/scratch/vers.c:4:10: fatal error: 'zmq.h' file not found
#include "zmq.h"
^
1 error generated.

error: command 'xcrun' failed with exit status 1

Failed with default libzmq, trying again with /usr/local
Configure: Autodetecting ZMQ settings...
Custom ZMQ dir: /usr/local
************************************************
xcrun clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.macosx-10.8-x86_64-2.7/scratch/vers.c -o build/temp.macosx-10.8-x86_64-2.7/scratch/vers.o
build/temp.macosx-10.8-x86_64-2.7/scratch/vers.c:4:10: fatal error: 'zmq.h' file not found
#include "zmq.h"
^
1 error generated.

error: command 'xcrun' failed with exit status 1

Warning: Failed to build or run libzmq detection test.

If you expected pyzmq to link against an installed libzmq, please check to make sure:

* You have a C compiler installed
* A development version of Python is installed (including headers)
* A development version of ZMQ >= 2.1.4 is installed (including headers)
* If ZMQ is not in a default location, supply the argument --zmq=<path>
* If you did recently install ZMQ to a default location,
try rebuilding the ld cache with `sudo ldconfig`
or specify zmq's location with `--zmq=/usr/local`

If you expected to get a binary install (egg), we have those for
current Pythons on OS X and Windows. These can be installed with
easy_install, but PIP DOES NOT SUPPORT EGGS.

You can skip all this detection/waiting nonsense if you know
you want pyzmq to bundle libzmq as an extension by passing:

`--zmq=bundled`

I will now try to build libzmq as a Python extension
unless you interrupt me (^C) in the next 10 seconds...

************************************************
1...
Using bundled libzmq
already have bundled/zeromq
attempting ./configure to generate platform.hpp
Warning: failed to configure libzmq:
/bin/sh: ./configure: No such file or directory

staging platform.hpp from: buildutils/include_darwin
************************************************
cc -c /tmp/crypto_box_keypairpmO9ft.c -o build/temp.macosx-10.8-x86_64-2.7/tmp/crypto_box_keypairpmO9ft.o
/tmp/crypto_box_keypairpmO9ft.c:1:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main (int argc, char **argv) {
^~~~
/tmp/crypto_box_keypairpmO9ft.c:2:5: warning: implicit declaration of function 'crypto_box_keypair' is invalid in C99 [-Wimplicit-function-declaration]
crypto_box_keypair();
^
2 warnings generated.
cc build/temp.macosx-10.8-x86_64-2.7/tmp/crypto_box_keypairpmO9ft.o -lsodium -o build/temp.macosx-10.8-x86_64-2.7/a.out
ld: library not found for -lsodium
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Warning: libsodium not found, zmq.CURVE security will be unavailable
************************************************
building 'zmq.libzmq' extension
xcrun clang -fno-strict-aliasing -fno-common -dynamic -I/usr/local/include -I/usr/local/opt/sqlite/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ibundled/zeromq/include -Ibundled -I/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c buildutils/initlibzmq.c -o build/temp.macosx-10.8-x86_64-2.7/buildutils/initlibzmq.o
...
... (lots more compiling)
...
xcrun clang -bundle -undefined dynamic_lookup -L/usr/local/lib -L/usr/local/opt/sqlite/lib -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk build/temp.macosx-10.8-x86_64-2.7/zmq/backend/cython/utils.o -o build/lib.macosx-10.8-x86_64-2.7/zmq/backend/cython/utils.so

no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'docs/gh-pages'
warning: no directories found matching 'bundled/uuid'
warning: no previously-included files found matching 'bundled/uuid/Makefile*'
warning: no previously-included files found matching 'bundled/zeromq/src/Makefile*'
warning: no previously-included files found matching 'setup.cfg'
warning: no previously-included files found matching 'zmq/libzmq*'
warning: no previously-included files matching '__pycache__/*' found anywhere in distribution
warning: no previously-included files matching '.deps/*' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
warning: no previously-included files matching '.git*' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
warning: no previously-included files matching '.mailmap' found anywhere in distribution
Successfully installed pyzmq
Cleaning up...


This didn't work:

jono@air:~ $ python
Python 2.7.3 (default, Mar 15 2013, 22:53:11)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/site-packages/zmq/__init__.py", line 35, in <module>
_libzmq = ctypes.CDLL(bundled[0], mode=ctypes.RTLD_GLOBAL)
File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/python2.7/site-packages/zmq/libzmq.so, 10): Symbol not found: __ZNSiD0Ev
Referenced from: /usr/local/lib/python2.7/site-packages/zmq/libzmq.so
Expected in: flat namespace
in /usr/local/lib/python2.7/site-packages/zmq/libzmq.so
>>>


So I interpreted the errors in the install process as not having libzmq. Following the instructions at https://github.com/zeromq/pyzmq/wiki/Building-and-Installing-PyZMQ, I uninstalled it, and tried:

jono@air:~ $ brew install zeromq
==> Downloading http://download.zeromq.org/zeromq-3.2.3.tar.gz
######################################################################## 100.0%
==> Patching
patching file tests/test_disconnect_inproc.cpp
==> ./configure --prefix=/usr/local/Cellar/zeromq/3.2.3
==> make
==> make install
==> Caveats
To install the zmq gem on 10.6 with the system Ruby on a 64-bit machine,
you may need to do:

ARCHFLAGS="-arch x86_64" gem install zmq -- --with-zmq-dir=/usr/local/opt/zeromq
==> Summary

Answer

After much effort, I went back to basics and managed to get it working by:

Building from source with:

brew install zeromq
...
python setup.py configure --zmq=/usr/local/Cellar/zeromq/4.0.3/
...
sudo python setup.py install
...

or

sudo port install 
...

I have no idea why the standard methods didn't work, but at least I have pyzmq working :)

Comments