Ben Kuhn - 3 months ago 22

Python Question

I am installing numpy/scipy/scikit-learn on OS X 10.9.4, and am getting errors about "numpy.dtype size changed, may indicate binary incompatibility".

Here's what I did to construct the repo:

`mkvirtualenv thm`

workon thm

pip install numpy scipy pandas ipython # and some other stuff

cd /path/to/our/repo

# run tests

Here's a traceback excerpt of a relevant warning (turned into an error because we use

`warnings.simplefilter('error')`

`======================================================================`

ERROR: Failure: RuntimeWarning (numpy.dtype size changed, may indicate binary in

compatibility)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/loader.py",

line 414, in loadTestsFromName

addr.filename, addr.module)

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py

", line 47, in importFromPath

return self.importFromDir(dir_path, fqname)

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/nose/importer.py

", line 94, in importFromDir

mod = load_module(part_fqname, fh, filename, desc)

File "/Users/ben/code/thm/alpha/prosper/base/stats/test_auc.py", line 3, in <m

odule>

import sklearn.metrics

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/

__init__.py", line 6, in <module>

from .metrics import (accuracy_score,

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/sklearn/metrics/metrics.py", line 27, in <module>

from scipy.spatial.distance import hamming as sp_hamming

File "/Users/ben/.virtualenvs/thm/lib/python2.7/site-packages/scipy/spatial/__init__.py", line 90, in <module>

from .ckdtree import *

File "__init__.pxd", line 155, in init scipy.spatial.ckdtree (scipy/spatial/ckdtree.c:20570)

RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility

I'm told that this warning is caused by scipy being compiled against a different version of numpy than the one installed. However, I installed them all with pip in what I thought was a pretty standard way, so this shouldn't be a problem, I would think.

Weirdly, although running our entire test suite as a whole (via

`python -m unittest discover`

`python -m unittest <module>`

According to the tests, here's some relevant version info:

`numpy version 1.9.0 (rev 07601a64cdfeb1c0247bde1294ad6380413cab66)`

scipy version 0.14.0 (built against numpy 1.9.0)

sklearn version 0.15.2

pandas version 0.14.1

Happy to provide more info on request!

Answer

How did you build sklearn 0.14.1? Did you build it against the same version of numpy as you did for scipy?

Recent versions of scikit-learn, scipy and numpy have prebuilt-packages. In particular scikit-learn 0.15.2 should be binary compatible with numpy 1.7+. I think the same is true with scipy 0.14.0 but you said you built it yourself from source, which is not what pip should do by default (it should just install the prebuilt whl package).

*Edit*: have you tried to do:

```
pip install -U scipy scikit-learn pandas
```

to make sure that you are using the latest stable versions of the whl for those packages?

*Edit*: The comment below has the actual answer that works and is presumably why this answer was accepted. Namely:

```
pip uninstall -y scipy scikit-learn
pip install --no-binary scipy scikit-learn
```