Vijay Vijay - 4 months ago 17
Python Question

Compiling numpy with OpenBLAS integration

I am trying to install

numpy
with
OpenBLAS
, however I am at loss as to how the
site.cfg
file needs to be written.

When the installation procedure was followed the installation completed without errors, however there is performance degradation on increasing the number of threads used by OpenBLAS from 1 (controlled by the environment variable OMP_NUM_THREADS).

I am not sure if the OpenBLAS integration has been perfect. Could any one provide a
site.cfg
file to achieve the same.

P.S.: OpenBLAS integration in other toolkits like Theano, which is based on Python, provides substantial performance boost on increasing the number of threads, on the same machine.

Answer

I just compiled numpy inside a virtualenv with OpenBLAS integration, and it seems to be working OK.

This was my process:

  1. Compile OpenBLAS:

    ~$ git clone https://github.com/xianyi/OpenBLAS
    ~$ cd OpenBLAS && make FC=gfortran
    ~$ sudo make PREFIX=/opt/OpenBLAS install
    

    If you don't have sudo rights you could set PREFIX= to a directory you have write privileges to (just modify the corresponding steps below accordingly).

  2. Make sure that the directory containing libopenblas.so is in your shared library search path.

    • To do this locally, you could edit your ~/.bashrc file to contain the line

      export LD_LIBRARY_PATH=/opt/OpenBLAS/lib:$LD_LIBRARY_PATH
      

      The LD_LIBRARY_PATH environment variable will be updated when you start a new terminal session (use $ source ~/.bashrc to force an update within the same session).

    • Another option that will work for multiple users is to create a .conf file in /etc/ld.so.conf.d/ containing the line /opt/OpenBLAS/lib, e.g.:

      ~$ sudo sh -c "echo '/opt/OpenBLAS/lib' > /etc/ld.so.conf.d/openblas.conf"
      

    Once you are done with either option, run

    ~$ sudo ldconfig
    
  3. Grab the numpy source code:

    ~$ git clone https://github.com/numpy/numpy
    ~$ cd numpy
    
  4. Copy site.cfg.example to site.cfg and edit the copy:

    ~$ cp site.cfg.example site.cfg
    ~$ nano site.cfg
    

    Uncomment these lines:

    ....
    [openblas]
    libraries = openblas
    library_dirs = /opt/OpenBLAS/lib
    include_dirs = /opt/OpenBLAS/include
    ....
    
  5. Check configuration, build, install (optionally in a virtualenv)

    ~$ python setup.py config
    

    The output should look something like this:

    ...
    openblas_info:
      FOUND:
        libraries = ['openblas', 'openblas']
        library_dirs = ['/opt/OpenBLAS/lib']
        language = c
        define_macros = [('HAVE_CBLAS', None)]
    
      FOUND:
        libraries = ['openblas', 'openblas']
        library_dirs = ['/opt/OpenBLAS/lib']
        language = c
        define_macros = [('HAVE_CBLAS', None)]
    ...
    

    Installing with pip is preferred over using python setup.py install, since it keeps track of package metadata and allows you to easily uninstall or upgrade numpy.

    ~$ pip install .
    
  6. Optional: you can use this script to test performance for different thread counts.

    ~$ OMP_NUM_THREADS=1 python build/test_numpy.py
    
    version: 1.10.0.dev0+8e026a2
    maxint:  9223372036854775807
    
    BLAS info:
     * libraries ['openblas', 'openblas']
     * library_dirs ['/opt/OpenBLAS/lib']
     * define_macros [('HAVE_CBLAS', None)]
     * language c
    
    dot: 0.099796795845 sec
    
    ~$ OMP_NUM_THREADS=8 python build/test_numpy.py
    
    version: 1.10.0.dev0+8e026a2
    maxint:  9223372036854775807
    
    BLAS info:
     * libraries ['openblas', 'openblas']
     * library_dirs ['/opt/OpenBLAS/lib']
     * define_macros [('HAVE_CBLAS', None)]
     * language c
    
    dot: 0.0439578056335 sec
    

There seems to be a noticeable improvement in performance for higher thread counts. However, I haven't tested this very systematically, and it's likely that for smaller matrices the additional overhead would outweigh the performance benefit from a higher thread count.