Alizaybak Alizaybak - 6 months ago 51
Python Question

How can I import an external C function into an IPython Notebook using Cython?

I'd like to import a C function into an IPython notebook using Cython. Currently, I'm trying to replicate the example in the Cython documentation, but I get a compilation error.

My Python code (from an iPython notebook):

import cython
%load_ext Cython


---------------------------------- new cell

%%cython
cdef extern from "spam.c":
void order_spam(int tons)


My C code:

// spam.c
#include <stdio.h>

static void order_spam(int tons)
{
printf("Ordered %i tons of spam!\n", tons);
}


Running this code, I get the following traceback and error message:

CompileError Traceback (most recent call last)
<ipython-input-13-8bb733557977> in <module>()
----> 1 get_ipython().run_cell_magic(u'cython', u'', u'\ncdef extern from "spam.c":\n void order_spam(int tons)')

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell)
2118 magic_arg_s = self.var_expand(line, stack_depth)
2119 with self.builtin_trap:
-> 2120 result = fn(magic_arg_s, cell)
2121 return result
2122

<decorator-gen-126> in cython(self, line, cell)

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
191 # but it's overkill for just that one bit of state.
192 def magic_deco(arg):
--> 193 call = lambda f, *a, **k: f(*a, **k)
194
195 if callable(arg):

/Users/danielacker/anaconda2/lib/python2.7/site-packages/Cython/Build/IpythonMagic.py in cython(self, line, cell)
276 build_extension.build_temp = os.path.dirname(pyx_file)
277 build_extension.build_lib = lib_dir
--> 278 build_extension.run()
279 self._code_cache[key] = module_name
280

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in run(self)
337
338 # Now actually compile and link everything.
--> 339 self.build_extensions()
340
341 def check_extensions_list(self, extensions):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extensions(self)
446
447 for ext in self.extensions:
--> 448 self.build_extension(ext)
449
450 def build_extension(self, ext):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extension(self, ext)
496 debug=self.debug,
497 extra_postargs=extra_args,
--> 498 depends=ext.depends)
499
500 # XXX -- this is a Vile HACK!

/Users/danielacker/anaconda2/lib/python2.7/distutils/ccompiler.pyc in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
572 except KeyError:
573 continue
--> 574 self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
575
576 # Return *all* object filenames, not just the ones we just built.

/Users/danielacker/anaconda2/lib/python2.7/distutils/unixccompiler.pyc in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
120 extra_postargs)
121 except DistutilsExecError, msg:
--> 122 raise CompileError, msg
123
124 def create_static_lib(self, objects, output_libname,

CompileError: command 'gcc' failed with exit status 1


I've tried searching Google for this error, but I can't seem to find anything relevant.

Jim Jim
Answer

The %cython magic command probaby doesn't seem to do the trick.

The %cython magic command doesn't really do the task here. In order to compile this you need to also supply the *.c source file and that (as far as I'm aware) isn't allowed with %cython. (The source file for it indicates that it only uses the text entered in the cell as a source file.)

Wrap your C functions:

Before presenting a probable solution, let me point out that the .pyx file you have created doesn't actually wrap the C function order_spam automatically. You can have it automatically wrapped if you specify it as cpdef inside the cdef extern block (alternatively you could wrap it on your own outside the cdef extern block, this allows more flexibility).

I'll use a filename of cyspam.pyx for the Cython file:

cdef extern from "spam.c":
    cpdef void order_spam(int tons)

Notice how I've prefixed the function declaration with cpdef, this instructs Cython to automatically wrap the function.

Use a setup.py script:

In order to have full control over the compilation process, you usually need to create a setup.py script that has all the required sources, include directories et cetera, specified.

Here's how the setup.py script should look like:

from distutils.core import setup, Extension
from Cython.Build import cythonize

# you specify the c source file in the sources list
ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
setup(name="C spam", ext_modules = cythonize([ext]))

You can either create a file like this via a simple text editor or via IPython with the %%writefile magic command. The setup.py script should of course be placed in the same directory as the cyspam.pyx and spam.c files.

Compile the files:

You can either open a terminal for this or use the %%bash command from IPython, either way does the trick. Issue the following command:

python setup.py build_ext --inplace 

--inplace places the generated .so file in the current directory.

After doing these you can easily import the file cyspam in Ipython and just call the wrapped C function:

Rounding up all commands from IPython:

All in all, if you want to do it only from IPython you'll issue the following commands:

In [1]: %%writefile setup.py
   ....: from distutils.core import setup, Extension
   ....: from Cython.Build import cythonize
   ....: ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
   ....: setup(name="C spam", ext_modules = cythonize([ext]))

In [2]: %%bash
   ...: python setup.py build_ext --inplace

In [3]: import cyspam

In [4]: cyspam.order_spam(1000)
You ordered 1000 ammount of spam!

As an alternative you could always create a .pyxbld file that specifies the needed arguments for pyximport.install(). This offers the same level of control but is probably more counter-intuitive for Python users who already have experience with setup.py scripts.

See Related:

How can I set Cython compiler flags when using pyximport?