Alizaybak Alizaybak - 1 year ago 180
Python Question

How can I import an external C function into an IPython Notebook using Cython?

I'd like to import a C function into an IPython notebook using Cython. Currently, I'm trying to replicate the example in the Cython documentation, but I get a compilation error.

My Python code (from an iPython notebook):

import cython
%load_ext Cython

---------------------------------- new cell

cdef extern from "spam.c":
void order_spam(int tons)

My C code:

// spam.c
#include <stdio.h>

static void order_spam(int tons)
printf("Ordered %i tons of spam!\n", tons);

Running this code, I get the following traceback and error message:

CompileError Traceback (most recent call last)
<ipython-input-13-8bb733557977> in <module>()
----> 1 get_ipython().run_cell_magic(u'cython', u'', u'\ncdef extern from "spam.c":\n void order_spam(int tons)')

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_cell_magic(self, magic_name, line, cell)
2118 magic_arg_s = self.var_expand(line, stack_depth)
2119 with self.builtin_trap:
-> 2120 result = fn(magic_arg_s, cell)
2121 return result

<decorator-gen-126> in cython(self, line, cell)

/Users/danielacker/anaconda2/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
191 # but it's overkill for just that one bit of state.
192 def magic_deco(arg):
--> 193 call = lambda f, *a, **k: f(*a, **k)
195 if callable(arg):

/Users/danielacker/anaconda2/lib/python2.7/site-packages/Cython/Build/ in cython(self, line, cell)
276 build_extension.build_temp = os.path.dirname(pyx_file)
277 build_extension.build_lib = lib_dir
--> 278
279 self._code_cache[key] = module_name

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in run(self)
338 # Now actually compile and link everything.
--> 339 self.build_extensions()
341 def check_extensions_list(self, extensions):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extensions(self)
447 for ext in self.extensions:
--> 448 self.build_extension(ext)
450 def build_extension(self, ext):

/Users/danielacker/anaconda2/lib/python2.7/distutils/command/build_ext.pyc in build_extension(self, ext)
496 debug=self.debug,
497 extra_postargs=extra_args,
--> 498 depends=ext.depends)
500 # XXX -- this is a Vile HACK!

/Users/danielacker/anaconda2/lib/python2.7/distutils/ccompiler.pyc in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
572 except KeyError:
573 continue
--> 574 self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
576 # Return *all* object filenames, not just the ones we just built.

/Users/danielacker/anaconda2/lib/python2.7/distutils/unixccompiler.pyc in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
120 extra_postargs)
121 except DistutilsExecError, msg:
--> 122 raise CompileError, msg
124 def create_static_lib(self, objects, output_libname,

CompileError: command 'gcc' failed with exit status 1

I've tried searching Google for this error, but I can't seem to find anything relevant.

Jim Jim
Answer Source

The %cython magic command probaby doesn't seem to do the trick.

The %cython magic command doesn't really do the task here. In order to compile this you need to also supply the *.c source file and that (as far as I'm aware) isn't allowed with %cython. (The source file for it indicates that it only uses the text entered in the cell as a source file.)

Wrap your C functions:

Before presenting a probable solution, let me point out that the .pyx file you have created doesn't actually wrap the C function order_spam automatically. You can have it automatically wrapped if you specify it as cpdef inside the cdef extern block (alternatively you could wrap it on your own outside the cdef extern block, this allows more flexibility).

I'll use a filename of cyspam.pyx for the Cython file:

cdef extern from "spam.c":
    cpdef void order_spam(int tons)

Notice how I've prefixed the function declaration with cpdef, this instructs Cython to automatically wrap the function.

Use a script:

In order to have full control over the compilation process, you usually need to create a script that has all the required sources, include directories et cetera, specified.

Here's how the script should look like:

from distutils.core import setup, Extension
from Cython.Build import cythonize

# you specify the c source file in the sources list
ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
setup(name="C spam", ext_modules = cythonize([ext]))

You can either create a file like this via a simple text editor or via IPython with the %%writefile magic command. The script should of course be placed in the same directory as the cyspam.pyx and spam.c files.

Compile the files:

You can either open a terminal for this or use the %%bash command from IPython, either way does the trick. Issue the following command:

python build_ext --inplace 

--inplace places the generated .so file in the current directory.

After doing these you can easily import the file cyspam in Ipython and just call the wrapped C function:

Rounding up all commands from IPython:

All in all, if you want to do it only from IPython you'll issue the following commands:

In [1]: %%writefile
   ....: from distutils.core import setup, Extension
   ....: from Cython.Build import cythonize
   ....: ext = Extension('cyspam', sources = ['cyspam.pyx', 'spam.c'])
   ....: setup(name="C spam", ext_modules = cythonize([ext]))

In [2]: %%bash
   ...: python build_ext --inplace

In [3]: import cyspam

In [4]: cyspam.order_spam(1000)
You ordered 1000 ammount of spam!

As an alternative you could always create a .pyxbld file that specifies the needed arguments for pyximport.install(). This offers the same level of control but is probably more counter-intuitive for Python users who already have experience with scripts.

See Related:

How can I set Cython compiler flags when using pyximport?

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download