Craig McQueen Craig McQueen - 5 months ago 35
Python Question

How should I structure a Python package that contains Cython code

I'd like to make a Python package containing some Cython code. I've got the the Cython code working nicely. However, now I want to know how best to package it.

For most people who just want to install the package, I'd like to include the

file that Cython creates, and arrange for
to compile that to produce the module. Then the user doesn't need Cython installed in order to install the package.

But for people who may want to modify the package, I'd also like to provide the Cython
files, and somehow also allow for
to build them using Cython (so those users would need Cython installed).

How should I structure the files in the package to cater for both these scenarios?

The Cython documentation gives a little guidance. But it doesn't say how to make a single
that handles both the with/without Cython cases.


I've done this myself now, in a Python package simplerandom (BitBucket repo - EDIT: now github) (I don't expect this to be a popular package, but it was a good chance to learn Cython).

This method relies on the fact that building a .pyx file with Cython.Distutils.build_ext (at least with Cython version 0.14) always seems to create a .c file in the same directory as the source .pyx file.

Here is a cut-down version of which I hope shows the essentials:

from distutils.core import setup
from distutils.extension import Extension

    from Cython.Distutils import build_ext
except ImportError:
    use_cython = False
    use_cython = True

cmdclass = { }
ext_modules = [ ]

if use_cython:
    ext_modules += [
        Extension("mypackage.mycythonmodule", [ "cython/mycythonmodule.pyx" ]),
    cmdclass.update({ 'build_ext': build_ext })
    ext_modules += [
        Extension("mypackage.mycythonmodule", [ "cython/mycythonmodule.c" ]),

    cmdclass = cmdclass,

I also edited to ensure that mycythonmodule.c is included in a source distribution (a source distribution that is created with python sdist):

recursive-include cython *

I don't commit mycythonmodule.c to version control 'trunk' (or 'default' for Mercurial). When I make a release, I need to remember to do a python build_ext first, to ensure that mycythonmodule.c is present and up-to-date for the source code distribution. I also make a release branch, and commit the C file into the branch. That way I have a historical record of the C file that was distributed with that release.