jdi jdi - 1 year ago 135
C++ Question

project structure for wrapping many c++ classes in cython to a single shared object

I have found partial answers between the docs, mailing lists, and this question here, but I wanted to get a more direct answer addressing my specifics...

I'm learning cython by trying to wrap small parts, little by little, of a library that I am already using that is currently wrapped in boost::python. I have contributed a tiny bit to this boost wrapper, and am using it as a c++ reference, while at the same time I am using ZeroMQ Python bindings as a cython reference.

My question is about project structure. The current boost version of this lib compiles to a single

, and that is my goal. I quickly found out that you cannot directly compile multiple
modules to a single
. I then started going down the route of defining the
's in
files, and their corresponding python exported implementation classes in
, and was trying to include them into a single
for compilation. While it worked at first, once I wrote a little more I hit problems with conflicting multiple definitions because of the
includes in different places.

I would love to hear a proper organizational approach that addresses the following questions and goals:

  • Naming the public classes the same as the
    (I am doing this now by having the cppclass in a different named
    and using the imported namespace to handle the similar names, ala Using cimport to resolve naming conflicts)

  • Single
    as the compiled output (acceptable approach?)

  • Do I use the
    multi-include approach into the main
    for that alone, or should that main
    contain anything else beyond just holding the includes?

  • Where to centrally define constants that will be exported in python?

  • Is there a preferred folder structure? Right now I have everything in a big
    directory beneath my
    . It gets confusing seeing so many
    pxi, pxd, pyx

  • Are
    completely unnecessary now? If not, do I need to use a cython-style ifndef guard to handle the multiple inclusions between different modules?

  • I know the ZeroMQ python bindings build multiple modules and use the package approach by including them through
    . Is that really the proper approach with cython?

For reference, the project I am practicing to re-wrap is PyOpenNI (openni). The pattern this boost project takes is to collect the common objects in one place, and then define a 1-to-1 header definition with the source, and then there is a huge wrapper that collects all of the definitions into the single location. And also the added custom exception handling and utilities.

jdi jdi
Answer Source

While waiting for a definitive answer, I kept playing around with organizing my code. The including of pyx files into a single pyx for compilation has been working so far.

My setup.py is simple like:

ext_modules = [
        include_dirs=['src/', '/usr/include/ni'],

The main openni.pyx looks like:

include "constants.pyx"
include "exceptions.pyx"
include "context.pyx"

I have a common libopenni.pxd to provide declaration-only externs to the rest of the modules.

I name my cppclass declarations a different pxd name than the pyx class definitions to avoid name collision:


cdef extern from "XnCppWrapper.h" namespace "xn":
    cdef cppclass Context:


from libopenni cimport *
from xncontext cimport Context as c_Context 

cdef class Context:
    cdef c_Context *handle   
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download