Daniel Standage Daniel Standage - 2 months ago 14
Python Question

Bundling C++ extension headers with a Python package source distribution

I'm writing a Cython wrapper to a C++ library that I would like to distribute as a Python package. I've come up with a dummy version of my package that looks like this (full source here).

$ tree
.
├── bogus.pyx
├── inc
│   └── bogus.hpp
├── setup.py
└── src
└── bogus.cpp
$
$ cat inc/bogus.hpp
#ifndef BOGUS
#define BOGUS

class bogus
{
protected:
int data;

public:
bogus();
int get_double(int value);
};

#endif
$
$ cat src/bogus.cpp
#include "bogus.hpp"

bogus::bogus() : data(0)
{

}

int bogus::get_double(int value)
{
data = value * 2;
return data;
}
$ cat bogus.pyx
# distutils: language = c++
# distutils: sources = src/bogus.cpp
# cython: c_string_type=str, c_string_encoding=ascii

cdef extern from 'bogus.hpp':
cdef cppclass bogus:
bogus() except +
int get_double(int value)

cdef class Bogus:
cdef bogus b
def get_double(self, int value):
return self.b.get_double(value)


With the following
setup.py
file, I can can confirm that the library installs correctly with
python setup.py install
and that it works correctly.

from setuptools import setup, Extension
import glob

headers = list(glob.glob('inc/*.hpp'))

bogus = Extension(
'bogus',
sources=['bogus.pyx', 'src/bogus.cpp'],
include_dirs=['inc/'],
language='c++',
extra_compile_args=['--std=c++11', '-Wno-unused-function'],
extra_link_args=['--std=c++11'],
)

setup(
name='bogus',
description='Troubleshooting Python packaging and distribution',
author='Daniel Standage',
ext_modules=[bogus],
install_requires=['cython'],
version='0.1.0'
)


However, when I build a source distribution using
python setup.py sdist build
, the C++ header files are not included and the C++ extension cannot be compiled.

How can I make sure the C++ header files get bundled with the source distribution?!?!

<rant>

Troubleshooting this has uncovered a tremendously convoluted and inconsistent mess of documentation, suggestions, and hacks, none of which have worked for me. Put a
graft
line in
MANIFEST.in
? Nope. The
package_data
or
data_files
options? Nope. Python packaging seems to have improved a lot in the last few years, but it is still nigh impenetrable for those of us that don't live and breathe Python packaging!

</rant>

Answer

Short answer

Put include inc/*.hpp in the MANIFEST.in file.

Long answer

Based on various blog posts and SO threads, I had tried the suggestion of declaring the files in a MANIFEST.in file. Following these instructions, I added a graft inc/ line to MANIFEST.in to include the entire directory. This did not work.

However, replacing this line with include inc/*.hpp did work. Arguably this should have been the first thing I tried, but being unfamiliar with the intricacies and warts of setuptools and distutils, I had no reason to expect that graft wouldn't work.

Comments