Milliams Milliams - 1 year ago 161
C++ Question

Convert a std::vector to a NumPy array without copying data

I have a C++ library which currently has some methods inside which return a

std::vector
defined like

public:
const std::vector<uint32_t>& getValues() const;


I'm currently working on wrapping the whole library for Python using SWIG and this is working well so far.

SWIG wraps this
getValues()
function fine such that it returns a Python tuple. The issue is in my Python-side code I want to convert this to a NumPy array. Of course I can do this by:

my_array = np.array(my_object.getValues(), dtype='uint32')


but this causes all the entries in the original vector to be first copied into a Python tuple by SWIG and then again into a numpy array by me. Since this vector could be very large, I'd rather avoid making these two copies and would like for a way to have SWIG create a numpy.array wrapper around the original vector data in memory.

I've read the documentation for numpy.i but that explicitly mentions that output arrays are not supported since they seem to be working under the assumption of C-style arrays rather than C++ vectors.

numpy.array's underlying data structure is just a C-style array as is a C++ std::vector so I would hope that it is feasible to have then access the same data in memory.

Is there any way to make SWIG return a numpy.array which doesn't copy the original data?

Answer Source

Apparently is is trivial to "cast" a C++ vector to (C) array, see answer on this question: How to convert vector to array C++

Next you can create a numpy array which will use that C array without copying, see discussion here: https://groups.google.com/forum/?fromgroups=#!topic/cython-users/Ubh2nG0_R9g or google for PyArray_SimpleNewFromData.

I wouldn't expect SWIG to do all these for you automatically, instead you should probably write a wrapper for your function getValues yourself, something like getValuesAsNumPyArray.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download