Python Question
Embed python / numpy in C++
I am trying to use python 3 (with numpy) in my C++ application.
This entails sending a C++ array to python, performing calculations and then retrieving the result in C++.
To do this I based myself on the code that was discussed here:
https://codereview.stackexchange.com/questions/92266/sending-a-c-array-to-python-numpy-and-back/92353#92353
and also here:
Sending a C++ array to Python and back (Extending C++ with Numpy).
While the example from the code review post basically works I am having troubles with the return values when I modified the python and C++ script: when I am trying to return a variable that was created in python the result is a vector of nan instead of the intended computations.
My guess is that the object somehow goes out of scope but I can't fix this problem.
I use the following python script in a file called mymodule.py:
import numpy
def array_tutorial(a):
print("array_tutorial - python")
print(a)
print("")
firstRow = a[0,:]
#beta = numpy.array([[10,20,30],[10,20,30],[10,20,30]])
#firstRow = beta[0,:]
return firstRow
def myfunction():
beta = numpy.array([[1,2,3],[1,2,3],[1,2,3]])
print("myfunction - python")
print(beta)
print("")
firstRow = beta[0,:]
return firstRow
My C++ code is in the file numpy_cpp.cpp which is a slightly changed and simplified version of the accepted answer to the code review post.
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include <stdio.h>
#include <iostream>
#include <stdlib.h>
#include <Python.h>
#include "numpy/arrayobject.h"
int main(int argc, char* argv[])
{
setenv("PYTHONPATH", ".", 0);
Py_Initialize();
import_array();
// Build the 2D array in C++
const int SIZE = 3;
npy_intp dims[2]{SIZE, SIZE};
const int ND = 2;
long double(*c_arr)[SIZE]{ new long double[SIZE][SIZE] };
for (int i = 0; i < SIZE; i++){
for (int j = 0; j < SIZE; j++){
c_arr[i][j] = i + j;}
}
// Convert it to a NumPy array.
PyObject *pArray = PyArray_SimpleNewFromData(ND, dims, NPY_LONGDOUBLE, reinterpret_cast<void*>(c_arr));
// import mymodule
const char *module_name = "mymodule";
PyObject *pName = PyUnicode_FromString(module_name);
PyObject *pModule = PyImport_Import(pName);
Py_DECREF(pName);
// import function
const char *func_name = "array_tutorial";
PyObject *pFunc = PyObject_GetAttrString(pModule, func_name);
PyObject *pReturn = PyObject_CallFunctionObjArgs(pFunc, pArray, NULL);
PyArrayObject *np_ret = reinterpret_cast<PyArrayObject*>(pReturn);
// Convert back to C++ array and print.
int len = PyArray_SHAPE(np_ret)[0];
long double* c_out;
c_out = reinterpret_cast<long double*>(PyArray_DATA(np_ret));
std::cout << "Printing output array - C++" << std::endl;
for (int i = 0; i < len; i++){
std::cout << c_out[i] << ' ';
}
std::cout << std::endl << std::endl;
// import function without arguments
const char *func_name2 = "myfunction";
PyObject *pFunc2 = PyObject_GetAttrString(pModule, func_name2);
PyObject *pReturn2 = PyObject_CallFunctionObjArgs(pFunc2, NULL);
PyArrayObject *np_ret2 = reinterpret_cast<PyArrayObject*>(pReturn2);
// convert back to C++ array and print
int len2 = PyArray_SHAPE(np_ret2)[0];
long double* c_out2;
c_out2 = reinterpret_cast<long double*>(PyArray_DATA(np_ret2));
std::cout << "Printing output array 2 - C++" << std::endl;
for (int i = 0; i < len2; i++){
std::cout << c_out2[i] << ' ';
}
std::cout << std::endl << std::endl;
Py_Finalize();
return 0;
}
Compared to the accepted answer I had to add
setenv("PYTHONPATH", ".", 0);
to make sure the python script is found, I added the second function call for the function "myfunction" that has no input arguments and I removed some of the error handling.
I am on Ubuntu 16.10 and and use
g++ -Wall numpy_cpp.cpp -I/usr/include/python3.5m/ -lpython3.5m
to compile and link (which goes fine except for one warning by import_array()). I am targeting python 3.
Running the program however gives the following console output:
array_tutorial - python
[[ 0.0 1.0 2.0]
[ 1.0 2.0 3.0]
[ 2.0 3.0 4.0]]
Printing output array - C++
0 1 2
myfunction - python
[[1 2 3]
[1 2 3]
[1 2 3]]
Printing output array 2 - C++
nan nan nan
It is the last output that gives me trouble, where python returns the first row of an numpy array that was set up in the python script.
From the python print statements it seems that the numpy array is fine (not nan), but once it is referred to C++ things go sideways.
If I uncomment the two lines above the return statement in the array_tutorial function I get the same (disappointing) result for the first function call.
My question is therefore how to get the correct values in C++ without the objects going (presumably) out of scope?
I apologize for the lengthy post, and thank you in advance for any help!
EDIT: As pointed out by lomereiter below the numpy arrays in python should be set up keeping the data types in mind.
This solves the problem.
A better python script that outputs the data type of the received array and specifies the data type of the declared array would be:
import numpy
def array_tutorial(a):
print("array_tutorial - python")
print(a)
print(numpy.dtype(a[0,0]))
print("")
firstRow = a[0,:]
#beta = numpy.array([[10,20,30],[10,20,30],[10,20,30]],dtype=numpy.float128)
#firstRow = beta[0,:]
return firstRow
def myfunction():
beta = numpy.array([[1,2,3],[1,2,3],[1,2,3]],dtype=numpy.float128)
print("myfunction - python")
print(beta)
print("")
firstRow = beta[0,:]
return firstRow
Recommended for you: Get network issues from WhatsUp Gold. Not end users.
Answer Source
You should specify dtype when you create the array in Python code. You are casting to long double in C++ code while the dtype is deduced to be int64 (on 64-bit platforms)
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download