J. Werner J. Werner - 4 months ago 19
Python Question

Call C code with cython and cython-code from c

this is my first question here. Up to now I got a lot of answers by other questions, but now there is no more answer.

The aim of my work is to work with a developed stack of a communication-module (get as .so), which is written in C. I want to combine it with python (cython), because all other software is already written so. After creating and testing the direction from cython to c, I worked the last days for direction from c to cython like here. The stack made an event-driffen call of a function in a c-function and I want to integrate a call of a cython function for logging and further data-handling. But after two days I hang up. It doesn't work, because the initmodulename-functioncall in the c-function raised an error.
So I developed the following minimal example to get it work, cython and c in both directions. It is an extended example to this one.

I have 3 files, the main.c

#include <python3.4/Python.h>
#include "caller.h"
int main() {
Py_Initialize();
initcaller();
call_quack();
Py_Finalize();
return 0;
}


the caller.pyx

from quacker import quack

cdef public void call_quack():
quack()

def run():
cdef extern from "main.c":
int main()
main()


and the quacker.py

def quack():
print("Quack!")


The target is to import caller, start run() as function, which call the c-function and call call_quack() back.

To compile I use (this comes from the main project):

CC="gcc -std=c99" CFLAGS="-DCPLB_VENDOR_EAG_TARGETSYSTEM_SHLIBSIEC104_ARM_LINUX -O2 -fPIC" IFLAGS="-I/usr/include/python3.4 -lpython3.4" python3.4 setup.py build_ext --inplace


with the setup.py

# setup.py file
import sys
import os
import shutil
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext


setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("caller",
sources=["caller.pyx",
],
include_dirs=["/usr/include//python3.4"],
extra_compile_args=["-fopenmp", "-O3"],
extra_link_args=["-DSOME_DEFINE_OPT",
"-L./some/extra/dependency/dir/"]
)
]
)


There is no error during compilation and linking.
But when I start python3.4 and import caller I get the following error

ImportError: /home/rvk/software/test/caller.cpython-34m.so: undefined symbol: initcaller


Can anyone help me with this issue? I never read any example about usind cython and c in both directions! Is it possible?

I already checked the cythonized c-File (caller.c) - there is a initcaller-method, but only for PY_MAJOR_VERSION < 3?!

Thanks a lot in advance

Edit

I got it work by remove the PyInitialze, initcaller and PyFinalize - function calls in the main.c. Maybe this is related to the issue, that I already declared the main.c in the pyx, so its part of the compiled library?! Don't know where the leak is concerning cython user guide

The new main.c:

#include <python3.4/Python.h>
#include "caller.h"
int main() {
call_quack();
return 0;
}


I also integrated it in the main-project. Here the challenge was, that the c-function, which should call the function in the cython-file, is a callback-c-function, so it is necessary to define the function in the cython file with gil

Answer

In caller.pyx there is

def run():
    cdef extern from "main.c":
        int main()
    main()

that causes trouble by including main.c, that in turn includes caller.h which is something unexpected by the code generated by cython. Furthermore, calling that main() defined in main.c might cause trouble when in happens in python interpreter. So cdef extern from "main.c": should be removed.

In case of python 3.x, it is like PyInit_modulename(), not initmodule()

int main() 
{
    Py_Initialize();
    PyInit_caller();
    call_quack();
    Py_Finalize();
    return 0;
}