Using PyCapsule in Cython

Question

Summary

I need to store a C struct inside a Python object for use in other parts boosted with Cython. I believe a PyCapsule is best suited for this purpose, but the results of my code is not what I am expecting. While the pointer address is correctly returned, the memory appears to have been freed.

Details

I am new to Cython and I am learning to use it to speed up part of my code. For the purpose of asking the question, I have simplified my code and an int is used instead of a struct.

I wrote CythonTest.pyx according to my understanding of the PyCapsule documentation and compiled it with setup.py using the standard command:

python setup.py build_ext --inplace

CythonTest.pyx

#cython: language_level=3



from cpython.pycapsule cimport PyCapsule_New, PyCapsule_IsValid, PyCapsule_GetPointer



class Test:

    def __init__(self):

        cdef int test = 10

        cdef const char *name = "test"

        self.vars = PyCapsule_New(<void *>&test, name, NULL)



        # Print pointer address

        print("{0:x}".format(<unsigned long long>test))



    def peek(self):

        cdef const char *name = "test"

        if not PyCapsule_IsValid(self.vars, name):

            raise ValueError("invalid pointer to parameters")

        cdef int *test = <int *>PyCapsule_GetPointer(self.vars, name)

        print(test[0])



        # Print pointer address

        print("{0:x}".format(<unsigned long long>test))

setup.py

from distutils.core import setup

from Cython.Build import cythonize



setup(ext_modules=cythonize("CythonTest.pyx"))

Then, I run this with the following Python script.

from CythonTest import Test



test = Test()

print(test.vars)

test.peek()

The console prints out the following:

cbde7ebe70

<capsule object "test" at 0x0000027516467930>

0

cbde7ebe70

It appears that the pointer has been successfully stored in a PyCapsule and retrieved as indicated by the identical address. However, 0 is now stored inside the address instead of 10. I am aware that using an int may have caused it to be garbage collected and changed the nature of the problem, but the same issue is observed when using PyMem_Malloc as well.

So the question is: what is the correct way of using PyCapsule?

Environment

Compiler: Visual Studio Express 2015

Cython: 0.26

OS: Windows 10 (64 bit)

Python: 3.5.3

Spyder (IDE): 3.2.3

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Answer 1

test is a local variable (in C) so does not exist beyond the end of the __init__ function, and therefore when you try to access it again in peek the memory has already been used for something else.

You can allocate memory for test of the heap instead, so that the variable persists as long as you need (you need to create a destructor to deallocate it though).

from libc.stdlib cimport malloc, free

# destructor
cdef void free_ptr(object cap):
   # This should probably have some error checking in
   # or at very least clear any errors raised once it's done
   free(PyCapsule_GetPointer(cap,PyCapsule_GetName(cap)))

class Test:
    def __init__(self):
       cdef int* test = malloc(sizeof(int)) 
       test[0] = 10
       cdef const char *name = "test"
       self.vars = PyCapsule_New(<void *>&test, name, &free_ptr)

       # etc

Using PyCapsule in Cython

Summary

Details

Environment

Sign up