Marko Budinich Marko Budinich -4 years ago 64
C Question

pointers as attributes in extended types cython

I'm a newbie to cython with a bit of knowledge in C and some python experience. Currently I'm trying to learn about Extended Types, but I can not understand what happens with pointers values in the following example (code below the explanation).

As an exercise, I'm implementing a dummy solver. The problem is represented by two numbers 'a' and 'b' and the solution by 's', where s = a*b.

I defined two corresponding C structures,

problem
and
solution
. The problem structure has two int members, 'a' and 'b'; the solution structure has one member 's'. There is one function to initialize the problem structure,
init_problem(problem *p,int a, int b)
. Also, there is one function that take pointers to the structs and returns a solution,
solution solve(problem *p)
. Finally, two other functions print the values (
void dump_problem(problem *p)
and
void dump_solution(solution *s)
). All of this using
cdef
declarations.

Next I used three ways to expose the C code to python: a
def
function
do_things(int a,int b)
which wraps the C functions, and two
cdef class
, one using structs as attributes and the other one using pointers to structs as attributes (
Solver_s
and
Solver_p
, respectively), including wraping methods to print problem and solutions. Solver_s class work as expected; however, when using Solver_p, pointers seems not to be initialized, returning incorrect values (see test_pointers and output sections).

I guess I missing a fine point about pointers and their scope, but I can not understand what is going on. Any help to get this is greatly appreciated. I'm using python 3.5.3 and cython 0.25.2 in OS X 10.11.6 (El Capitan)

P.S: First time asking in SO, so if I'm not being clear I'll glad to clarify!

pointers.pyx

from libc.stdio cimport printf

cdef struct problem:
int a
int b

cdef struct solution:
int s

cdef void init_problem(problem *p,int a, int b):
p.a = a
p.b = b

cdef solution solve(problem *p):
cdef solution s
s.s = p.a * p.b
return(s)

cdef void dump_problem(problem *p):
printf("Problem dump: a = %d,b = %d\n",p.a,p.b)

cdef void dump_solution(solution *s):
printf("Solution dump: s= %d\n",s.s)

def do_things(int a,int b):
cdef problem p
init_problem(&p,a,b)
cdef solution s = solve(&p)
dump_problem(&p)
dump_solution(&s)

cdef class Solver_s: #Structs as attributes of Solver
cdef problem p
cdef solution s
def __cinit__(self,int a,int b):
print("\tInside Solver_s __cinit__")
init_problem(&self.p,a,b)
dump_problem(&self.p)
self.s = solve(&self.p)
dump_solution(&self.s)
print("\tGetting out of Solver_s __cinit__")

def show_problem(self):
dump_problem(&self.p)

def show_solution(self):
dump_solution(&self.s)

cdef class Solver_p: #Pointers to structs as attributes
cdef problem *pptr
cdef solution *sptr

def __cinit__(self,int a, int b):
print("\tInside Solver_p __cinit__")
cdef problem p
self.pptr = &p
cdef solution s
self.sptr = &s
init_problem(self.pptr,a,b)
dump_problem(self.pptr) #It shows right values
self.sptr[0] = solve(self.pptr)
dump_solution(self.sptr) #It shows right values
print("\tGetting out of Solver_p __cinit__")


def show_problem(self):
dump_problem(self.pptr)

def show_solution(self):
dump_solution(self.sptr)


test_pointers.py

import pyximport; pyximport.install()
import pointers

print("\tSolving as a function")
pointers.do_things(2,3)

print("\tSolving as a Extended Type, structs as attributes")
sol_s = pointers.Solver_s(4,5)
print("\t###Problem definition unsing Solver_s methods###")
sol_s.show_problem()
print("\t###Solution definition using Solver_s methods###")
sol_s.show_solution()


print("\tSolving as a Extended Type, pointers to structs as attributes")
sol_p = pointers.Solver_p(6,7)
print("\t###Problem definition unsing Solver_p methods###")
print("\t###Gives weird values###")
sol_p.show_problem()
print("\t###Solution definition using Solver_p methods###")
print("\t###Gives weird values###")
sol_p.show_solution()


Output

Solving as a function
Problem dump: a = 2,b = 3
Solution dump: s= 6
Solving as a Extended Type, structs as attributes
Inside Solver_s __cinit__
Problem dump: a = 4,b = 5
Solution dump: s= 20
Getting out of Solver_s __cinit__
###Problem definition unsing Solver_s methods###
Problem dump: a = 4,b = 5
###Solution definition using Solver_s methods###
Solution dump: s= 20
Solving as a Extended Type, pointers to structs as attributes
Inside Solver_p __cinit__
Problem dump: a = 6,b = 7
Solution dump: s= 42
Getting out of Solver_p __cinit__
###Problem definition unsing Solver_p methods###
###Gives weird values###
Problem dump: a = 1,b = 0
###Solution definition using Solver_p methods###
###Gives weird values###
Solution dump: s= 185295816

Answer Source

In Solver_p.__cinit__, p and s are local variables which only exist for the duration of the call to __cinit__. The Solver_p instance lasts beyond the call, and thus for most of its lifetime the pointers are to invalid data.

The solution is to allocate heap memory instead:

# at the top
from libc.stdlib cimport malloc, free

cdef class Solver_p:
    # ....
    def __cinit__(self,...):
       self.pptr = <problem*>malloc(sizeof(problem))
       self.sptr = <solution*>malloc(sizeof(solution))
       # ...
    def __dealloc__(self):
       free(self.pptr)
       free(self.sptr)
       # ...

You need to be careful to make sure that all the memory you allocate is appropriately freed.


My advice is that if you don't understand how to use pointers correctly in C then you should not be using them in Cython.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download