Dezmond Goff - 1 year ago 67

Linux Question

I'm trying to write a Cython module that calculates pairwise distances as part of a larger class of locality sensitive hashes. Instead of writing code for each type and each distance metric, I am attempting to create one cdef function that takes various extension types that inherit from Metric:

`cdef class Metric:`

def __init__(self):

pass

cdef class Euclidean(Metric):

cdef numeric c_evaluate(self, numeric[:] x, numeric[:] y, int dims):

....

cdef numeric[:,:] pairwise(numeric[:] x, numeric[:] y, Metric func, bint symmetric):

...

dm[i,j] = func.c_evaluate(x,y,dims)

...

To access this function from Python:

`def py_pairwise(numeric[:,:] x, numeric[:,:] y, str func, bint symmetric = 1, **kwargs):`

cdef Metric mfunc = to_Metric(func, **kwargs)

return pairwise(x, y, mfunc, symmetric)

However I keep getting the error that "c_distance.[Metric] object has no attribute 'c_evaluate'". I'm wondering if the c_evaluate method isn't accessible because the class object is created in python code through the python function to_Metric, though I thought def and cdef functions were supposed to be able to call each other freely within a Cython module. The method works if I change c_evaluate to a cpdef method, but I'm not sure if this fixes the problem by allowing the cdef object to pass through python to cython or simply uses the slower python method. Any suggestions (I'm also not at my home computer so I don't have all the code right now. Will update later/on request)?

Edit: That typo isn't in the original functions (there could still be others):

`ctypedef fused floating:`

float

double

cdef class Euclidean(Metric):

cdef public floating c_evaluate(self, floating[:] x, floating[:] y, int dims):

cdef int i

cdef floating tmp, d = 0

for i in range(dims):

tmp = x[i]-y[i]

d += tmp*tmp

return sqrt(d)

#@cython.boundscheck(False)

#@cython.wraparound(False)

def py_pairwise(numeric[:,::1] x, numeric[:,::1] y,str metric, bint symmetric,**kwargs):

cdef Metric func = to_Metric(metric,**kwargs)

return pairwise(x,y,func,symmetric)

cdef numeric[:,::1] pairwise(numeric[:,::1] x,numeric[:,::1] y, Metric met, bint symmetric):#

cdef int n,m,k,i,j

n = x.shape[0]

m = y.shape[0]

dims = x.shape[1]

if numeric in floating:

mdtype = np.float

else:

mdtype = np.int

#mdtype = np.float

cdef numeric[:,::1] dm = (np.empty((n,m),dtype = mdtype)).fill(0)

if symmetric:

interval = lambda i,n,m: range(i+1,m)

else:

interval = lambda i,n,m: range(m)

for i in range(n):

for j in interval(i,n,m):

dm[i,j] = met.c_evaluate(x[i,:],y[j,:],dims)

return np.asarray(dm)

Also, to_Metric:

`def to_Metric(str m, **kwargs):`

if len(kwargs) == 0:

if m == 'euclidean':

met = Euclidean()

elif m in {'cos','cosine'}:

met = Cosine()

elif m in {'hamming','matching'}:

met = Hamming()

else:

raise ValueError('Unrecognized metric {}'.format('\''+m+'\''))

else:

if m in {'pnorm','p-norm'}:

met = Pnorm(kwargs['p'])

elif m == 'maximal':

met = Maximal(kwargs['m1'],kwargs['m2'],kwargs['sep'])

else:

raise ValueError('Unrecognized metric {}'.format('\''+m+'\''))

return met

Answer Source

The issue is that `c_evaluate`

is associated with the class `Euclidean`

and because of this can **only** be used with objects that are known to be of type `Euclidean`

. However, in `pairwise`

you declare the type of `met`

to be `Metric`

.

Because you declared `c_evaluate`

function as `cdef`

it can only be found at compile time. If you want the `c_evaluate`

to be found at runtime like a standard Python function, you should declare it as `def`

.

If you need the function to be found at compile time (which makes calling it quicker) then you should either make `c_evaluate`

be a function of the `Metric`

object, or you should make `pairwise`

only take a `Euclidean`

object.