Mohammed Rashid Chowdhury Mohammed Rashid Chowdhury - 5 months ago 65
C++ Question

Using Opencv Cuda functions from python

For one of my course projects, I need to use the OpenCVs GPU libraries. I am working on an existing code where OpenCV python is used and my work is to find a way to access the OpenCV Cuda libraries as right now there are no accessible Python bindings to OpenCV's various CUDA modules.

Two of the functions that i extremely need right now are

cuda::warpPerspective
and
cv::cuda::DescriptorMatcher::knnMatch()
.

i tried to implement the
warpPerspective
by following what @ostrumvulpes suggested in Accessing OpenCV CUDA Functions from Python (No PyCUDA) and it is working perfectly. Right now i am stuck in
DescriptorMatcher::knnMatch()
. To be more precise, i need to use the brute-force descriptor matchers
knnmatch
function (CUDA). i searched online for example written in C++, so that i get an initial idea how to convert it via cython to make it work.

Most of the examples that i found is like following:

Ptr<cuda::DescriptorMatcher> matcher =
cuda::DescriptorMatcher::createBFMatcher();
vector< vector< DMatch> > matches;
matcher->knnMatch(descriptors_object_Gpu, descriptors_scene_Gpu, matches, 2);


To implement these three lines i first added what i thought was necessary in the .pxd file. My pxd file looks like the following:

GpuWrapper.pxd

from libcpp cimport bool
from cpython.ref cimport PyObject
from libcpp.vector cimport vector

# References PyObject to OpenCV object conversion code borrowed from OpenCV's own conversion file, cv2.cpp
cdef extern from 'pyopencv_converter.cpp':
#mrc689 April 20,2017
void import_array()
cdef PyObject* pyopencv_from(const Mat& m)
cdef bool pyopencv_to(PyObject* o, Mat& m)

cdef extern from 'opencv2/imgproc.hpp' namespace 'cv':
cdef enum InterpolationFlags:
INTER_NEAREST = 0
cdef enum ColorConversionCodes:
COLOR_BGR2GRAY

cdef extern from 'opencv2/core/core.hpp':
cdef int CV_8UC1
cdef int CV_32FC1

cdef extern from 'opencv2/core/core.hpp' namespace 'cv':
cdef cppclass Size_[T]:
Size_() except +
Size_(T width, T height) except +
T width
T height
ctypedef Size_[int] Size2i
ctypedef Size2i Size
cdef cppclass Scalar[T]:
Scalar() except +
Scalar(T v0) except +

cdef extern from 'opencv2/core/core.hpp' namespace 'cv':
cdef cppclass Mat:
Mat() except +
void create(int, int, int) except +
void* data
int rows
int cols

#added to test the Algorithm class inside core.hpp on May5th 12.52 AM.
cdef cppclass Algorithm:
Algorithm() except +

cdef extern from 'opencv2/core/base.hpp' namespace 'cv':
cdef enum NormTypes:
NORM_INF= 1,
NORM_L1= 2,
NORM_L2= 4,
NORM_HAMMING= 6,
NORM_HAMMING2= 7,

cdef extern from 'opencv2/core/cuda.hpp' namespace 'cv::cuda':
cdef cppclass GpuMat:
GpuMat() except +
void upload(Mat arr) except +
void download(Mat dst) const
cdef cppclass Stream:
Stream() except +

cdef extern from 'opencv2/core/types.hpp' namespace 'cv':
cdef cppclass DMatch:
DMatch() except +
float distance
int imgIdx
int queryIdx
int trainIdx

cdef extern from 'opencv2/core/cvstd.hpp' namespace 'cv':
cdef cppclass Ptr[T]:
T element_type
Ptr() except +


cdef extern from 'opencv2/cudafeatures2d.hpp' namespace 'cv::cuda':
cdef cppclass DescriptorMatcher:
@staticmethod
Ptr[DescriptorMatcher] createBFMatcher(int normType) except+
#Expected to see error here
void knnMatch(GpuMat queryDescriptors, GpuMat trainDescriptors, vector[vector[DMatch]] &matches,int k)

cdef extern from 'opencv2/cudawarping.hpp' namespace 'cv::cuda':
cdef void warpPerspective(GpuMat src, GpuMat dst, Mat M, Size dsize, int flags, int borderMode, Scalar borderValue, Stream& stream)
# Function using default values
cdef void warpPerspective(GpuMat src, GpuMat dst, Mat M, Size dsize, int flags)


and my pyx looks like this:

GpuWrapper.pyx

import numpy as np # Import Python functions, attributes, submodules of numpy
cimport numpy as np # Import numpy C/C++ API

def match_feature(np.ndarray[np.float32_t, ndim=3] _src,
np.ndarray[np.float32_t, ndim=2] _M):

np.import_array()
# Create GPU/device InputArray for src
cdef Mat src_mat
cdef GpuMat src_gpu
pyopencv_to(<PyObject*> _src, src_mat)
src_gpu.upload(src_mat)

cdef Mat src_mat_2
cdef GpuMat src_gpu_2
pyopencv_to(<PyObject*> _M, src_mat_2)
src_gpu_2.upload(src_mat_2)

cdef Ptr[DescriptorMatcher] matcher= Ptr()
matcher = DescriptorMatcher.createBFMatcher(4)
cdef vector[vector[DMatch]] matches
matcher.knnMatch(src_gpu,src_gpu_2,matches,2)
print("no problem so far")


When i tried to compile it i got an error which says
'Ptr[DescriptorMatcher]' has no attribute 'knnMatch'
.

Now as far i understood, the Ptr is a shared pointer of type
DescriptorMatcher
, So there must be something wrong in my way of defining Ptr from .pxd file.

i just dont know how to fix it. I will really appreciate if someone can help me solving it.

Answer Source

I don't think you're using Ptr correctly (it needs dereferencing in Cython before you can get to knnMatch).

A good place to look at how to make Ptr is the C++ standard library wrappers built into Cython which wrap the similar classes std::shared_ptr and std::unique_ptr.

You don't want to do the line T element_type since this isn't interpreted as a typedef (like in the OpenCV headers) - it's interpreted as having a member called element_type of type T (which doesn't exist).

You possibly want to set up some of the other constructors for Ptr. As it stands you've only wrapped the default empty one. (It doesn't look like it matters for your code since you get it from a factory function).

Most importantly, you also want to set up the dereference operator (operator*). That's probably all you need to implement for it to work:

cdef cppclass Ptr[T]:
    Ptr() except + 
    Ptr(Ptr*) except +
    T& operator* () # probably no exceptions

To use it you use the cython.operator.dereference:

# at the top
from cython.operator cimport dereference

# later
dereference(matcher).knnMatch(src_gpu,src_gpu_2,matches,2)

(I haven't looked at the rest of the code in detail so I have no comment on whether it is right)