Synthetix Synthetix -4 years ago 120
C Question

How to map optical flow field (float) to pixel data (char) for image warping?

I've been playing with the optical flow functions in OpenCV and am stuck. I've successfully generated X and Y optical flow fields/maps using the Farneback method, but I don't know how to apply this to the input image coordinates to warp the images. The resulting X and Y fields are of 32bit float type (0-1.0), but how does this translate to the coordinates of the input and output images? For example, 1.0 of what? The width of the image? The difference between the two?

Plus, I'm not sure what my loop would look like to apply the transform/warp. I've done plenty of loops to change color, but the pixels always remain in the same location. Moving pixels around is new territory for me!

Update: I got this to work, but the resulting image is messy:

//make a float copy of 8 bit grayscale source image
IplImage *src_img = cvCreateImage(img_sz, IPL_DEPTH_32F, 1);
cvConvertScale(input_img,src_img,1/255.0); //convert 8 bit to float

//create destination image
IplImage *dst_img = cvCreateImage(img_sz, IPL_DEPTH_32F, 1);

for(y = 0; y < flow->height; y++){

//grab flow maps for X and Y
float* vx = (float*)(velx->imageData + velx->widthStep*y);
float* vy = (float*)(vely->imageData + vely->widthStep*y);

//coords for source and dest image
const float *srcpx = (const float*)(src_img->imageData+(src_img->widthStep*y));
float *dstpx = (float*)(dst_img->imageData+(dst_img->widthStep*y));

for(x=0; x < flow->width; x++)
{
int newx = x+(vx[x]);
int newy = (int)(vy[x])*flow->width;
dstpx[newx+newy] = srcpx[x];
}
}


I could not get this to work. The output was just garbled noise:

cvRemap(src_img,dst_img,velx,vely,CV_INTER_CUBIC,cvScalarAll(0));

Answer Source

The flow vectors are velocity values. If the pixel in image 1 at position (x, y) has the flow vector (vx, vy) it is estimated to be at position (x+vx, y+vy) (so the values aren't really in the [0, 1] range - they can be bigger, and be negative too). Easiest way to do the warping is to create floating point images with those values (x+vx for the x direction, similar for y), and then use cv::remap.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download