Hossein Hossein - 4 months ago 64
Java Question

Fast transformation of BGR BufferedImage to YUV using FFMpeg

I wanted to transform a TYPE_3BYTE_BGR BufferedImage in Java to yuv using the sws_scale function of FFMpeg through JNI. I first extract the data of my image from the BufferedImage as

byte[] imgData = ((DataBufferByte) myImage.getRaster().getDataBuffer()).getData();

byte[] output = processImage(toSend,0);


Then I pass it to the processImage function which is a native function. The C++ side looks like this:

JNIEXPORT jbyteArray JNICALL Java_jni_JniExample_processData
(JNIEnv *env, jobject obj, jbyteArray data, jint index)
{

jboolean isCopy;
uint8_t *test = (uint8_t *)env->GetPrimitiveArrayCritical(data, &isCopy);
uint8_t *inData[1]; // RGB24 have one plane
inData[0] = test;


SwsContext * ctx = sws_getContext(width,height,AV_PIX_FMT_BGR24, (int)width, (int)width,
AV_PIX_FMT_YUV420P, 0, 0, 0, 0);

int lumaPlaneSize = width *height;
uint8_t *yuv[3];

yuv[0] = new uint8_t[lumaPlaneSize];
yuv[1] = new uint8_t[lumaPlaneSize/4];
yuv[2] = new uint8_t[lumaPlaneSize/4];

int inLinesize[1] = { 3*nvEncoder->width }; // RGB stride
int outLinesize[3] = { 3*width ,3*width ,3*width }; // YUV stride

sws_scale(ctx, inData, inLinesize, 0, height , yuv, outLinesize);


However, after running the code, I get the warning:
[swscaler @ 0x7fb598659480] Warning: data is not aligned! This can lead to a speedloss, everything crashes.
, and everything crashes on the last line. Am I doing things properly in terms of passing the correct arguments to sws_scale? (specially the strides).

Update:
There was a separate bug here:
SwsContext * ctx = sws_getContext(width,height,AV_PIX_FMT_BGR24, (int)width, (int)width,0,NULL,NULL,NULL)
which should be changed to:
SwsContext * ctx = sws_getContext(width,height,AV_PIX_FMT_BGR24, (int)height, (int)width,0,NULL,NULL,NULL)

Answer

The first problem I see - wrong strides for output image:

yuv[0] = new uint8_t[lumaPlaneSize];
yuv[1] = new uint8_t[lumaPlaneSize/4];
yuv[2] = new uint8_t[lumaPlaneSize/4];

int inLinesize[1] = { 3*nvEncoder->width }; // RGB stride
int outLinesize[3] = { 3*width ,3*width ,3*width }; // YUV stride
//                     ^^^^^^^  ^^^^^^^  ^^^^^^^

Allocated planes are not large enough for passed strides. YUV420 uses one byte for each channel, so 3 is redundant and leads to bound violation. due rescaler skips a lot of space when goes to next line. Next, actual chroma width is a half of luma width, so if you want tight-packed luma and chroma planes without gaps at line ends use next:

int outLinesize[3] = { width , width / 2 , width / 2 }; // YUV stride

Allocation sizes remain the same.

Comments