user1810087 user1810087 - 3 months ago 41
C++ Question

segmetation fault while avcodec_encode_video2

I have some problems while trying to encode a AVFrame to a packet.

Before reading the whole code, the input stuff is working, i tested it. The output stuff is from an example here. I think there is the problem. but the segmentation fault occurs in the loop near the end.

Here is my reduced code:


void nmain() {

// input stuff
AVFormatContext *formatCtxIn=0;
AVInputFormat *formatIn=0;
AVCodecContext *codecCtxIn=0;
AVCodec *codecIn;
AVPacket *pktIn;

av_register_all();
avdevice_register_all();
avcodec_register_all();


formatIn = av_find_input_format("dshow");

if(!formatIn)
return;


AVDictionary *avoption=0;
av_dict_set(&avoption, "rtbufsize", "1000000000", NULL);

if(avformat_open_input(&formatCtxIn, "video=Integrated Camera", formatIn, &avoption)!=0)
return;

if(avformat_find_stream_info(formatCtxIn, NULL)<0)
return;

codecCtxIn = formatCtxIn->streams[0]->codec;
codecIn = avcodec_find_decoder(codecCtxIn->codec_id);

if(avcodec_open2(codecCtxIn, codecIn, NULL)<0)
return;


// end input stuff
//------------------------------------------------------------------------------
// output stuff

AVOutputFormat *formatOut=0;
AVFormatContext *formatCtxOut=0;
AVStream *streamOut=0;
AVFrame *frame=0;
AVCodec *codecOut=0;
AVPacket *pktOut;

const char *filename = "test.mpeg";

formatOut = av_guess_format(NULL, filename, NULL);
if(!formatOut)
formatOut = av_guess_format("mpeg", NULL, NULL);
if(!formatOut)
return;

formatCtxOut = avformat_alloc_context();
if(!formatCtxOut)
return;

formatCtxOut->oformat = formatOut;

sprintf(formatCtxOut->filename, "%s", filename);

if(formatOut->video_codec != AV_CODEC_ID_NONE) {
AVCodecContext *ctx;

codecOut = avcodec_find_encoder(formatOut->video_codec);
if(!codecOut)
return;

streamOut = avformat_new_stream(formatCtxOut, codecOut);
if(!streamOut)
return;

ctx = streamOut->codec;

ctx->bit_rate = 400000;
ctx->width = 352;
ctx->height = 288;
ctx->time_base.den = 25;
ctx->time_base.num = 1;
ctx->gop_size = 12;
ctx->pix_fmt = AV_PIX_FMT_YUV420P;

if(ctx->codec_id == AV_CODEC_ID_MPEG2VIDEO)
ctx->max_b_frames = 2;
if(ctx->codec_id == AV_CODEC_ID_MPEG1VIDEO)
ctx->mb_decision = 2;


if(formatCtxOut->oformat->flags & AVFMT_GLOBALHEADER)
ctx->flags |= CODEC_FLAG_GLOBAL_HEADER;
}

if(streamOut) {
AVCodecContext *ctx;
ctx = streamOut->codec;

if(avcodec_open2(ctx, codecOut, NULL) < 0)
return;
}

if(!(formatCtxOut->flags & AVFMT_NOFILE))
if(avio_open(&formatCtxOut->pb, filename, AVIO_FLAG_WRITE) < 0)
return;

avformat_write_header(formatCtxOut, NULL);


// doit

pktIn = new AVPacket;
pktOut = new AVPacket;
av_init_packet(pktOut);
pktOut->data = 0;

frame = avcodec_alloc_frame();
if(!frame)
return;

for(;;) {
if(av_read_frame(formatCtxIn, pktIn) >= 0) {
av_dup_packet(pktIn);

int fff;
if(avcodec_decode_video2(codecCtxIn, frame, &fff, pktIn) < 0)
std::cout << "bad frame" << std::endl;

if(!fff)
return; // ok

static int counter=0;
SaveFrame(frame, codecCtxIn->width, codecCtxIn->height, counter++); // work fine

// here a segmentation fault is occured.
if(avcodec_encode_video2(streamOut->codec, pktOut, frame, &fff) < 0)
std::cout << "bad frame" << std::endl;
}
}
}


// only for testing
// add to ensure frame is valid
void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame) {
FILE *pFile;
char szFilename[32];
int y;

// Open file
sprintf(szFilename, "frame%d.ppm", iFrame);
pFile=fopen(szFilename, "wb");
if(pFile==NULL)
return;

// Write header
fprintf(pFile, "P6\n%d %d\n255\n", width, height);

// Write pixel data
for(y=0; y<height; y++)
fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);

// Close file
fclose(pFile);
}


What am i doing wrong?

While debugging i didn't found any problems with the parameters.
streamOut->codec
is filled.
pktOut
is allocated and
frame
is filled with the picture encoded before.
I think the problem is while creating the output codec but watching the example and looking to the doxypages it seems to be correct.

test

The trace route is from QT using msvc11 and framework 5.

I also tried to run with dr. memory and get this:

Error #26: UNADDRESSABLE ACCESS: reading 0x00000000-0x00000004 4 byte(s)
# 0 replace_memcpy [d:\derek\drmemory\withwiki\trunk\drmemory\replace.c:203]
# 1 avcodec-54.dll!ff_dct_quantize_c +0xd463 (0x6a482364 <avcodec-54.dll+0x3c2364>)
# 2 avcodec-54.dll!avcodec_encode_video2+0xb7 (0x6a56a5b8 <avcodec-54.dll+0x4aa5b8>)
# 3 nmain [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:610]
# 4 main [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:182]
Note: @0:00:06.318 in thread 5312
Note: instruction: mov (%edx) -> %ebx


It seems like the reading process while memcpy is going wrong.




Version:



I've forgot to mention the version of libav/ffmpeg i'm using:

libavutil 51. 76.100 / 51. 76.100
libavcodec 54. 67.100 / 54. 67.100
libavformat 54. 33.100 / 54. 33.100
libavdevice 54. 3.100 / 54. 3.100
libavfilter 3. 19.103 / 3. 19.103
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 16.100 / 0. 16.100
libpostproc 52. 1.100 / 52. 1.100





Addendum:



Function
SafeFrame
is copied from tutorial 1.

Answer

Finally i solved my problem.

The problem is (apart from the documentation of libav) avpacket is not a (real) copy of the picture in the packet. it just points to the data of the packet. You have to make a copy, or better you have to let it libav do.

So first i created a new avframe for the output and a buffer on which the output avframe is pointing to.

AVFrame *outpic = avcodec_alloc_frame();
nbytes = avpicture_get_size(codecCtxOut->pix_fmt, codecCtxOut->width, codecCtxOut->height);
uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes);

This buffer is used for the conversion from input to output. Then in the loop i have to fillup the outpic (avframe) with the buffer. I have found in the code that this function is filling up the plane pointers with the buffer. see here

avpicture_fill((AVPicture*)outpic, outbuffer, AV_PIX_FMT_YUV420P, codecCtxOut->width, codecCtxOut->height);

Then i converted the inpic to outpic using sws_scale. But first you have to setup the swscontext.

SwsContext* swsCtx_ = sws_getContext(codecCtxIn->width, codecCtxIn->height,
                                     codecCtxIn->pix_fmt,
                                     codecCtxOut->width, codecCtxOut->height,
                                     codecCtxOut->pix_fmt,
                                     SWS_BICUBIC, NULL, NULL, NULL);

sws_scale(swsCtx_, inpic->data, inpic->linesize, 0, codecCtxIn->height, outpic->data, outpic->linesize);

Then you can encode the outpic into pktout (avpacket for output). But first do free the output packet, otherwise you will get an error and a leak... see here

av_free_packet(pktOut);

if(avcodec_encode_video2(streamOut->codec, pktOut, outpic, &fff) < 0) {
  std::cout << "shit frame" << std::endl;
  continue;
}
// and write it to the file
formatOut->write_packet(formatCtxOut, pktOut);

So now it works (nearly fine) for me. Still a small memory leak, but this i can spot later.