Aleksey Aleksey - 3 months ago 214
C++ Question

LIVE555 how to use h264 framer class to get nal units for ffmpeg

I'm trying to create a small app which will save frames from inoming h264 stream.
I took a testRTSP programm as example and made several changes in

DummySink::afterGettingFrame
function to decode frames with the help of ffmpeg library.
As I understand from frameSize, my first two frames are SPS units, so I concatenate them with my third frame and then send new big frame to ffmpeg decoder. But that doesnt work. ffmpeg tells me that my first frame is too big for SPS and then it tells me that there is no frame... I dont know what I need to change here.

void DummySink::afterGettingFrame(unsigned frameSize, unsigned numTruncatedBytes,
struct timeval presentationTime, unsigned /*durationInMicroseconds*/)
{
u_int8_t start_code[4] = { 0x00, 0x00, 0x00, 0x01 };
int stCodeLen = 4;

if (frameSize == 50)
{
//add start code
memcpy(bufferWithStartCode, start_code, stCodeLen);
shiftPtr += stCodeLen;
memcpy(bufferWithStartCode + shiftPtr, fReceiveBuffer, frameSize);
shiftPtr += frameSize;
}
else if (frameSize == 4)
{
memcpy(bufferWithStartCode + shiftPtr, fReceiveBuffer, frameSize);
shiftPtr += frameSize;
}
else
{
if (shiftPtr == 0)
{
memcpy(bufferWithStartCode, start_code, stCodeLen);
shiftPtr += stCodeLen;
}
memcpy(bufferWithStartCode + shiftPtr, fReceiveBuffer, frameSize);
avpkt.size = frameSize + shiftPtr;
avpkt.data = bufferWithStartCode;
shiftPtr = 0;
if (!avcodec_send_packet(cContext, &avpkt))
{
envir() << "error sending to decoder";

}
if (!avcodec_receive_frame(cContext, picture))
{
envir() << "error rx from decoder";
}
if (picture)
{
FILE *f;
char buffer[32]; // The filename buffer.
snprintf(buffer, sizeof(char) * 32, "file%i.txt", frame_num);
f = fopen(buffer, "w");
fprintf(f, "P5\n%d %d\n%d\n", fSubsession.videoWidth(), fSubsession.videoHeight(), 255);
for (int i = 0;i < fSubsession.videoHeight();i++)
fwrite(picture->data[0] + i * (picture->linesize[0]), 1, fSubsession.videoWidth(), f);
fclose(f);
}
}

envir() << frameSize << "\n";


frame_num++;

// Then continue, to request the next frame of data:
continuePlaying();

Answer

I've found solution for my problem.

There is function void DummySink::afterGettingFrame(...) In "testRTSP" example of live555. All I was need to do is to assembly my frame everytime when function got frame:

[start_code sps pps start_code frame_data]

frame_data is fReceiveBuffer at this point. start_code is char array [0,0,0,1].

And push new data to ffmpeg decoder:

m_packet.size = frameBufSize + frameSize; // size of assembled frame
m_packet.data = frameBuf; // assembled frame

if (avcodec_send_packet(m_decoderContext, &m_packet) != 0)
{
    envir() << "error in sending packet to decoder" << "\n";
}
if (avcodec_receive_frame(m_decoderContext, pFrame) == 0)

No additional settings for decoderContext! Just init everything like in tutorials (http://dranger.com/ffmpeg/ - this is up-to-date tutorials for ffmpeg c++ library) and you good to go. I used memcpy to unite data in one big array. memcpy(frameBuf, startCode, 4); frameBufSize += 4;

for (int i = 0; i < numSPropRecords; i++)
{
    memcpy(frameBuf + frameBufSize, sPropRecords[i].sPropBytes, sPropRecords[i].sPropLength);
    frameBufSize += sPropRecords[i].sPropLength;
}

memcpy(frameBuf + frameBufSize, startCode, 4);
frameBufSize += 4;
memcpy(frameBuf + frameBufSize, fReceiveBuffer, frameSize);

m_packet.size = frameBufSize + frameSize;
m_packet.data = frameBuf;

You can get sps and pps data from subsession (check "openRTSP" for detailed example or "H264orH264FileSink.h")

spsppsunits = subsession.fmtp_spropparametersets();
sPropRecords = parseSPropParameterSets(spsppsunits, numSPropRecords);