Kevin Kevin - 6 months ago 10
Objective-C Question

How can I convert a RTCI420Frame object to a texture?

I have a webrtc app, but I want more control over the video view. So I'm busy with implementing my own renderer. I got to the point where I start receiving frames via this callback:

renderer:(RTCVideoRenderer *)renderer didReceiveFrame:(RTCI420Frame *)frame


It passes a
RTCI420Frame
as parameter. I need to convert the image data in
frame
to a texture somehow. I can't figure out how. I don't think it's possible with
GLKTextureLoader
(that's what I've been using to load textures from disk up untill now).

How do I get the image data out of
frame
and create a texture out of it?

Answer

Here is my current class to handle everything WebRTC-related in my apps.

These classes are meant as a starting point for native WebRTC on iOS. Simply extend a view controller on BWRTCViewController and set a delegate. Now you can start testing right away, without having to worry about implementing the entire call sequence yourself. You only have to worry about signaling.

// your call view controller .h
#import <BWRTCViewController.h>

@interface CallViewController : BWRTCViewController <BWRTCViewControllerDelegate>
@end


// your call view controller .m
- (void)viewDidLoad {
    [super viewDidLoad];
    if (/*this is the caller*/) {
        [super callerSequence];

        // wait until callee is ready to receive your offer, then call:
        [super startNegotiating];
    } else {
        /*callee side doesn't have to do a thing*/
    }
}

// received a remote sdp
- (void)receivedSdp {
    [super receivedSessionDescription:/*your sdp description*/
                             withType:/*your sdp type*/];
}

// received a remote ice candidate
- (void)receivedIce {
    [super receivedIceCandidate:/*your ice candidate*/
                         sdpMid:/*your ice sdpMid*/
                  sdpMLineIndex:/*your ice sdpMLineIndex*/];
}

// got a local sdp
- (void) sendSessionDescription:(NSString *)sessionDescription_
                       withType:(NSString *)type_ {
    // use your signaling interface to send the sdp to the remote peer
}

// got a local ice candidate
- (void) sendICECandidate:(NSString *)candidate_
                   sdpMid:(NSString *)sdpMid_
            sdpMLineIndex:(NSInteger)sdpMLineIndex_ {
    // use your signaling interface to send the ice to the remote peer
}