Alexandre Pinheiro Alexandre Pinheiro - 2 months ago 7
Javascript Question

Get frame change in video HTML5

I need detect all frames changes of a video in HTML 5, i tried the follow code, but i can't get all changes, only i can get eight or nine updates per second..

<body>
<canvas id="Canvas" width="800" height="450" style="position:absolute; left:5; top:5">Can't Load Canvas</canvas>
<video id="videoPlay" muted controls>
<source src="assets/soccer.webm" type="video/webm">
<source src="assets/soccer.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
</body>
<script type="text/javascript">
videoPlay.addEventListener('timeupdate', function (){
putOverlay();
});
function putOverlay(){
console.log(videoPlay.currentTime);
console.log("another frame");
}
</script>
</html>


I tried the following code too, but with a timer there is a loss of synchronization between the frame and the value given by the timer, over time..

<body>
<canvas id="LeCanvas" width="800" height="450" style="position:absolute; left:5; top:5">Can't Load Canvas</canvas>
<!-- Video -->
<video id="videoPlay" controls>
<source src="assets/soccer.webm" type="video/webm">
<source src="assets/soccer.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
</body>
<!-- Play Video-->
<script>
//Play Damn Video
var videoPlay = $('#videoPlay')[0];
videoPlay.play(1);
</script>
<!-- Canvas Animation -->
<script>
//Animation
function animate() {
console.log("frame change");
}
setInterval(animate, 1000/29.97);
</script>


Any suggestions for my problem?

Sorry my english

regards
Alex

Answer

As noted in the comments above, timeupdate only fires every 15-250ms, and in practice it appears to be closer to 250ms. It seems as though timeupdate was designed to provide enough accuracy to display the currentTime every second. So your best option is to run a timer that runs at the rate you want and query currentTime on every iteration.

But rather than use setInterval or even setTimeout, you should be using requestAnimationFrame. This will run approximately every 16ms (60fps), but it will be synchronized with the video card refresh rate. If you use setTimeout, you may see flicker or screen tearing on certain devices, especially mobile ones. It also allows the browser to throttle your frame rate when the tab is not visible to save on CPU cycles (and therefore battery usage).

It's also better than setInterval because, if for some reason your render code takes longer than the 30 or 16ms you've allotted, setInterval may pile up calls and cause timing problems, but requestAnimationFrame will skip frames to get you back on track.

Assuming you've used the polyfill as above, the code would look something like this:

var video = document.getElementById('videoPlay'),
    lastTime = -1;
function draw() {
    var time = video.currentTime();
    if (time !== lastTime) {
        console.log('time: ' + time);
        //todo: do your rendering here
        lastTime = time;
    }

    //wait approximately 16ms and run again
    requestAnimationFrame(draw);
}

draw();

The above code has a check on time to make sure you don't draw the same frame twice in a row. You don't really need that, but it will save you a few CPU cycles. But when drawing a video to canvas, the frame is a couple of milliseconds behind what's reported by currentTime. So if you pause the video and then skip around, you will almost definitely be one frame behind. Usually, if the currentTime hasn't changed since my last draw but the video is paused, I'll keep drawing every cycle for another half second or so. Or you could just take the time check out.