Streaming ffmpeg with nodejs on web sockets to html

The last months we (with Sergi Lario) had some fun implementing a GUI to control a media server done by playmodes. We used web technologies and had the need to stream the content of the media server to our app. This article gave us the inspiration about how to easily stream video from ffmpeg to HTML5 canvas.

In resume, data gets streamed from ffmpeg through and is received frame by frame as Motion JPEG. Then we encoded the images as Base64, sent them over web sockets to the frontend and draw them to HTML.

Here follows a short description of the essential parts:

Nodejs starts a child process and receives the stream as Motion JPEG:

1

2

3

4

5

6

7

8

9

10

11

var ffmpeg = require('child_process').spawn("ffmpeg", [

"-re",

"-y",

"-i",

"udp://"+this.streamIP+":"+this.streamPort,

"-preset",

"ultrafast",

"-f",

"mjpeg",

"pipe:1"

]);

Actually this is the code to make it work on Linux (ubuntu 14.04 with ffmpeg 2.6), but I couldn’t make it run on Windows and really would apreciate any tips for that.

Data is received frame by frame and sent over web sockets (socket.io) to the frontend. Here it would be nice to sharpen images using ImageMagick, but I didn’t have success with that part:

1

2

3

4

ffmpeg.stdout.on('data', function (data) {

var frame = new Buffer(data).toString('base64');

io.sockets.emit('canvas',frame);

});

Finally the data is received on the frontend and drawn as CSS background to HTML. Alternativly it can be painted directly into the canvas:

1

2

3

4

5

this.socket.on('canvas', function(data) {

try {

$('#videostream').css('background', 'transparent url(data:image/jpeg;base64,'+data+') top left / 100% 100% no-repeat');

} catch(e){ }

});

You find the test program here on github, these are the steps you have to follow to reproduce the described example:

1. Set up ffmpeg and make a test stream, for example using the webcam:

ffmpeg -f video4linux2 -i /dev/video0 -f h264 -vcodec libx264 -r 10 -s 160x144 -g 0 -b 800000 udp://127.0.0.1:1234

2. Adapt your directory and server IP and port and run node server program from server directory by typing:

node streamserver.js

3. Open browser, type your IP and port like http://localhost:8080/ and you should see your video stream embeded into the website.

Tags: canvas, ffmpeg, HTML5, javascript, nodejs, socket.io, stream, video