Video Streaming with ASP.NET SignalR and HTML5
I have already talked about SignalR in this blog. I think it is one of the most interesting technologies that Microsoft put out recently, not because it is something substantially new – AJAX, long polling and server-sent events have been around for quite some time -, but because of how easy and extensible they made it.
Most of the examples of SignalR usually are about chat. You know that I have been digging into HTML5 lately, and I already posted on media acquisition using HTML5’s getUserMedia API. This time, however, I’m going to talk about video streaming!
I wrote a simple ASP.NET Web Forms control that leverages SignalR to build a real-time communication channel, and this channel transmits images as Data URIs. The source of the feed comes from getUserMedia and is therefore available in all modern browsers, except, alas, one that shall remain unnamed. Any browser, however, can be used to display the streaming feed.
So, the architecture is quite simple:
-
One SignalR Hub for the communication channel;
-
One ASP.NET Web Forms control, that renders an HTML5 VIDEO tag that is used to display the video being acquired, on any compatible browser.
And that’s it. Of course, the page where we are hosting the control needs to have references to the required JavaScript files (currently jQuery >= 1.6.4 and jQuery-SignalR 2.0.3), these are included automatically when you add the Nuget package.
Here are some nice screenshots of my now famous home wall, where one of the browser instances, Chrome, is broadcasting to Firefox, Opera and Safari. Unfortunately, IE is not supported.
So, all we need is a control declaration on an ASP.NET Web Forms page, which could look like this:
1: <web:VideoStreaming runat="server" ID="video" ClientIDMode="Static" Width="300px" Height="300px" Interval="100" Source="True" ScalingMode="TargetSize" StreamingMode="Target" TargetClientID="received" OnStreamed="onStreamed" Style="border: solid 1px black" />
The non-ordinary properties that the VideoStreaming control supports are:
-
Interval: the rate in milliseconds that the control broadcasts that the video is being broadcast; do not set it to less than 200, from my experience, it will not work well;
-
OnStreamed: the name of a JavaScript callback function to call which will receive the streamed image as a Data URI when the StreamingMode is set to Event, not used in any of the other modes;
-
Source: if the control is set as a streaming source, or just as a receiver (default);
Here is an example of a JavaScript function that serves as the callback function for the Event streaming mode:
1: function onStreamed(imageUrl, imageWidth, imageHeight)
2: {
3: //for StreamingMode="Event", draw an image on an existing canvas
4: //the onload event is for cross-browser compatibility
5: //in this example, we are using the canvas width and height
6: var canvas = document.getElementById('received');
7: var ctx = canvas.getContext('2d');
8: var img = new Image();
9: img.src = imageUrl;
10: img.onload = function()
11: {
12: ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
13: }
14: }
And a simple CANVAS element for Target:
1: <canvas id="received" width="300" height="300" style="border: solid 1px black"></canvas>
The reason for the two main streaming modes, Target and Event are flexibility: with Target, you directly draw the streamed picture on a CANVAS or IMG (it will detect automatically what to use), which needs to be already present, and with Event, you can do your own custom processing.
The source for the VideoStreaming control is this:
1: [assembly: OwinStartup(typeof(VideoStreaming))]
2:
3: public class VideoStreaming : WebControl
4: {
5: public const String Url = "/videostreaming";
6:
7: public VideoStreaming() : base("video")
8: {
9: this.Interval = 100;
10: this.OnStreamed = String.Empty;
11: this.ScalingMode = VideoScalingMode.None;
12: this.Source = false;
13: this.StreamingMode = VideoStreamingMode.Event;
14: this.TargetClientID = String.Empty;
15: }
16:
17: [DefaultValue(false)]
18: public Boolean Source
19: {
20: get;
21: set;
22: }
23:
24: [DefaultValue(VideoStreamingMode.Event)]
25: public VideoStreamingMode StreamingMode
26: {
27: get;
28: set;
29: }
30:
31: [DefaultValue(100)]
32: public Int32 Interval
33: {
34: get;
35: set;
36: }
37:
38: [DefaultValue("")]
39: public String OnStreamed
40: {
41: get;
42: set;
43: }
44:
45: [DefaultValue("")]
46: public String TargetClientID
47: {
48: get;
49: set;
50: }
51:
52: [DefaultValue(VideoScalingMode.None)]
53: public VideoScalingMode ScalingMode
54: {
55: get;
56: set;
57: }
58:
59: public static void Configuration(IAppBuilder app)
60: {
61: app.MapSignalR(Url, new HubConfiguration());
62: }
63:
64: protected override void OnLoad(EventArgs e)
65: {
66: var sm = ScriptManager.GetCurrent(this.Page);
67: var streamingAction = String.Empty;
68: var size = (this.ScalingMode == VideoScalingMode.OriginalSize) ? ", imageWidth, imageHeight" : (this.ScalingMode == VideoScalingMode.TargetSize) ? ", canvas.width, canvas.height" : (this.ScalingMode == VideoScalingMode.ControlSize) ? String.Format(", {0}, {1}", this.Width.Value, this.Height.Value) : String.Empty;
69:
70: switch (this.StreamingMode)
71: {
72: case VideoStreamingMode.Event:
73: if (String.IsNullOrWhiteSpace(this.OnStreamed) == true)
74: {
75: throw (new InvalidOperationException("OnStreamed cannot be empty when using streaming mode Event"));
76: }
77: streamingAction = String.Format("{0}(imageUrl, imageWidth, imageHeight)", this.OnStreamed);
78: break;
79:
80: case VideoStreamingMode.Target:
81: if (String.IsNullOrWhiteSpace(this.TargetClientID) == true)
82: {
83: throw (new InvalidOperationException("TargetClientID cannot be empty when using streaming mode Target"));
84: }
85: streamingAction = String.Format("var canvas = document.getElementById('{0}'); if (canvas.tagName == 'CANVAS') {{ var ctx = canvas.getContext('2d'); var img = new Image(); }} else if (canvas.tagName == 'IMG') {{ var img = canvas; }}; img.src = imageUrl; img.width = imageWidth; img.height = imageHeight; if (typeof(ctx) != 'undefined') {{ img.onload = function() {{\n ctx.drawImage(img, 0, 0{1}); \n}} }};", this.TargetClientID, size);
86: break;
87:
88: case VideoStreamingMode.Window:
89: streamingAction = String.Format("if (typeof(window.videoWindow) == 'undefined') {{ window.videoWindow = window.open(imageUrl, '_blank', 'width=imageWidth,height=imageHeight'); }} else {{ window.videoWindow.location.href = imageUrl; }};");
90: break;
91: }
92:
93: var initScript = String.Format("\ndocument.getElementById('{0}').connection = $.hubConnection('{1}', {{ useDefaultPath: false }}); document.getElementById('{0}').proxy = document.getElementById('{0}').connection.createHubProxy('videoStreamingHub');\n", this.ClientID, Url);
94: var startStreamScript = String.Format("\ndocument.getElementById('{0}').startStream = function(){{\n var video = document.getElementById('{0}'); video.proxy.on('send', function(imageUrl, imageWidth, imageHeight) {{\n {2} \n}}); video.connection.start().done(function() {{\n if ((true == {3}) && (video.paused == true) && (video.src == '')) {{\n navigator.getUserMedia = (navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia); navigator.getUserMedia({{ video: true, audio: false }}, function (stream) {{\n video.src = window.URL.createObjectURL(stream); \n}}, function (error) {{\n debugger; \n}}); \n}}; if (video.intervalId) {{\n window.cancelAnimationFrame(video.intervalId); \n}}; var fn = function(time) {{\nif (time >= {1} && video.intervalId != 0) {{ var canvas = document.createElement('canvas'); var context = canvas.getContext('2d'); context.drawImage(video, 0, 0, canvas.width, canvas.height); var picture = canvas.toDataURL(); video.proxy.invoke('send', picture, video.videoWidth, video.videoHeight); }}; window.requestAnimationFrame(fn); \n}}; if (true == {3}) {{ video.intervalId = window.requestAnimationFrame(fn); }}; video.play(); \n}}) }}\n", this.ClientID, this.Interval, streamingAction, this.Source.ToString().ToLower());
95: var stopStreamScript = String.Format("\ndocument.getElementById('{0}').stopStream = function(){{ var video = document.getElementById('{0}'); if (video.intervalId) {{ window.cancelAnimationFrame(video.intervalId); }}; video.intervalId = 0; video.pause(); video.connection.stop(); }};\n", this.ClientID);
96: var script = String.Concat(initScript, startStreamScript, stopStreamScript);
97:
98: if (sm != null)
99: {
100: this.Page.ClientScript.RegisterStartupScript(this.GetType(), String.Concat(Url, this.ClientID), String.Format("Sys.WebForms.PageRequestManager.getInstance().add_pageLoaded(function() {{ {0} }});\n", script), true);
101: }
102: else
103: {
104: this.Page.ClientScript.RegisterStartupScript(this.GetType(), String.Concat(Url, this.ClientID), script, true);
105: }
106:
107: if (this.Width != Unit.Empty)
108: {
109: this.Attributes.Add(HtmlTextWriterAttribute.Width.ToString().ToLower(), this.Width.ToString());
110: }
111:
112: if (this.Height != Unit.Empty)
113: {
114: this.Attributes.Add(HtmlTextWriterAttribute.Height.ToString().ToLower(), this.Height.ToString());
115: }
116:
117: this.Attributes.Remove("autoplay");
118: this.Attributes.Remove("controls");
119: this.Attributes.Remove("crossorigin");
120: this.Attributes.Remove("loop");
121: this.Attributes.Remove("mediagroup");
122: this.Attributes.Remove("muted");
123: this.Attributes.Remove("poster");
124: this.Attributes.Remove("preload");
125: this.Attributes.Remove("src");
126:
127: base.OnLoad(e);
128: }
129: }
In essence, it is very simple, although it has some nasty inline JavaScript. Basically, what it does is:
-
Renders a VIDEO tag element with the specified Width and Height;
-
Register a SignalR hub;
-
Attaches some JavaScript methods to the generated HTML VIDEO tag (startStream, stopStream);
-
Removes any eventual VIDEO-related attribute that may be present on the control declaration;
-
It uses requestAnimationFrame for periodically (every Interval in milliseconds) broadcasting the image obtained from getUserMedia and drawn on the rendered VIDEO tag as a Data URI to the SignalR hub.
Of course, we now need the hub code, which is as simple as it could be, just a method that takes the image as a Data URI and its original dimensions and broadcasts it to the world:
1: public class VideoStreamingHub : Hub
2: {
3: public void Send(String imageUrl, Int32 imageWidth, Int32 imageHeight)
4: {
5: this.Clients.All.Send(imageUrl, imageWidth, imageHeight);
6: }
7: }
And finally the simple enumerations used by the VideoStreaming control:
1: public enum VideoScalingMode
2: {
3: /// <summary>
4: /// No scaling is performed.
5: /// </summary>
6: None,
7:
8: /// <summary>
9: /// Use the original size, coming from getUserMedia.
10: /// </summary>
11: OriginalSize,
12:
13: /// <summary>
14: /// Use the target CANVAS size.
15: /// </summary>
16: TargetSize,
17:
18: /// <summary>
19: /// Use the VideoStreaming control size.
20: /// </summary>
21: ControlSize
22: }
23:
24: public enum VideoStreamingMode
25: {
26: /// <summary>
27: /// No streaming.
28: /// </summary>
29: None,
30:
31: /// <summary>
32: /// Raises a JavaScript event.
33: /// </summary>
34: Event,
35:
36: /// <summary>
37: /// Draws directly to a target CANVAS.
38: /// </summary>
39: Target,
40:
41: /// <summary>
42: /// Draws in a new window.
43: /// </summary>
44: Window
45: }
And there you have it! Just drop the VideoStreaming control on an ASPX page and you’re done! If you want to have it broadcast, you need to set its Source property to true, for example, on the containing page:
1: protected override void OnInit(EventArgs e)
2: {
3: var source = false;
4:
5: if (Boolean.TryParse(this.Request.QueryString["Source"], out source) == true)
6: {
7: this.video.Source = source;
8: }
9:
10: if (source == false)
11: {
12: //if we are not broadcasting, just hide the VIDEO tag
13: this.video.Style[HtmlTextWriterStyle.Display] = "none";
14: }
15:
16: base.OnInit(e);
17: }
For starting and stopping streaming, you just call the JavaScript functions startStream and stopStream:
1: <script type="text/javascript">
1:
2:
3: function startStreaming()
4: {
5: document.getElementById('video').startStream();
6: }
7:
8: function stopStreaming()
9: {
10: document.getElementById('video').stopStream();
11: }
12:
</script>
2: <input type="button" value="Start Streaming" onclick="startStreaming()"/>
3: <input type="button" value="Stop Streaming" onclick="stopStreaming()" />
That’s it. Very simple video streaming without the need for any plugin or server. Hope you enjoy it!