Deploying media servers for WebRTC has two major challenges, scaling beyond a single server as well as optimizing the media latency for all users in the conference. While simple sharding approaches like “send all users in conference X to server Y” are easy to scale horizontally, they are far from optimal in terms of the media latency which is a key factor in the user experience. Distributing a conference to a network of servers located close to the users and interconnected with each other on a reliable backbone promises a solution to both problems at the same time. Boris Grozev from the Jitsi team describes the cascading SFU problem in-depth and shows their approach as well as some of the challenges they ran into.
Search Results
Search Results for: sfu
If you plan to have multiple participants in your WebRTC calls then you will probably end up using a Selective Forwarding Unit (SFU). Capacity planning for SFU’s can be difficult – there are estimates to be made for where they should be placed, how much bandwidth they will consume, and what kind of servers you need.
To help network architects and WebRTC engineers make some of these decisions, webrtcHacks contributor Dr. Alex Gouaillard and his team at CoSMo Software put together a load test suite to measure load vs. video quality. They published their results for all of the major open source WebRTC SFU’s. This suite based is the Karoshi Interoperability Testing Engine (KITE) Google funded and uses on webrtc.org to show interoperability status. The CoSMo team also developed a machine learning based video quality assessment framework optimized for real time communications scenarios.
You don’t need to fit an SFU opponent when testing simulcast. Image: Hall of Mirrors scene from Bruce Lee’s Enter the Dragon
Simulcast is one of the more interesting aspects of WebRTC for multiparty conferencing. In a nutshell, it means sending three different resolution (spatial scalability) and different frame rates (temporal scalability) at the same time. Oscar Divorra’s post contains the full details.
Usually, one needs a SFU to take advantage of simulcast. But there is a hack to make the effect visible between two browsers — or inside a single page. This is very helpful for single-page tests or fiddling with simulcast features, particular the ability to enable only certain spatial layers or to control the target bitrate of a particular stream.
Atlassian’s HipChat acquired BlueJimp, the company behind the Jitsi open source project. Other than for positive motivation, why should WebRTC developers care? Well, Jitsi had its Jitsi Video Bridge (JVB) which was one of the few open source Selective Forwarding Units (SFU) projects out there. Jitsi’s founder and past webrtcHacks guest author, Emil Ivov, was a major advocate for this architecture in both the standards bodies and in the public. As we have covered in the past, SFU’s are an effective way to add multiparty video to WebRTC. Beyond this one component, Jitsi was also a popular open source project for its VoIP client, XMPP components, and much more.
WebRTC has made getting and sending real time video streams (mostly) easy. The next step is doing something with them, and machine learning lets us have some fun with those streams. Last month I showed how to run Computer Vision (CV) locally in the browser. As I mentioned there, local is nice, but sometimes more performance is needed so you need to run your Machine Learning inference on a remote server. In this post I’ll review how to run OpenCV models server-side with hardware acceleration on Intel chipsets using Intel’s open source Open WebRTC Toolkit (OWT).
Note: Intel sponsored this post. I have been wanting to play around with the OWT server since they demoed some of its CV capabilities at Kranky Geek and this gave me a chance to work with their development team to explore its capabilities. Below I share some background on OWT, how to install it locally for quick testing, and show some of the models.
Time for another opinionated post. This time on… end-to-end encryption (e2ee). Zoom apparently claims it supports e2ee while it can not satisfy that promise. Is WebRTC any better?
Zoom does not have End to End Encryption
Let’s get to the bottom of things fast: Boo Zoom!
I reviewed how Zoom’s implements their web client last year.
I’m not really surprised of their general lack of e2ee given that their web client did not provide any encryption on top of TLS or WebRTC’s DataChannel. For reasons we will discuss below, this means they weren’t doing any obvious e2ee there.
When most people think of WebRTC they think of video communications. Similarly, home surveillance is usually associated with video streaming. That’s why I was surprised to hear about a home security project that leverages WebRTC not for video streaming, but for the DataChannel. WebRTC’s DataChannel might not demo as well as a video call, but as you will see, it is a very convenient way to setup peer-to-peer information transfer.
Ivelin Ivanov is a long-time open source contributor in a variety of projects and organizations like RedHat, Mobicents, and Telestax. Recently he started to address some home IoT privacy concerns with a new open source project, Ambianic. His project includes many interesting elements, including computer vision on a Raspberry Pi (one of my favorite topics), but for this post I asked him to talk about how he leveraged WebRTC and the DataChannel in his architecture.
{“editor”, “chad hart“}
SDP has been a frequent topic, both here on webrtcHacks as well as in the discussion about the standard itself. Modifying the SDP in arcane ways is referred to as SDP munging. This post gives an introduction into what SDP munging is, why its done and why it should not be done. This is not a guide to SDP munging.
An analysis of the most popular open-source WebRTC repos on GitHub with a review of how WebRTC itself is doing there.
When running WebRTC at scale, you end up hitting issues and frequent regressions. Being able to quickly identify what exactly broke is key to either preventing a regression from landing in Chrome Stable or adapting your own code to avoid the problem. Chrome’s bisect-builds.py tool makes this process much easier than you would suspect. Arne from appear.in gives you an example of how he used this to workaround an issue that came up recently.
{“editor”, “Philipp Hancke“}
In this post I am going to provide a blow-by-blow account of how a change to Chrome triggered a bug in appear.in and how we went about determining exactly what that change was.
