Popular this Month

webrtcH4cKS: ~ Improving Scale and Media Quality with Cascading SFUs (Boris Grozev)

Posted by Boris Grozev on November 12, 2018
Posted in: Technology.
Tagged: , , .
Leave a Comment

Deploying media servers for WebRTC has two major challenges, scaling beyond a single server as well as optimizing the media latency for all users in the conference. While simple sharding approaches like “send all users in conference X to server Y” are easy to scale horizontally, they are far from optimal in terms of the media latency which is a key factor in the user experience. Distributing a conference to a network of servers located close to the users and interconnected with each other on a reliable backbone promises a solution to both problems at the same time. Boris Grozev from the Jitsi team describes the cascading SFU problem in-depth and shows their approach as well as some of the challenges they ran into. ...

Continue Reading

webrtcH4cKS: ~ How Zoom’s web client avoids using WebRTC

Posted by Philipp Hancke on October 23, 2018
Posted in: Reverse-Engineering.
Tagged: , , , , .
1 comment

Rube Goldberg’s Professor Butts and the Self-Operating Napkin (1931)

Zoom has a web client that allows a participant to join meetings without downloading their app. Chris Koehncke was excited to see how this worked (watch him at the upcoming KrankyGeek event!) so we gave it a try. It worked, removing the download barrier. The quality was acceptable and we had a good chat for half an hour.

Opening chrome://webrtc-internals showed only getUserMedia being used for accessing camera and microphone but no  RTCPeerConnection like a WebRTC call should have. This got me very interested – how are they making calls without WebRTC? ...

Continue Reading

webrtcH4cKS: ~ Breaking Point: WebRTC SFU Load Testing (Alex Gouaillard)

Posted by Alex Gouaillard@gmail.com on October 18, 2018
Posted in: Technology.
Tagged: , , , , , , , .
6 comments

If you plan to have multiple participants in your WebRTC calls then you will probably end up using a Selective Forwarding Unit (SFU).  Capacity planning for SFU’s can be difficult – there are estimates to be made for where they should be placed, how much bandwidth they will consume, and what kind of servers you need.

To help network architects and WebRTC engineers make some of these decisions, webrtcHacks contributor Dr. Alex Gouaillard and his team at CoSMo Software put together a load test suite to measure load vs. video quality. They published their results for all of the major open source WebRTC SFU’s. This suite based is the Karoshi Interoperability Testing Engine (KITE) Google funded and uses on webrtc.org to show interoperability status. The CoSMo team also developed a machine learning based video quality assessment framework optimized for real time communications scenarios. ...

Continue Reading

webrtcH4cKS: ~ Messenger was not forced to wiretap but…

Posted by Philipp Hancke on October 1, 2018
Posted in: Reverse-Engineering.
Tagged: , , .
Leave a Comment
This Phone Is Tapped.jpg


By david drexler – Flickr, CC BY 2.0, Link

Back in August, Reuters reported on a “secret legal fight” between the FBI and Facebook about wiretapping Messenger calls. The Verge as they found our old post about reverse-engineering Messenger from 2015 and had a number of follow-up questions on it for a Messenger wiretapping article they ran. Technical details on the case are quite hard to find so I was not able to dig deeper into the specifics around wiretapping.

Reuters now reports that Facebook will not be forced to wiretap Messenger calls with the FBI noting: ...

Continue Reading

webrtcH4cKS: ~ Guide to WebRTC with Safari in the Wild (Chad Phillips)

Posted by Chad Phillips on September 7, 2018
Posted in: Guide.
Tagged: , , , , .
10 comments

It has been more than a year since Apple first added WebRTC support to Safari. My original post reviewing the implementation continues to be popular here, but it does not reflect some of the updates since the first limited release. More importantly, given its differences and limitations, many questions still remained on how to best develop WebRTC applications for Safari.

I ran into Chad Phillips at Cluecon (again) this year and we ended up talking about his arduous experience making WebRTC work on Safari. He had a great, recent list of tips and tricks so I asked him to share it here. ...

Continue Reading

webrtcH4cKS: ~ VR Video Calling with WebRTC and WebVR (Dan Jenkins)

Posted by Dan Jenkins on August 27, 2018
Posted in: Guide.
Tagged: , , , , , , .
Leave a Comment

WebRTC isn’t the only cool media API on the Web Platform. The Web Virtual Reality (WebVR) spec was introduced a few years ago to bring support for virtual reality devices in a web browser. It has since been migrated to the newer WebXR Device API Specification.

I was at ClueCon earlier this summer where Dan Jenkins gave a talk showing that it is relatively easy to add a WebRTC video conference streams into a virtual reality environment using WebVR using FreeSWITCH. FreeSWITCH is one of the more popular open source telephony platforms and has had WebRTC for a few years. WebRTC; WebVR; Open Source – obviously this was good webrtcHacks material. ...

Continue Reading

webrtcH4cKS: ~ Suspending Simulcast Streams for Savvy Streamlining (Brian Baldino)

Posted by Brian Baldino on August 6, 2018
Posted in: Technology.
Tagged: , , , , , , .
1 comment

If you’re new to WebRTC, Jitsi was the first open source Selective Forwarding Unit (SFU) and continues to be one of the most popular WebRTC platforms. They were in the news last week because their parent group inside Atlassian was sold off to Slack but the team clarified this does not have any impact on the Jitsi team. Helping to show they are still chugging along, they released a new feature they wanted to talk about – off-stage layer suspension. This is a technique for minimizing bandwidth and CPU consumption when using simulcast. Simulcast is a common technique used in multi-party video scenarios. See Oscar Divorra’s post on this topic and that Fippo post just last week for more on that. Even if you are not implementing a  simulcast, this is a good post for understanding how to control bandwidth and to see some follow-along reverse-engineering on how Google does things in its Hangouts upgrade called Meet. ...

Continue Reading

webrtcH4cKS: ~ A playground for Simulcast without an SFU

Posted by Philipp Hancke on July 31, 2018
Posted in: Guide.
Tagged: , .
1 comment

You don’t need to fit an SFU opponent when testing simulcast. Image: Hall of Mirrors scene from Bruce Lee’s Enter the Dragon

Simulcast is one of the more interesting aspects of WebRTC for multiparty conferencing. In a nutshell, it means sending three different resolution (spatial scalability) and different frame rates (temporal scalability) at the same time. Oscar Divorra’s post contains the full details.

Usually, one needs a SFU to take advantage of simulcast. But there is a hack to make the effect visible between two browsers — or inside a single page. This is very helpful for single-page tests or fiddling with simulcast features, particular the ability to enable only certain spatial layers or to control the target bitrate of a particular stream. ...

Continue Reading

webrtcH4cKS: ~ Chrome Screensharing Blues – preparing for getDisplayMedia

Posted by Philipp Hancke on June 14, 2018
Posted in: Technology.
Tagged: , , , .
3 comments
What happens when you screen share on a computer that's already sharing your screen

The Chrome Webstore has decided to stop allowing inline installation for Chrome extensions. This has quite an impact on WebRTC applications since screensharing in Chrome currently requires an extension. Will the getDisplayMedia API come to the rescue?

Screensharing in Chrome

When screensharing was introduced in Chrome 33, it required implementation via an extension as a way to address the security concerns. This was better than the previous experience of putting this capability behind a flag which lead to sites asking their users to change that flag… that got those sites an official yikes. ...

Continue Reading

webrtcH4cKS: ~ Smile, You’re on WebRTC – Using ML Kit for Smile Detection

Posted by Gustavo Garcia on June 11, 2018
Posted in: Guide.
Tagged: , , , , , , .
Leave a Comment

Now that it is getting relatively easy to setup video calls (most of the time), we can move on to doing fun things with the video stream. With new advancements in Machine Learning (ML) and a growing number of API’s and libraries out there, computer vision is also getting  easier to do. Google’s ML Kit is a recent example of a new machine learning based library that makes gives quick access to computer vision outputs.

To show how to use Google’s new ML Kit to detect user smiles on a live WebRTC stream, I would like to welcome back past webrtcHacks author and WebRTC video master  Gustavo Garcia Bernardo of Houseparty. Joining him I would like to also welcome mobile WebRTC expert, Roberto Perez of TokBox.  They give some background on doing facial detection, show some code samples, but more importantly share their learnings for optimum configuration of smile detection inside a Real Time Communications (RTC) app. ...

Continue Reading