Guide

webrtcH4cKS: ~ Using getDisplayMedia for local recording with audio on Jitsi

Posted by Chad Hart on June 30, 2020
Posted in: Guide.
Tagged: , , , .
1 comment
local Jitsi recording hack with getDisplayMedia audio capture and mediaRecorder

I wanted to add local recording to my own Jitsi Meet instance. The feature wasn’t built in the way I wanted, so I set out on a hack to build something simple. That lead me down the road to  discovering that:

  1. getDisplayMedia for screen capture has many quirks,
  2. mediaRecorder for media recording has some of its own unexpected limitations, and
  3. Adding your own HTML/JavaScript to Jitsi Meet is pretty simple

Read on for plenty of details and some reference code. My result is located in this repo.

The Problem

I built a Jitsi Meet server a few months ago with the intention of updating my Build your own phone company with WebRTC and a weekend post. There are a billion posts/videos on how to set up Jitsi Meet, and I don’t have anything new or interesting to add to the technosphere there. However, one feature I really wanted to implement is recording. I often do demos and recording the session for others and future reference. On the surface, my requirements here are simple – record my audio and the audio and video of the Jitsi Meet session on demand and save the file locally. This sounds like a simple feature to add, but… ...  Continue reading

webrtcH4cKS: ~ True End-to-End Encryption with WebRTC Insertable Streams

Posted by Philipp Hancke on April 12, 2020
Posted in: Guide.
Tagged: , , , .
4 comments

A couple of weeks ago, the Chrome team announced an interesting Intent to Experiment on the blink-dev list about an API to do some custom processing on top of WebRTC. The intent comes with an explainer document written by Harald Alvestrand which shows the basic API usage. As I mentioned in my last post, this is the sort of thing that maybe able to help add End-to-End Encryption (e2ee) in middlebox scenarios to WebRTC.

I had been watching the implementation progress with quite some interest when former webrtcHacks guest author Emil Ivov of jitsi.org reached out to discuss collaborating on exploring what this API is capable of. Specifically, we wanted to see if WebRTC Insertable Streams could solve the problem of end-to-end encryption for middlebox devices outside of the user’s control like Selective Forwarding Units (SFUs) used for media routing. ...  Continue reading

webrtcH4cKS: ~ Accelerated Computer Vision inside a WebRTC Media Server with Intel OWT

Posted by Chad Hart on April 3, 2020
Posted in: Guide.
Tagged: , , .
Leave a Comment

WebRTC has made getting and sending real time video streams (mostly) easy. The next step is doing something with them, and machine learning lets us have some fun with those streams. Last month I showed how to run Computer Vision (CV) locally in the browser. As I mentioned there, local is nice, but sometimes more performance is needed so you need to run your Machine Learning inference on a remote server. In this post I’ll review how to run OpenCV models server-side with hardware acceleration on Intel chipsets using Intel’s open source Open WebRTC Toolkit (OWT). ...  Continue reading

webrtcH4cKS: ~ Stop touching your face using a browser and TensorFlow.js

Posted by Chad Hart on March 19, 2020
Posted in: Guide.
Tagged: , , .
1 comment

Don’t touch your face! To prevent the spread of disease, health bodies recommend not touching your face with unwashed hands. This is easier said than done if you are sitting in front of a computer for hours.  I wondered, is this a problem that can be solved with a browser?

We have a number of computer vision + WebRTC experiments here. Experimenting with running computer vision locally in the browser using TensorFlow.js has been on my bucket list and this seemed like a good opportunity. A quick search revealed somebody already thought of this 2 week ago. That site used a model that requires some user training – which is interesting but can make it flaky. It also wasn’t open source for others to expand on, so I did some social distancing via coding isolation over the weekend to see what was possible. ...  Continue reading

webrtcH4cKS: ~ Private Home Surveillance with the WebRTC DataChannel (Ivelin Ivanov)

Posted by Ivelin Ivanov on February 25, 2020
Posted in: Guide.
Tagged: , .
4 comments

When most people think of WebRTC they think of video communications. Similarly, home surveillance is usually associated with video streaming. That’s why I was surprised to hear about a home security project that leverages WebRTC not for video streaming, but for the DataChannel. WebRTC’s DataChannel might not demo as well as a video call, but as you will see, it is a very convenient way to setup peer-to-peer information transfer.

Ivelin Ivanov is a long-time open source contributor in a variety of projects and organizations like RedHat, Mobicents, and Telestax. Recently he started to address some home IoT privacy concerns with a new open source project, Ambianic. His project includes many interesting elements, including computer vision on a Raspberry Pi (one of my favorite topics), but for this post I asked him to talk about how he leveraged WebRTC and the DataChannel in his architecture.  ...  Continue reading

webrtcH4cKS: ~ Does Chromium-based Edge’s WebRTC Look Like Chrome?

Posted by Chad Hart on January 16, 2020
Posted in: Guide.
Tagged: , .
3 comments

WebRTC has a new browser – kind of. Yesterday Microsoft’s  “new” Edge browser based on Chromium – commonly referred to Edgium – went GA. This certainly will make life easier for WebRTC developers since the previous Edge had many differences from other implementations. The big question is how different is Edgium from Chrome for WebRTC usage?

The short answer is there is no real difference, but you can read below for background details on the tests I ran. If you’re new around WebRTC the rundown may give you some ideas for testing your own product. ...  Continue reading

webrtcH4cKS: ~ Bisecting Browser Bugs (Arne Georg Gisnås Gleditsch)

Posted by Arne George Gisnås Gleditsch on May 14, 2019
Posted in: Guide.
Tagged: , , , .
2 comments

When running WebRTC at scale, you end up hitting issues and frequent regressions. Being able to quickly identify what exactly broke is key to either preventing a regression from landing in Chrome Stable or adapting your own code to avoid the problem. Chrome’s bisect-builds.py tool makes this process much easier than you would suspect. Arne from appear.in gives you an example of how he used this to workaround an issue that came up recently.
{“editor”, “Philipp Hancke“}

In this post I am going to provide a blow-by-blow account of how a change to Chrome triggered a bug in appear.in and how we went about determining exactly what that change was. ...  Continue reading

webrtcH4cKS: ~ How Janus Battled libFuzzer and Won (Alessandro Toppi)

Posted by Alessandro Toppi on March 6, 2019
Posted in: Guide.
Tagged: , , , , .
Leave a Comment

Thanks to work initiated by Google Project Zero, fuzzing has become a popular topic within WebRTC since late last year.  It was clear WebRTC was lacking in this area. However, the community has shown its strength by giving this topic an immense amount of focus and resolving many issues.  In a previous post, we showed how to break the Janus Server RTCP parser. The Meetecho team behind Janus did not take that lightly. They got to the bottom of what turned out to be quite a big project. In this post Alessandro Toppi of Meetecho will walk us through how they fixed this problem and built an automated process to help make sure it doesn’t happen again. ...  Continue reading

webrtcH4cKS: ~ First steps with QUIC DataChannels

Posted by Philipp Hancke on February 11, 2019
Posted in: Guide, Technology.
Tagged: , , .
7 comments

QUIC-based DataChannels are being considered as an alternative to the current SCTP-based transport. The WebRTC folks at Google are experimenting  with it:

Let’s test this out. We’ll do a simple single-page example similar to the WebRTC datachannel sample that transfers text. It offers a complete working example without involving signaling servers and also allows comparing the approach to WebRTC DataChannels more easily. ...  Continue reading

webrtcH4cKS: ~ Let’s get better at fuzzing in 2019 – here’s how

Posted by Philipp Hancke on December 14, 2018
Posted in: Guide.
Tagged: , , , , .
1 comment

Fuzzing is a Quality Assurance and security testing technique that provides unexpected, often random data to a program input to try to break it. Natalie Silvanovich from Google’s Project Zero team has had quite some fun fuzzing various different RTP implementations recently.

She found vulnerabilities in:

In a nutshell, she found a bunch of vulnerabilities just by throwing unexpected input at parsers. The range of applications which were vulnerable to this shows that the WebRTC/VoIP community does not yet have a process for doing this work ourselves. Meanwhile, the WebRTC folks at Google will have to improve their processes as well. ...  Continue reading