- How Zoom’s web client avoids using WebRTC (DataChannel Update)
- Guide to WebRTC with Safari in the Wild (Chad Phillips)
- Using getDisplayMedia for local recording with audio on Jitsi
- Open Source Cloud Gaming with WebRTC
- Breaking Point: WebRTC SFU Load Testing (Alex Gouaillard)
- True End-to-End Encryption with WebRTC Insertable Streams
- Computer Vision on the Web with WebRTC and TensorFlow
Popular this Month
webrtcH4cKS: ~ Implementing REDundant audio on an SFU
Chrome recently added the option of adding redundancy to audio streams using the RED format as defined in RFC 2198, and Fippo wrote about the process and implementation in a previous article. You should catch-up on that post, but to summarize quickly RED works by adding redundant payloads with different timestamps in the same packet. If you lose a packet in a lossy network then chances are another successfully received packet will have the missing data resulting in better audio quality.
That was in a simplified one-to-one scenario, but audio quality issues often have the most impact on larger multi-party calls. As a follow-up to Fippo’s post, Jitsi Architect and Improving Scale and Media Quality with Cascading SFUs author Boris Grozev walks us through his design and tests for adding audio redundancy to a more complex environment with many peers routing media through a Selective Forwarding Unit (SFU). ...
webrtcH4cKS: ~ RED: Improving Audio Quality with Redundancy
Back in April 2020 a Citizenlab reported on Zoom’s rather weak encryption and stated that Zoom uses the SILK codec for audio. Sadly, the article did not contain the raw data to validate that and let me look at it further. Thankfully Natalie Silvanovich from Googles Project Zero helped me out using the Frida tracing tool and provided a short dump of some raw SILK frames. Analysis of this inspired me to take a look at how WebRTC handles audio. In terms of perception, audio quality is much more critical for the perceived quality of a call as we tend to notice even small glitches. Mere ten seconds of this audio analysis were enough to set me off on quite an adventure investigating possible improvements to the audio quality provided by WebRTC. ...
I wanted to add local recording to my own Jitsi Meet instance. The feature wasn’t built in the way I wanted, so I set out on a hack to build something simple. That lead me down the road to discovering that:
getDisplayMediafor screen capture has many quirks,mediaRecorderfor media recording has some of its own unexpected limitations, and- Adding your own HTML/JavaScript to Jitsi Meet is pretty simple
Read on for plenty of details and some reference code. My result is located in this repo.
Want to keep up on our latest posts? Please click here to subscribe to our mailing list if you have not already. We only email post updates. You can also follow us on twitter at @webrtcHacks for blog updates. ...
webrtcH4cKS: ~ Open Source Cloud Gaming with WebRTC
Software as a Service, Infrastructure as a Service, Platform as a Service, Communications Platform as a Service, Video Conferencing as a Service, but what about Gaming as a Service? There have been a few attempts at Cloud Gaming, most notably Google’s recently launched Stadia. Stadia is no stranger to WebRTC, but can others leverage WebRTC in the same way?
Thanh Nguyen set out to see if this was possible with his open source project, CloudRetro. CloudRetro is based on the popular go-based WebRTC library, pion (thanks to Sean of Pion for helping review here). In this post, Thanh gives an architectural review of how he build the project along with some of the benefits and challanges he ran into along the way. ...
webrtcH4cKS: ~ True End-to-End Encryption with WebRTC Insertable Streams
A couple of weeks ago, the Chrome team announced an interesting Intent to Experiment on the blink-dev list about an API to do some custom processing on top of WebRTC. The intent comes with an explainer document written by Harald Alvestrand which shows the basic API usage. As I mentioned in my last post, this is the sort of thing that maybe able to help add End-to-End Encryption (e2ee) in middlebox scenarios to WebRTC.
I had been watching the implementation progress with quite some interest when former webrtcHacks guest author Emil Ivov of jitsi.org reached out to discuss collaborating on exploring what this API is capable of. Specifically, we wanted to see if WebRTC Insertable Streams could solve the problem of end-to-end encryption for middlebox devices outside of the user’s control like Selective Forwarding Units (SFUs) used for media routing. ...
WebRTC has made getting and sending real time video streams (mostly) easy. The next step is doing something with them, and machine learning lets us have some fun with those streams. Last month I showed how to run Computer Vision (CV) locally in the browser. As I mentioned there, local is nice, but sometimes more performance is needed so you need to run your Machine Learning inference on a remote server. In this post I’ll review how to run OpenCV models server-side with hardware acceleration on Intel chipsets using Intel’s open source Open WebRTC Toolkit (OWT). ...
Time for another opinionated post. This time on… end-to-end encryption (e2ee). Zoom apparently claims it supports e2ee while it can not satisfy that promise. Is WebRTC any better?
Zoom does not have End to End Encryption
Let’s get to the bottom of things fast: Boo Zoom!
I reviewed how Zoom’s implements their web client last year.
I’m not really surprised of their general lack of e2ee given that their web client did not provide any encryption on top of TLS or WebRTC’s DataChannel. For reasons we will discuss below, this means they weren’t doing any obvious e2ee there. ...
webrtcH4cKS: ~ Stop touching your face using a browser and TensorFlow.js
Don’t touch your face! To prevent the spread of disease, health bodies recommend not touching your face with unwashed hands. This is easier said than done if you are sitting in front of a computer for hours. I wondered, is this a problem that can be solved with a browser?
We have a number of computer vision + WebRTC experiments here. Experimenting with running computer vision locally in the browser using TensorFlow.js has been on my bucket list and this seemed like a good opportunity. A quick search revealed somebody already thought of this 2 week ago. That site used a model that requires some user training – which is interesting but can make it flaky. It also wasn’t open source for others to expand on, so I did some social distancing via coding isolation over the weekend to see what was possible. ...
When most people think of WebRTC they think of video communications. Similarly, home surveillance is usually associated with video streaming. That’s why I was surprised to hear about a home security project that leverages WebRTC not for video streaming, but for the DataChannel. WebRTC’s DataChannel might not demo as well as a video call, but as you will see, it is a very convenient way to setup peer-to-peer information transfer.
Ivelin Ivanov is a long-time open source contributor in a variety of projects and organizations like RedHat, Mobicents, and Telestax. Recently he started to address some home IoT privacy concerns with a new open source project, Ambianic. His project includes many interesting elements, including computer vision on a Raspberry Pi (one of my favorite topics), but for this post I asked him to talk about how he leveraged WebRTC and the DataChannel in his architecture. ...
