Walkthrough

All posts tagged Walkthrough

webrtcH4cKS: ~ Using getDisplayMedia for local recording with audio on Jitsi

Posted by Chad Hart on June 30, 2020
Posted in: Guide.
Tagged: , , , .
1 comment
local Jitsi recording hack with getDisplayMedia audio capture and mediaRecorder

I wanted to add local recording to my own Jitsi Meet instance. The feature wasn’t built in the way I wanted, so I set out on a hack to build something simple. That lead me down the road to  discovering that:

  1. getDisplayMedia for screen capture has many quirks,
  2. mediaRecorder for media recording has some of its own unexpected limitations, and
  3. Adding your own HTML/JavaScript to Jitsi Meet is pretty simple

Read on for plenty of details and some reference code. My result is located in this repo.

The Problem

I built a Jitsi Meet server a few months ago with the intention of updating my Build your own phone company with WebRTC and a weekend post. There are a billion posts/videos on how to set up Jitsi Meet, and I don’t have anything new or interesting to add to the technosphere there. However, one feature I really wanted to implement is recording. I often do demos and recording the session for others and future reference. On the surface, my requirements here are simple – record my audio and the audio and video of the Jitsi Meet session on demand and save the file locally. This sounds like a simple feature to add, but… ...  Continue reading

webrtcH4cKS: ~ Accelerated Computer Vision inside a WebRTC Media Server with Intel OWT

Posted by Chad Hart on April 3, 2020
Posted in: Guide.
Tagged: , , .
Leave a Comment

WebRTC has made getting and sending real time video streams (mostly) easy. The next step is doing something with them, and machine learning lets us have some fun with those streams. Last month I showed how to run Computer Vision (CV) locally in the browser. As I mentioned there, local is nice, but sometimes more performance is needed so you need to run your Machine Learning inference on a remote server. In this post I’ll review how to run OpenCV models server-side with hardware acceleration on Intel chipsets using Intel’s open source Open WebRTC Toolkit (OWT). ...  Continue reading

webrtcH4cKS: ~ First steps with QUIC DataChannels

Posted by Philipp Hancke on February 11, 2019
Posted in: Guide, Technology.
Tagged: , , .
7 comments

QUIC-based DataChannels are being considered as an alternative to the current SCTP-based transport. The WebRTC folks at Google are experimenting  with it:

Let’s test this out. We’ll do a simple single-page example similar to the WebRTC datachannel sample that transfers text. It offers a complete working example without involving signaling servers and also allows comparing the approach to WebRTC DataChannels more easily. ...  Continue reading

webrtcH4cKS: ~ Let’s get better at fuzzing in 2019 – here’s how

Posted by Philipp Hancke on December 14, 2018
Posted in: Guide.
Tagged: , , , , .
1 comment

Fuzzing is a Quality Assurance and security testing technique that provides unexpected, often random data to a program input to try to break it. Natalie Silvanovich from Google’s Project Zero team has had quite some fun fuzzing various different RTP implementations recently.

She found vulnerabilities in:

In a nutshell, she found a bunch of vulnerabilities just by throwing unexpected input at parsers. The range of applications which were vulnerable to this shows that the WebRTC/VoIP community does not yet have a process for doing this work ourselves. Meanwhile, the WebRTC folks at Google will have to improve their processes as well. ...  Continue reading

webrtcH4cKS: ~ Computer Vision on the Web with WebRTC and TensorFlow

Posted by Chad Hart on December 3, 2017
Posted in: Guide.
Tagged: , , .
26 comments

TensorFlow is one of the most popular Machine Learning frameworks out there – probably THE most popular one. One of the great things about TensorFlow is that many libraries are actively maintained and updated. One of my favorites is the TensorFlow Object Detection API.   The Tensorflow Object Detection API classifies and provides the location of multiple objects in an image. It comes pre-trained on nearly 1000 object classes with a wide variety of pre-trained models that let you trade off speed vs. accuracy. ...  Continue reading

webrtcH4cKS: ~ Debugging VP8 is more fun than it used to be

Posted by Philipp Hancke on March 28, 2017
Posted in: Guide.
Tagged: , , , , .
3 comments

Editor Note: Fippo uses a lot of advanced WebRTC terms below – if you are a regular reader of this blog then don’t let that scare  you. Wireshark is a great tool for diagnosing media issues and inspecting signaling packets even if you’re not building a media server. {“editor”, “chad hart“}

Stuff breaks all the time and then you need to debug it. My favorite tool for this remains Wireshark as we have seen previously. Its fairly useful for debugging all the ICE and DTLS stuff but recently I’ve had to debug the media traffic itself. ...  Continue reading

webrtcH4cKS: ~ How to limit WebRTC bandwidth by modifying the SDP

Posted by Nick Gauthier on October 26, 2016
Posted in: Guide.
Tagged: , , , .
16 comments

WebRTC 1.0 uses SDP for negotiating capabilities between parties.  While there are a growing number of objects coming to WebRTC to avoid this protocol from the 90’s , the reality is SDP will be with us for some time. If you want to do things like change codecs or adjust bandwidth limits, then you’re going to need to “munge” SDP for the time being.

At a recent WebRTC Boston, Nick Gauthier of MeetSpace described how he used SDP modification and other techniques to jam up to 10 video callers into a single conference without a media server. Not everyone has a good reason to do this, but there are certainly plenty of applications where having more precise control of your bandwidth consumption would be useful. You can see his video here or check out his technique and thorough explanation on how to munge SDP to adjust individual bandwidth usage below. ...  Continue reading

webrtcH4cKS: ~ Let’s Encrypt – how get to free SSL for WebRTC

Posted by Chad Hart on August 1, 2016
Posted in: Guide.
Tagged: , , , , , , , , .
7 comments

Way back in 47 (version that is), Chrome started to mandate the use of HTTPS in conjunction with getUserMedia. To use HTTPS you need a SSL/TLS certificate.  Xander Dumaine covered this a bit for us before, but I still see a lot of people out there struggle with it. As it so happens, the certificate for my own personal website is about to expire and I’m not too excited about paying $70/year to renew it. Fortunately, there is a new way to get certificates for free through Let’s Encrypt. Let’s Encrypt is a non-profit certificate authority that formed with the backing of many major industry players like Mozilla, Akamai, Cisco, and many others to simplify and automate the process of setting up encryption for your website. Oh, and its completely free. ...  Continue reading

webrtcH4cKS: ~ Your Browser as a Audio Conference Server with WebRTC & Web Audio (Alexey Aylarov)

Posted by Chad Hart on May 20, 2016
Posted in: Guide.
Tagged: , , , , .
4 comments

Conference calling is a multi-billion dollar industry that is mostly powered by expensive, high-powered conferencing servers. Now you can replicate much of this functionality for free with a modern browser using the combination of WebRTC and WebAudio.

Like with video, multi-party audio can utilize a few architectures:

  1. Full mesh – each client sends their audio to every other client; the individual streams are then combined locally before they come out of your speaker
  2. Mixed with a conferencing server acting as a Multipoint Control Unit (MCU) – the MCU combines each stream and sends a single set to each client
  3. Routed with a conferencing server in a Selective Forwarding Unit (SFU) mode – each client sends a single stream to the server where it is replicated and sent to the others

This architecture represents a fourth type: client-mixed type where one of the clients acts like the server. This provides the server-less benefits of mesh conferencing without the excessive bandwidth usage and stream management challenges. ...  Continue reading

webrtcH4cKS: ~ Update: Anatomy of a WebRTC SDP (Antón Román)

Posted by Chad Hart on May 5, 2016
Posted in: Guide.
Tagged: , , .
1 comment

Session Description Protocol (SDP) is a fundamental, but very unintuitive concept behind how WebRTC works today. Its no wonder that the Anatomy of a WebRTC SDP post and the interactive SDP guide by Quobis CTO, Antón Román has been so popular here on webrtcHacks. With all things WebRTC, things have changed and we were due for an update.

We also had some rendering issues on the interactive guide. After failing to figure out how to fix it, I decided to completely rewrite it. It is still has some issues, so please make your pull requests to fix and update it on our github repo here. ...  Continue reading