All I want for Christmas is Hangouts to use WebRTC on Firefox

As the year 2017 comes to an end, there was a small present. Hangouts started to support Firefox with WebRTC instead of rejecting access – plugin access had been unavailable since Firefox 53 removed NPAPI in April 2017. While it had been public for a while that the Firefox WebRTC team had been testing this, it was a nice Christmas present to see this shipped. Tsahi Levent-Levi was one of the first people to notice.
This comes at a time where other Google teams are being criticized for promoting Chrome-only experiences. Kudos to the Hangouts team for showing that you still care about the web as an open platform!

Last time I wrote about Hangouts in mid 2014, it used quite a bunch of non-standard techniques like SDES and RTP DataChannel. Neither works in Firefox which was a technical blocker for supporting it (also there were rumors about NaCl and the ‘hats’ feature being responsible as I mentioned in the old blog post).

However, as part of modernizing the video hangouts product with Meet things changed. Opus has long since been the default audio codec for example. And while Hangouts in Chrome is not 100% WebRTC 1.0 spec compatible (for example I have already seen Chrome use DTLS-SRTP instead of SDES), the Firefox implementation seems to play a bit differently and is more standards-compliant:

Comparing the WebRTC Spec to FireFox Hangouts and Chrome Hangouts

Feature	WebRTC/RTCWeb Specifications	Hangouts/Firefox	Hangouts/Chrome
encryption	DTLS-SRTP	DTLS-SRTP	SDES but DTLS-SRTP was observed in the wild
data channels	DTLS/SCTP-based	DTLS/SCTP-based	unreliable RTP-based
ICE	ICE RFC 5245(bis)	ICE (lite)	ICE (lite)
Audio codec	Opus or G.711	Opus	Opus
Multiple streams	JSEP / Unified Plan	JSEP / Unified Plan	Plan B
Simulcast	draft-ietf-mmusic-sdp-simulcast	slightly outdated version of simulcast draft	SDP munging

A closer look at the SDP

Lets look at the SDP. Unfortunately the webrtc-externals extension has been broken in Firefox since FF57 and nobody has had time to figure out why exactly. But the SDP we get from about:webrtc is quite interesting actually:

We can lear a lot from this, so let’s dig into it. For a deeper review of WebRTC SDP in general, check out the SDP Anatomy guide.

RTP payload type

Firefox creates the offer which means it chooses the RTP payload type for audio and video packets and the server needs to do a rewrite on every packet. This is quite trivial but takes developer effort.

Simulcast

The a=rid  and a=simulcast  lines show that simulcast is used. Firefox doesn’t implement the latest IETF draft here but unlike Chrome it does not need SDP munging, relying on RTCRtpSender.setParameters . Firefox has a unit-test showing the basic usage if you’re interested. This is of course so much better than SDP munging (even though it is still SDP which we all should be afraid of). On the wire, Firefox simulcast uses the RID header extension to “tag” the different video streams.
Simulcast was not very stable in Firefox until FF57 which might further explain the timing.

REMB

REMB is used for bandwidth estimation instead of the newer transport-cc mechanism which is not supported in Firefox yet. In Chrome transport-cc is used which means it is terminated in the server. Making the two work together nicely might have been quite some effort as bandwidth-estimation is one of the hardest problems in WebRTC.

SCTP Data Channels

As we can see in the m=application  media section, SCTP data channels are used. That is a fair bit of engineering effort but probably happened as part of the general overhaul for Meet earlier as RTP data channels don’t work together with DTLS-SRTP. This also indicates that the control messages sent via the data channel are important. Unfortunately they are protobuf-encoded so relatively hard to make sense of.

Unified Plan

The most interesting thing is the use of “Unified Plan” SDP (which has since merged into JSEP) with a total of five m-lines. The lack of unified plan in Chrome is apparently not a concern. This is not very surprising, if you want to use a single PeerConnection you can simply munge the SDP to whatever format the browser desired. Jitsi has done this for ages in the sdp-interop package. It would be interesting to check if this manipulation happens in the client-side javascript or in the server. Leave a comment if you do!

What is missing in the SDP is also interesting. Since Firefox does not support RTX, ulpfec and red, the server needs to unwrap those packets and either drop them or, in the RTX case, transform them back to normal RTP packets. This is far from trivial in my experience.

ice-lite

In the SDP from the server we also find ice-lite which is a simplified version of ICE allowed by RFC 5245 that is well-suited for media servers with public ip addresses and easy to implement. Hangouts moved from google-ice towards ice-lite a while ago. Check the full about:webrtc here for the gory details.

tl;dr

Google made Hangouts work with Firefox with some help from Mozillas WebRTC team. Yes, this took Google quite a while – more than three years. It was about time given the massive scale of Hangouts. But if folks like the Hangouts team and the Mozilla WebRTC team work together awesome things happen. Many thanks to everyone involved!

{“author”: “Philipp Hancke“}