• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
webrtcHacks

webrtcHacks

Guides and information for WebRTC developers

  • Home
  • About
    • Chad Hart
    • Philipp Hancke
  • Subscribe
  • Contact
  • Show Search
Hide Search

Reverse-Engineering Blackbox Exploration, Firefox, hangouts, SDP Philipp Hancke · December 21, 2017

All I want for Christmas is Hangouts to use WebRTC on Firefox

Tsahi discovered Hangouts on Firefox started working again and quickly called Fippo to investigate

As the year 2017 comes to an end, there was a small present. Hangouts started to support Firefox with WebRTC instead of rejecting access – plugin access had been unavailable since Firefox 53 removed NPAPI in April 2017. While it had been public for a while that the Firefox WebRTC team had been testing this, it was a nice Christmas present to see this shipped. Tsahi Levent-Levi was one of the first people to notice.
This comes at a time where other Google teams are being criticized for promoting Chrome-only experiences. Kudos to the Hangouts team for showing that you still care about the web as an open platform!

Last time I wrote about Hangouts in mid 2014, it used quite a bunch of non-standard techniques like SDES and RTP DataChannel. Neither works in Firefox which was a technical blocker for supporting it (also there were rumors about NaCl and the ‘hats’ feature being responsible as I mentioned in the old blog post).

However, as part of modernizing the video hangouts product with Meet things changed. Opus has long since been the default audio codec for example. And while Hangouts in Chrome is not 100% WebRTC 1.0 spec compatible (for example I have already seen Chrome use DTLS-SRTP instead of SDES), the Firefox implementation seems to play a bit differently and is more standards-compliant:

Comparing the WebRTC Spec to FireFox Hangouts and Chrome Hangouts

 Feature WebRTC/RTCWeb Specifications Hangouts/Firefox Hangouts/Chrome
encryption DTLS-SRTP DTLS-SRTP SDES but DTLS-SRTP was observed in the wild
data channels DTLS/SCTP-based DTLS/SCTP-based unreliable RTP-based
ICE ICE RFC 5245(bis) ICE (lite) ICE (lite)
Audio codec Opus or G.711 Opus Opus
Multiple streams JSEP / Unified Plan JSEP / Unified Plan Plan B
Simulcast draft-ietf-mmusic-sdp-simulcast slightly outdated version of simulcast draft SDP munging

A closer look at the SDP

Lets look at the SDP. Unfortunately the webrtc-externals extension has been broken in Firefox since FF57 and nobody has had time to figure out why exactly. But the SDP we get from about:webrtc is quite interesting actually:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
v=0
o=mozilla...THIS_IS_SDPARTA-57.0.1 8208570803153758710 3 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 79:67:68:53:C0:3C:4D:60:1B:DD:D5:FE:BA:D0:86:3C:30:44:FE:4B:14:CB:ED:E4:D3:21:22:88:F9:25:F2:F5
a=group:BUNDLE mid_0 mid_1 mid_2 mid_3 mid_4
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 37842 UDP/TLS/RTP/SAVPF 109
c=IN IP4 84.20.98.117
a=candidate:0 1 UDP 2122252543 192.168.1.230 37842 typ host
a=candidate:3 1 TCP 2105524479 192.168.1.230 9 typ host tcptype active
a=candidate:1 1 UDP 1686052863 84.20.98.117 37842 typ srflx raddr 192.168.1.230 rport 37842
a=sendrecv
a=end-of-candidates
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=ice-pwd:72af2dc778d255334cb38661e30fc57a
a=ice-ufrag:80e16c7c
a=mid:mid_0
a=msid:{7a66c1a7-b588-4d8d-ab7c-92cb4a248aea} {1dda5d9f-d705-4db7-bfb1-1153833be2a4}
a=rtcp-mux
a=rtpmap:109 opus/48000/2
a=setup:active
a=ssrc:490004612 cname:{90bfee9f-4610-43b0-993f-a064332e442b}
m=video 37842 UDP/TLS/RTP/SAVPF 120
c=IN IP4 84.20.98.117
a=sendrecv
a=extmap:1 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3/sendonly urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=fmtp:120 max-fs=12288;max-fr=60
a=ice-pwd:72af2dc778d255334cb38661e30fc57a
a=ice-ufrag:80e16c7c
a=mid:mid_1
a=msid:{444747d4-c2d2-45ce-a761-c572c0648e18} {6030f14c-68c2-4f35-8803-af7011d4bbb8}
a=rid:f send
a=rid:h send
a=rid:q send
a=rtcp-fb:120 nack
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-mux
a=rtpmap:120 VP8/90000
a=setup:active
a=simulcast: send rid=f;h;q
a=ssrc:944082137 cname:{90bfee9f-4610-43b0-993f-a064332e442b}
a=ssrc:3093937015 cname:{90bfee9f-4610-43b0-993f-a064332e442b}
a=ssrc:3682296736 cname:{90bfee9f-4610-43b0-993f-a064332e442b}
m=application 37842 DTLS/SCTP 5000
c=IN IP4 84.20.98.117
a=sendrecv
a=ice-pwd:72af2dc778d255334cb38661e30fc57a
a=ice-ufrag:80e16c7c
a=mid:mid_2
a=rtcp-mux
a=sctpmap:5000 webrtc-datachannel 256
a=setup:active
a=max-message-size:1073741823
m=audio 37842 UDP/TLS/RTP/SAVPF 109
c=IN IP4 84.20.98.117
a=recvonly
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
a=ice-pwd:72af2dc778d255334cb38661e30fc57a
a=ice-ufrag:80e16c7c
a=mid:mid_3
a=rtcp-mux
a=rtpmap:109 opus/48000/2
a=setup:active
a=ssrc:2828437981 cname:{90bfee9f-4610-43b0-993f-a064332e442b}
m=video 37842 UDP/TLS/RTP/SAVPF 120
c=IN IP4 84.20.98.117
a=recvonly
a=extmap:1 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3/sendonly urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=fmtp:120 max-fs=12288;max-fr=60
a=ice-pwd:72af2dc778d255334cb38661e30fc57a
a=ice-ufrag:80e16c7c
a=mid:mid_4
a=rtcp-fb:120 nack
a=rtcp-fb:120 ccm fir
a=rtcp-fb:120 goog-remb
a=rtcp-mux
a=rtpmap:120 VP8/90000
a=setup:active
a=ssrc:3511641524 cname:{90bfee9f-4610-43b0-993f-a064332e442b}

We can lear a lot from this, so let’s dig into it. For a deeper review of WebRTC SDP in general, check out the SDP Anatomy guide.

RTP payload type

1
a=rtpmap:109 opus/48000/2

Firefox creates the offer which means it chooses the RTP payload type for audio and video packets and the server needs to do a rewrite on every packet. This is quite trivial but takes developer effort.

Simulcast

1
2
3
4
5
a=simulcast: recv rid=f;h;q
a=rid:f recv
a=rid:h recv
a=rid:q recv
a=extmap:3/recvonly urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id

The a=rid  and a=simulcast  lines show that simulcast is used. Firefox doesn’t implement the latest IETF draft here but unlike Chrome it does not need SDP munging, relying on RTCRtpSender.setParameters . Firefox has a unit-test showing the basic usage if you’re interested. This is of course so much better than SDP munging (even though it is still SDP which we all should be afraid of). On the wire, Firefox simulcast uses the RID header extension to “tag” the different video streams.
Simulcast was not very stable in Firefox until FF57 which might further explain the timing.

REMB

1
a=rtcp-fb:120 goog-remb

REMB is used for bandwidth estimation instead of the newer transport-cc mechanism which is not supported in Firefox yet. In Chrome transport-cc is used which means it is terminated in the server. Making the two work together nicely might have been quite some effort as bandwidth-estimation is one of the hardest problems in WebRTC.

SCTP Data Channels

1
m=application 9 DTLS/SCTP 5000

As we can see in the m=application  media section, SCTP data channels are used. That is a fair bit of engineering effort but probably happened as part of the general overhaul for Meet earlier as RTP data channels don’t work together with DTLS-SRTP. This also indicates that the control messages sent via the data channel are important. Unfortunately they are protobuf-encoded so relatively hard to make sense of.

Unified Plan

1
m=audio 9 UDP/TLS/RTP/SAVPF 109 0

The most interesting thing is the use of “Unified Plan” SDP (which has since merged into JSEP) with a total of five m-lines. The lack of unified plan in Chrome is apparently not a concern. This is not very surprising, if you want to use a single PeerConnection you can simply munge the SDP to whatever format the browser desired. Jitsi has done this for ages in the sdp-interop package. It would be interesting to check if this manipulation happens in the client-side javascript or in the server. Leave a comment if you do!

What is missing in the SDP is also interesting. Since Firefox does not support RTX, ulpfec and red, the server needs to unwrap those packets and either drop them or, in the RTX case, transform them back to normal RTP packets. This is far from trivial in my experience.

ice-lite

1
a=ice-lite

In the SDP from the server we also find ice-lite which is a simplified version of ICE allowed by RFC 5245 that is well-suited for media servers with public ip addresses and easy to implement. Hangouts moved from google-ice towards ice-lite a while ago. Check the full about:webrtc here for the gory details.

tl;dr

Google made Hangouts work with Firefox with some help from Mozillas WebRTC team. Yes, this took Google quite a while – more than three years. It was about time given the massive scale of Hangouts. But if folks like the Hangouts team and the Mozilla WebRTC team work together awesome things happen. Many thanks to everyone involved!

{“author”: “Philipp Hancke“}

Reverse-Engineering Blackbox Exploration, Firefox, hangouts, SDP

Related Posts

  • How does the new Azure Communication Services implement WebRTC? (Gustavo Garcia)How does the new Azure Communication Services implement WebRTC? (Gustavo Garcia)
  • How does Hangouts use WebRTC? webrtc-internals analysisHow does Hangouts use WebRTC? webrtc-internals analysis
  • How Cloudflare Glares at WebRTC with WHIP and WHEPHow Cloudflare Glares at WebRTC with WHIP and WHEP
  • Meet vs. Duo – 2 faces of Google’s WebRTCMeet vs. Duo – 2 faces of Google’s WebRTC

RSS Feed

Reader Interactions

Comments

  1. Tsahi Levent-Levi says

    December 21, 2017 at 2:29 pm

    I understood there are people who don’t understand how that screenshot was taken and how I got to the conclusion that it works.

    It is rather simple.

    I opened Firefox 57 (the latest update on my Ubuntu machine), went to hangouts.google.com, logged in with my gmail account, called fippo – and took the screenshot while talking with him

    Reply
  2. Paul Gregoire says

    December 22, 2017 at 10:40 am

    Where is the ‘ice-lite’ line in the SDP, because I don’t see it; FF is specifying trickle in the one shown. When looking at the SDP in FF in ‘about’, you’ll see the aggregated content to include the candidates, this doesn’t mean they were included in the SDP when it was sent to the server. Just my two cents for clarification.

    Reply
    • Philipp Hancke says

      December 22, 2017 at 12:14 pm

      Thanks Paul! ice-lite is in the description from the server. Updating the post…

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

  • Sponsored. Become a webtcHacks sponsor

Email Subscription

Subscribe to our mailing list

* indicates required

Twittering

Tweets by @webRTChacks
webrtcHacksguides and information for WebRTC developers

Footer

SITE

  • Post List
  • About
  • Contact

Categories

  • Guide
  • Other
  • Reverse-Engineering
  • Standards
  • Technology

Tags

apple Blackbox Exploration Brief camera Chrome code computer vision DataChannel debug e2ee Edge extension gateway getUserMedia ICE ims insertable streams ios ip leakage janus jitsi MCU Microsoft NAT opensource Opus ORTC Promo Q&A raspberry pi Safari SDES SDP sfu signaling simulcast standards TURN video vp8 w3c Walkthrough Web Audio webrtc-internals wireshark

Follow

  • Twitter
  • YouTube
  • GitHub
  • RSS

webrtcHacks · copyright © 2023