WebRTC 1.0 uses SDP for negotiating capabilities between parties. While there are a growing number of objects coming to WebRTC to avoid this protocol from the 90’s , the reality is SDP will be with us for some time. If you want to do things like change codecs or adjust bandwidth limits, then you’re going to need to “munge” SDP for the time being.
At a recent WebRTC Boston, Nick Gauthier of MeetSpace described how he used SDP modification and other techniques to jam up to 10 video callers into a single conference without a media server. Not everyone has a good reason to do this, but there are certainly plenty of applications where having more precise control of your bandwidth consumption would be useful. You can see his video here or check out his technique and thorough explanation on how to munge SDP to adjust individual bandwidth usage below.
{“editor”, “chad hart“}
Without intervention, WebRTC PeerConnection will consume all available bandwidth to deliver the best possible quality. This is great when having a meeting is all you’re doing, but what about when you try to open GMail and it crawls while your WebRTC call stutters due to the new lack of bandwidth? Or perhaps you’d like to provide user-controllable quality levels for folks with poor connections. Or, in our case at MeetSpace, you’re trying to run a 10-way mesh call and the PeerConnections are simply contending with each other.
In this post, we’ll walk through how to parse the SDP payload and modify it on-the-fly in JavaScript to set a maximum bandwidth cap.
Where to Modify the SDP
The first thing we have to do is get the SDP payload. The SDP is first generated when a PeerConnection creates an Offer for a peer:
|
1 2 3 4 5 6 7 |
peerConnection.createOffer( function(offer) { console.debug("The offer SDP:", offer.sdp); peerConnection.setLocalDescription(offer); // your signaling code to communicate the offer goes here } ); |
What we’re going to do is intervene and modify the offer’s SDP payload before we pass it to the remote side:
|
1 2 3 4 5 6 7 8 |
peerConnection.createOffer( function(offer) { peerConnection.setLocalDescription(offer); // modify the SDP after calling setLocalDescription offer.sdp = setMediaBitrates(offer.sdp); // your signaling code to communicate the offer goes here } ); |
Here, we’re calling a new function
setMediaBitrates that is going to modify the SDP payload and return it back to us. (I’ll explain this in the next section after finishing the Offer-Answer process.)
Note that we are not allowed to modify the offer between createOffer (or createAnswer) and setLocalDescription so we do the modification before signaling it to the remote side.
When the offer reaches our peer, it also needs to modify the SDP when it generates its answer. This is because the offer says “This is the bandwidth I can handle” and the answer also says “Well, this is the bandwidth I can handle” so if you want both sides to be limited, you have to modify the answer SDP too. It’s very similar:
|
1 2 3 4 5 6 7 8 |
peerConnection.setRemoteDescription(new RTCSessionDescription(offer)).then(function() { peerConnection.createAnswer().then(function(answer) { peerConnection.setLocalDescription(answer); // modify the SDP after calling setLocalDescription answer.sdp = setMediaBitrates(answer.sdp); // your signaling code to communicate the answer goes here }; }; |
Now that our hooks are in place to modify the SDP, we can get started!
Parsing the SDP
I highly recommend the post Update: Anatomy of a WebRTC SDP (Antón Román) because that’s where I got started too. That will give you a good overview of what the SDP is and what it looks like.
Next, I like to go right to the spec: RFC 4566 SDP: Session Description Protocol. That is a link right to page 7, section 5, which describes the SDP format at a high level.
The TL;DR is that the SDP is a UTF-8 chunk of text broken up into lines with each line being <type>=<value>.
The other really important part is a little further down within section 5: the ordering of types. I won’t repeat all of them here, but the part we need is that there’s a section at the beginning of the SDP, followed by repeated media descriptions. The media descriptions always go m, i, c, b, k, a.
The next place we need to look is in RFC 3556 Session Description Protocol (SDP) Bandwidth Modifiers for RTP Control Protocol (RTCP) Bandwidth. This RFC describes how to set bandwidth by using a b line, with b=AS:XXX where the XXX is the bandwidth we want to set. The AS part means it’s “Application Specific Maximum” which means it’s limiting the total bandwidth. Also from this RFC we can see that the bandwidth value should be in kilobits per second (kbps).
That means our code should do this:
- Skip lines until we find m=audio or m=video (depending on which one we’re setting)
- Skip i and c lines
- If we’re on a b line, replace it
- If we’re not on a b line, add a new b line
Modifying the SDP to Add a Bandwidth Constraint
Most of the time in WebRTC video calls, there’s a media description for video, and a media description for audio. In our example, we’ll limit video bandwidth to 500kb/s, and audio bandwidth to 50kb/s. Let’s check out the code:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
function setMediaBitrates(sdp) { return setMediaBitrate(setMediaBitrate(sdp, "video", 500), "audio", 50); } function setMediaBitrate(sdp, media, bitrate) { var lines = sdp.split("\n"); var line = -1; for (var i = 0; i < lines.length; i++) { if (lines[i].indexOf("m="+media) === 0) { line = i; break; } } if (line === -1) { console.debug("Could not find the m line for", media); return sdp; } console.debug("Found the m line for", media, "at line", line); // Pass the m line line++; // Skip i and c lines while(lines[line].indexOf("i=") === 0 || lines[line].indexOf("c=") === 0) { line++; } // If we're on a b line, replace it if (lines[line].indexOf("b") === 0 { console.debug("Replaced b line at line", line); lines[line] = "b=AS:"+bitrate; return lines.join("\n"); } // Add a new b line console.debug("Adding new b line before line", line); var newLines = lines.slice(0, line) newLines.push("b=AS:"+bitrate) newLines = newLines.concat(lines.slice(line, lines.length)) return newLines.join("\n") } |
And that’s it! Honestly, when I first confronted the SDP I was really overwhelmed by it. There are so many different parts to it that it felt insurmountable. But at the end of the day, it’s a series of lines and each line is there for a different reason, and if you break it down it’s actually pretty approachable.
We didn’t need regular expressions (in fact, that would be a bad idea for this kind of grammar) because all the sections are in a nice order. And in our case, we wanted to replace a b line fully if it wasn’t there, so we didn’t have to do any parsing of a complex line.
I hope this helps you limit bandwidth on your WebRTC calls, and also helped you feel more comfortable with the SDP and also the RFC specifications that describe it.
{“author”: “Nick Gauthier“}
Got this from Nils (Firefox team): The blog post should mention that b=AS only works with Chrome. For Firefox you need to look for b=TIAS to accomplish the same.
Hey @Victor,
Are you sure about that point?
I currently seems to “work” for me on Firefox 51 & 52 ;
At least it doesn’t trigger any error when setting the updated RTCSessionDescription as remote description.
Maybe it doesn’t have any effect on bandwidth, though? How to mesure it?
And I am wondering, as “b=AS” is following the RFC 3556, what would be the point for Firefox not to follow it?
Hey,
Thanks for this tip,
If you’re interested, here is an ES6 rewrite of the same method 🙂
Feel free to use it if you wish.
setMediaBitrate(sdp, mediaType, bitrate) {
let sdpLines = sdp.split(‘\n’),
mediaLineIndex = -1,
mediaLine = ‘m=’ + mediaType,
bitrateLineIndex = -1,
bitrateLine = ‘b=AS:’ + bitrate,
mediaLineIndex = sdpLines.findIndex(line => line.startsWith(mediaLine));
// If we find a line matching “m={mediaType}”
if (mediaLineIndex && mediaLineIndex < sdpLines.length) {
// Skip the media line
bitrateLineIndex = mediaLineIndex + 1;
// Skip both i=* and c=* lines (bandwidths limiters have to come afterwards)
while (sdpLines[bitrateLineIndex].startsWith('i=') || sdpLines[bitrateLineIndex].startsWith('c=')) {
bitrateLineIndex++;
}
if (sdpLines[bitrateLineIndex].startsWith('b=')) {
// If the next line is a b=* line, replace it with our new bandwidth
sdpLines[bitrateLineIndex] = bitrateLine;
} else {
// Otherwise insert a new bitrate line.
sdpLines.splice(bitrateLineIndex, 0, bitrateLine);
}
}
// Then return the updated sdp content as a string
return sdpLines.join('\n');
}
Thanks for sharing!
With this we limit download bandwidth or upload bandwidth or both? If answer is both. Is there a way to control each one of them individually?
Yes.
I did this and although I set the lines correctly (according to the sdp output I do afterward) the following happens:
As soon as the AUDIO bandwidth is changed, the VIDEO bandwidth goes to the same bandwidth (according to the CHROME graphical analsys at chrome:webrtc-internals).
I can repeat this behavior anytime Chrome 57 (PC) connected to Chrome 57 (Mac). If I connect from Chrome 57 (Mac) to Vivaldi, the outcome is as expected and correct.
Anybody suggestions?
PS: I am changing the bandwidth on the fly: When the connections first is established, audio and video bandwidth are set correctlys. When I change audio BW during a connections, BOTH audio and video bandwidth change to the value set for Audio BW, although correct value for video is given and inserted.
Hey Armin,
Sorry but I haven’t tried the on-the-fly adjustments yet. I’m waiting for applyConstraints to drop for chrome so I can adjust frame rate and size at the same time. I dunno how much it’s worth to only adjust bandwidth without adjusting frame size and height (although I tend to be more concerned about cpu usage).
Here is the remoteDescription (chrome connecting to firefox, but it is the same with Chrome to Chrome).
v=0
o=mozilla…THIS_IS_SDPARTA-52.0.1 306592031755684620 0 IN IP4 0.0.0.0
s=-
t=0 0
a=fingerprint:sha-256 8C:E4:68:9A:0C:DA:15:DC:FC:18:92:02:D2:93:9A:81:D0:BE:BF:86:99:EC:5E:CD:C8:5F:99:78:9F:DA:AA:32
a=group:BUNDLE audio video
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 9 UDP/TLS/RTP/SAVPF 111 126
c=IN IP4 0.0.0.0
b=AS:160
a=sendrecv
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=fmtp:111 vbr=1; minptime=10; ptime=20; useinbandfec=1; usedtx=0; sprop-maxcapturerate=48000; maxplaybackrate=48000; maxaveragebitrate=320000
a=fmtp:126 0-15
a=ice-pwd:aeb7995cdaf18ceaae9ca2225bf8ba67
a=ice-ufrag:680cb6e6
a=mid:audio
a=msid:{0b6ba407-0b05-2a4c-987d-16932f6b46a0} {f9c2bca5-a6e4-f742-b4e2-ce31d8e97315}
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtpmap:126 telephone-event/8000/1
a=setup:active
a=ssrc:2245107761 cname:{64d3d89b-9c3c-904d-bd59-060fb2c89417}
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
b=AS:250
a=sendrecv
a=fmtp:96 max-fs=12288;max-fr=60
a=ice-pwd:aeb7995cdaf18ceaae9ca2225bf8ba67
a=ice-ufrag:680cb6e6
a=mid:video
a=msid:{0b6ba407-0b05-2a4c-987d-16932f6b46a0} {c4854cab-8bc6-3a4a-abc3-c700cafa228c}
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 goog-remb
a=rtcp-mux
a=rtpmap:96 VP8/90000
a=setup:active
a=ssrc:1569125431 cname:{64d3d89b-9c3c-904d-bd59-060fb2c89417}
As you can see, video should be (stay) at 250 but it will drop to 160 (same as Audio BW) as soon as remoteDescription is set.
I get the EXACT same remoteDescription on connecting but there it works.
Any idea? Highly appreciated!
Hey guys, I know this post is a bit old, but I don’t know if someone could help me answering some simple questions about this. I am just an enthusiast of webRTC and I am just starting so I apologize if some questions may seem silly or stupid.
At this moment this is still the only way to control the bandwidth of a webRTC session? I ask this because in the article ORTC is mentioned, what happens with it? there is another ay to manipulate the BW?
Another question that I would like to do is, what happens if I change the bandwidth limit but I do not change the camera resolution through the getUserMedua() method? for example, what happens if I have a resolution of 320×240 and I am using a good bandwidth to transmit that resolution, and then I inrease the bandwidth limit for the video? It should stay in the old resolution, right? could anybody confirm it?
I appreciate all the help and thanks in advance for all the possible answers.
Regards.
Hi Eliomar – you should see the example here: https://webrtc.github.io/samples/src/content/peerconnection/bandwidth/
Hey Chad! Thanks for your Reply, it was very useful to learn a lot of things.
In that code they do the bandwidth control with both methods SDP and ORTC, but in that code I think they are handling the bandwidth in general and not for each media (Audio and Video), I tried to enable Audio in that demo (because it was disable) and when I modify the bandwidth it has an strange behavior with BW, the AudioBW is the one modified and the VideoBW goes the lowest value possible. I am guessing that it could maybe a confusion between the bandwidth of Video and Audio.
Do you know where can I read more about how to modify the bandwidth separately with ORTC? Just like here it is made with SDP for Audio and Video Separately?
By the way, I am testing this code and it seems to have some errors to work properly.
Thanks in advance for your answer Chad.
Regards.
Great post, thanks so much for your help.
I modified the SDP offer before sending it to the remote peer, but I don’t have easy access to that peer so I modify the SDP answer that I receive before calling setRemoteDescription(new RTCSessionDescription(answer)) on my local peer.
I also stumbled trying to set the kbps to 5 instead of the 50 you used in the article. I wanted to make sure I could tell the difference, but 5 was too low to be negotiated properly. I could hear the difference with 50 just fine.
What are the minimum bitrate values I can set for both audio and video? Should I also edit the constraints when calling getUserMedia to achieve a lag-free call? I want to get decent audio quality while maintaining a lag-free connection.
In general, you should let the WebRTC stack use its available bandwidth estimation mechanisms to manage the bitrate automatically. Lowering the available bandwidth to something too small will limit the quality. Often times you can adjust your UI to use smaller dimensions to limit bandwidth usage too.
Also, note there is an official sample for adjusting bandwidth as an additional reference: https://webrtc.github.io/samples/src/content/peerconnection/bandwidth/