Search Results

Search Results for: tim panton

It has been a few years since the WebRTC codec wars ended in a detente. H.264 has been around for more than 15 years so it is easy to gloss over the the many intricacies that make it work.

Reknown hackathon star, live-coder, and |pipe| CTO Tim Panton was working on a drone project where he needed a light-weight H.264 stack for WebRTC, so he decided to build one. This is certainly not an exercise I would recommend for most, but Tim shows it can be an enlightening experience if not an easy one. In this post, Tim walks us through his step-by-step discovery as he just tries to get video to work. Check it out for an enjoyable alternative to reading through RFCs specs for an intro on H.264! ...  Continue reading

As WebRTC implementations and field trials evolve, field experience is telling us there are still a number of open issues to make this technology deployable in the real world and the fact that we would probably do some things differently if we started all over again. As an example, see the recent W3C discussion What is missing for building (WebRTC) real services or Quobis‘ CTO post on WebRTC use of SDP.

Tim Panton, contextual communications consultant at Westhawk Ltd,  has gone through some of these issues. During the last couple of years we had the chance to run some workshops together and have some good discussions in the IETF and W3C context. Tim’s expertise is very valuable and I thought it would be a good idea to have him here to share some of his experiences with our readers. It ended up as a rant. ...  Continue reading

Pion seemingly came out of nowhere to become one of the biggest and most active WebRTC communities. Pion is a Go-based set of WebRTC projects. Golang is an interesting language, but it is not among the most popular programming languages out there, so what is so special about Pion? Why are there so many developers involved in this project? 

To learn more about this project and how it came to be among the most active WebRTC organizations, I interviewed its founder – Sean Dubois. We discuss Sean’s background and how be got started in RTC, so see the interview for his background.  I really wanted to understand why he decided to build a new WebRTC project and why he continues to spend so much of his free time on it. ...  Continue reading

As you may have heard, Whatsapp discovered a security issue in their client which was actively exploited in the wild. The exploit did not require the target to pick up the call which is really scary.
Since there are not many facts to go on, lets do some tea reading…

The security advisory issued by Facebook says

A buffer overflow vulnerability in WhatsApp VOIP stack allowed remote code execution via specially crafted series of SRTCP packets sent to a target phone number.

This is not much detail, investigations are probably still ongoing. I would very much like to hear a post-mortem how WhatsApp detected the abuse.

We know there is an issue with SRTCP, the control protocol used with media transmission. This can mean two things:

  1. there is an issue with how RTCP packets are decrypted, i.e. at the SRTCP layer
  2. there is an issue in the RTCP parser

SRTCP is quite straightforward so a bug in the RTCP parser is more likely. As I said last year, I was surprised Natalie Silvanovichs fuzzer (named “Fred” because why not?) did not find any issues in the RTCP parser.

We actually have a bit of hard facts provided by the binary diff Checkpoint Research wherein they analyzed how the patched version is different.

They found two interesting things:

  • there is an additional size check in the RTCP module, ensuring less than 1480 bytes
  • RTCP is processed before the call is answered

Lets talk about RTCP

RTCP, the Realtime Control Protocol, is a rather complicated protocol described in RFC 3550. It provides feedback about how the RTP media stream is doing such as packet loss. A UDP datagram can multiplex multiple individual RTCP packets into what is called a compound packet. When a RTCP compound packet is encrypted using SRTCP, all of the packets are encrypted together with a single authentication tag that is usually 12 bytes long.
To make demuxing compound packets possible, each individual RTCP packet specifies its length in a 16 bit field. For example a sender report packet starts like this:

The length field is defined as

length: 16 bits
The length of this RTCP packet in 32-bit words minus one,
including the header and any padding. (The offset of one makes
zero a valid length and avoids a possible infinite loop in
scanning a compound RTCP packet, while counting 32-bit words
avoids a validity check for a multiple of 4.)

which is… rather complicated. It particular this definition means that the RTCP parser MUST validate the length field against the length of the datagram and the remaining bytes in the packet. Some RTCP packets even have additional internal length fields.

For the first packet in a compound packet length validation is usually done by the library implementing SRTCP like libSRTP. Mind you that WhatsApp probably uses PJSIP and PJMEDIA, or at least they did in back in 2015 when I took a look.

The length check for the second packet needs to be done by the RTCP library. I would not be surprised if this is where things went south. Been there, done that. And it remains a bit unclear whether the length field is validated against the remaining bytes. 1480 seems like a very odd number to check for though. At first I thought this made sense since it was 1492 minus the 12 bytes for the SRTCP tag but the maximum payload size of UDP turned out to be 1472 bytes, not 1492. So now I end up being confused again…

Don’t process data from strangers

There is another issue here. As the New York Times article said it looks like the victims received calls they never answered.

Checkpoint’s analysis ...  Continue reading

As WebRTC has matured to a state where it’s first implementations are ready for companies to launch real services around it, the readiness of various companies to adopt WebRTC has fanned out quite a bit. Some are already charging ahead as early adopters, while others are playing it conservative. Of those in the conservative camp, one of the common doubts that gives them pause is: “What about IE?”

When speaking to those interested in WebRTC, but concerned about Internet Explorer (IE), many times we’ve tried to assure them not to worry: our friends in Redmond won’t be too far behind. We often point to the undeniably significant contributions from Microsoft to WebRTC, especially considering that they bring to the table two titans of VoIP industry (Lync and Skype). We highlight some of their early IE WebRTC demos (using beta code) as signs of progress. We’ve rationalized the absence of  a Microsoft equivalent to what Chrome and Firefox are shipping, by noting the slower release cycle for IE. However, we’ve come to realize that to some, IE support is a really big deal.

Even though the Firefox and Chrome browser share has ballooned to a combined ~%60 overall (depending on which source you believe), there are certain classes of companies and industry segments where IE usage is much greater. This can be due to things like user demographic, or IT policies for end user devices. No matter how you spin it, to some, the promise of ubiquitous web based real-time communications nowhere close to a reality, until Microsoft climbs aboard. For those waiting, news last week from Microsoft would suggest that you won’t have to wait much longer for IE to catch up to the rest.

What’s the news?

Announced through Microsoft’s Skype blog, a number of juicy details surrounding their WebRTC plans were revealed:

At a high level, this news reflects Microsoft’s deep commitment to RTC on the web, and lays out their plans for the near future. This all sounds good, right? As is so often the case, the devil is in the details.

What does this mean?

Looking just beneath the surface of Microsoft’s plans here, a few details stand out as potentially having significant implications:

  • The API in IE: Imminent plans to deliver ORTC (or WebRTC 1.1 as ORTC supporters call it), in IE…but not WebRTC 1.0
  • The Codecs: H.264 for video & Opus, G.711, G.722 for audio. VP8 is not part of the deal
  • Microsoft’s interests outside the web: The importance of interoperability with “billions of existing communications endpoints”

The API in IE

Microsoft’s intentions are further clarified on their IE Web Platform Status app:

It would seem that IE intends to skip implementation of the current form of WebRTC (aka WebRTC v1.0), and jump right to ORTC. Supporters of ORTC have proclaimed it “WebRTC 1.1”, which could provoke some debate. Declaring your own plans to be the industry’s “v1.1”, while the industry as a whole has not yet finalized consensus on exactly what “v1.0” is, could certainly be seen as premature. At the same time, it would be accurate to point out that some of the concepts and mechanics of ORTC are gaining support and popularity, to the point that are being adopted by the standard “v1.0” specification, and players like Google are adding support for it in their implementations.

Call it what you will, you may recall our previous post, which points out that ORTC is backwards compatible with WebRTC, and the differences lie mostly in the browser API and signaling framework. The intention is that much of the on-the-wire protocols (STUN/ICE/TURN, DTLS-SRTP, Media stream bundling, etc.) would be the same.

While a “world wide web” without completely homogeneous browser API implementations sounds like it would be bad, it is certainly much closer to the reality we see today. Through the W3C, HTML5 aspires to deliver a state of nirvana for web developers, in which they have to write code once, and it works on all different browsers exactly the same. However, browser capability detection and handling is common practice on the web, due to the realities of browser makers in differing stages of implementation and/or interpretation of HTML5. While everyone agrees that WebRTC should strive to have all implementations be as close the same as possible…if you take pragmatic look at a world in which Microsoft does not implement WebRTC 1.0 and goes right to a backwards-compatible ORTC, it may not be so bad. Certainly it would be much worse if IE elected to use completely different on-the wire protocols, as this would certainly defeat interoperability entirely. Now with RTC on the web, as with other browser based capabilities, the work of interoperability will fall to the JavaScript developers to sort things out. Most WebRTC JS libraries already do detection for varying WebRTC implementations today. With IE ORTC on the scene, it is likely that JS libraries will evolve to detect, leverage, and normalize to their API surface, making today’s WebRTC apps operate with little modification.

One caveat here, is that the previously mentioned points about being on-the-wire compatible with other implementations of WebRTC is not entirely true (yet). This is because of one one major yet-to-be-determined item for the IETF: what video codecs shall be used. So…what does Microsoft say about the codecs?

With IE ORTC on the scene, it is likely that JS libraries will evolve to detect, leverage, and normalize to their API surface, making today’s WebRTC apps operate with little modification.

The Codecs

While the news of an upcoming IE implementation is mostly positive, the announcement could also be seen as a thinly veiled attempt to show momentum on one side of a video codec debate, that has been on the back-burner for a while. They clearly state their plans with regard to the codecs they intent to implement:  Opus, G.711, G.722 for audio and H.264 for video. No VP8.


When it comes to audio codecs, the WebRTC specification indicates both Opus and G.711 as the Mandatory-to-Implement (MTI) audio codecs for any WebRTC-complaint implementation. However, the IETF has recently adopted a document that recommends other voice codecs to interoperate with legacy networks, which includes G.722 among others.



As mentioned, Microsoft announced plans to support H.264 only which means won’t be able to interoperate with other implementations supporting VP8 only, like Google’s Chrome.

This creates a rather large rift in RTC on the web, as Chrome browsers and IE browsers will have no common codec with which to share a video call. That said, some WebRTC community members are willing to take some progress vs. no progress, saying, “even though Microsoft’s announcement is about ORTC and H.264,  it is better than nothing. At least it will provide audio and hopefully datachannels with no plugins in IE, plus video support interoperable with a large percentage of the market. One can always deploy a transcoding solution as a fallback“. I’m sure transcoding vendors will be very happy with this sentiment.

As highlighted in previous posts, the last time the video codec topic was discussed at the IETF it was not possible to reach a consensus. The decision at that time was to put the topic on-hold, to spend the cycles working on other topics in order to progress with the standardization effort. So now, after several months, the can of worms has been re-opened. Next week there will be a new IETF meeting in Honolulu, and the Video Codec topic is part of the agenda for the RTCWeb WG. It’s worth noting that within the IETF there are some voices, mostly from VP8 supporters, claiming it does not make much sense to re-discuss this now, as nothing has changed substantially since last time it was discussed (note 

VP8 is currently under consideration to become an international  ...  Continue reading

Biggie vs. Tupac. Gates vs. Jobs. Apple vs. Samsung.  Nothing catches people’s attention for no legitimate reason like a feud. Unfortunately this isn’t just a celebrity phenomenon. Feuds have been endemic even to real communications as well. From the very beginning, Elisha Gray’s dispute with Alexander Graham Bell over the original telephone patent showed the industry has a propensity for squabbles. Unfortunately we have become so accustomed to feuds that we sometimes fabricate battles that do not really exist. I fear that this is often the case with one of the most important, but misunderstood efforts affecting WebRTC’s future – Object Real Time Communications (ORTC).

We first covered ORTC more than a year ago with our interview with Iñaki Baz and Tim Panton further highlighted the pain of SDP in his post. You can navigate some of the complexities of SDP yourself using our interactive SDP tool. So, has anything really happened with the efforts to remove SDP from WebRTC, or SDP the way it will be forever?

To get the most direct truth possible,  I asked W3C ORTC Community Group (CG) Chair, Robin Raymond to set the record straight in an interview. In addition to chairing the ORTC CG, Robin is Chief Architect at Hookflash and has had a full career as a developer, architect, CTO, and CEO.

{“intro-by”, “chad“}

=&0=& Before we dig in, can you give our readers some background on ORTC – Object Real-Time Communications – and how it came about? =&1=&

=&2=&Ideas for ORTC began at Hookflash, where all 3 stakeholders came from a SIP-centric background and knew that we wanted a more adaptable protocol for our next project. There really wasn’t one that fit our needs. We began work on a P2P communications model called Open Peer. While building Open Peer, we realized that this new model was easier if it was not required to conform to the constraints of old. It widened our gaze considerably.

Based on previous experiences in the SIP world, we were concerned that an SDP offer/answer model used as the exclusive API surface for WebRTC would not only make our lives at Hookflash more difficult but would slow the industry as a whole. I don’t really want to re-iterate the reasonings as it would be redundant, but suffice it to say we felt very strongly about conveying our thoughts on how Real-time Communication on the Web might evolve. So, we got involved in the WebRTC standards discussions. One can read some of the background on that here.

=&3=&So how is this effort manifested today?

=&2=& At the heart of ORTC is a JavaScript API for performing real time communications on a browser, mobile or even on a server.The ORTC API was designed within the ORTC W3C Community Group originally founded by Hookflash. This group now has more than 70 members, including many large corporations.

The Community Group published the ORTC API Public Draft on August 20, 2014. Microsoft published a blog post publicly supporting the draft. Hookflash also issued a blog post of our own with quotes from Google and Microsoft, showing an initial glimpse of solidarity.

=&0=& So why do some think that ORTC is a Microsoft initiative?

=&2=& That is a common misconception and it is completely incorrect. Microsoft was an early supporter of ORTC and participated to help refine ORTC, as have other companies such as Google, who also actively participate in the WebRTC Working Group.

The previous CU-RTC-Web alternative proposal fiasco left some to believe Microsoft was only interested in providing “their” alternative. When Hookflash posted our concerns with the current API approach to the WebRTC API, Microsoft and others shared similar concerns.

Instead of trying to force an alternative strategy upon others – which we have no power to do anyway – Hookflash formed a W3C Community Group of companies and individuals who shared a common belief that we could help progress the WebRTC technology along orthogonally without disrupting the current WebRTC efforts. I’m happy that Microsoft decided to join such an open effort and has been actively supportive, but by no means do they “own” ORTC.

There is no conspiracy here... This isn't Betamax vs. VHS

=&0=& Is ORTC similar to Microsoft’s former CU-RTC-Web prototype? Is ORTC just a continuation of CU-RTC-Web under a different name?