We had a lot of traffic to Victor’s post on the WebRTC mandatory video codec earlier this week. Given the news from Cisco yesterday we figured this warranted a quick follow-up post beyond what we could add to the comments area.
Quick debate recap
Engineers don’t like lawyers, and as Victor mentioned in his post earlier this week, much of the debate over assigning a mandatory video codec for WebRTC has been about avoiding the lawyers. While debate over the technical merits of H.264 vs. VP8 yielded no overwhelming winner (they are both great codecs), the debate has more recently revealed it’s true form as a mostly IPR related issue. The H.264 camp speculated that there could be legal issues with VP8 despite Google’s claims otherwise. There are certainly inherent issues with H.264. Which one has more risk? It would take lots of lawyers to sort through this and no one pays for lawyers to go to standards meetings. Even if they did, it wouldn’t matter – lawyers use arbitrators and the legal system, not technical standards procedures to work through disagreements.
Unfortunately the attempts to avoid the lawyers have failed. One of the most significant barriers preventing use of H.264’s use as a Mandatory To Implement (MTI) codec for WebRTC, as stated by many vocal H.264 opponents (see passionate comments on webrtchacks), has been the patents and licensing fees associated with it.
Cisco announced this morning that they intend to remove this barrier by contributing open source H.264 implementations and absorbing the MPEG-LA licensing fees. In concert with this announcement, Mozilla has acknowledged the contribution, and announced they will be delivering H.264 to Firefox in the near future.
How does this change things?
What Cisco is attempting to do is bear the brunt of the legal battle by assuming responsibility for H.264 royalties under specific circumstances. At a high-level, this is not much different than what Google attempted with VP8. The main difference is there are 8 other major backers of H.264 and hundreds of millions of devices (maybe Billions) that already support H.264.
Materially, the possible outcomes in the debate remain the same. However, Mozilla now appears to be in the column in support of the H.264 proposal (or possibly H.264 & VP8), and the legal implications and licensing obligations of WebRTC’s use of H.264 are more aligned with the principles of the open web.
Who cares about H.264?
While theories, accusations, and speculation abound, the facts are that many of the H.264 proponents are those that have invested money, and mounds of high quality engineering in IP based real-time communications outside of the web for many years now. The result of that investment is that most IP based video systems and services use H.264 today, and this is not easily changed. When you consider that most mobile devices contain dedicated hardware support for accelerating the encode/decode operations for H.264, it represents a massive deployment that can’t simply be updated at the flip of a switch.
While some argue that existing video-over-IP implementations are, or should be separate from the web, many would like to be able to interoperate between web and non-web applications at a reasonable cost. Most agree that the notion of transcoding between H.264 and VP8 for sessions that cross this divide would be a very cost prohibitive proposition at scale. Those who build outside the web but want to interoperate with WebRTC are faced with either implementing the tools and technologies of WebRTC in their non-web applications, or making use of gateway functionality within elements they have already deployed .
If VP8 became the one and only MTI video codec for WebRTC, the implications are huge for those web outsiders that want to allow their consumers the ability to have real-time-communications to/from the web. In the end, the price of floating the cost for a “free” Web Real Time Communications, may be significantly less than overhauling all the established video RTC work that has been done to date.
How will this work?
Cisco is going to provide a codec binary module and open source everything. Mozilla is going to bake this into Firefox. Other developers can incorporate the binary module into their code just as Mozilla is doing and Cisco will pay the charge. Alternatively they can utilize the source code to build their own binary, but Cisco is not covering the royalty charges for this. The binary has to come from Cisco for them to cover the licensing charges. Implementors (browser makers) will need to connect to Cisco.com and trust their compilation of the open source project. Ideally, this will be transparent to web developers, and nothing will change in how they use the W3C WebRTC APIs. The promise would be that web apps running on browsers containing Cisco’s openH264 implementation will have any associated licensing fees covered by Cisco, no strings attached.
The Good, the Bad, and the Ugly
The Good news for the WebRTC community is that Cisco just lowered one of the major barriers to adoption of H.264. This is going to cost Cisco real money based on statements they have made – money that others might have had to pay. This move also increases H.264. Cisco should be commended for taking bold action to try something new for the benefit of WebRTC, even if there are many that disagree on the debate.
The bad news is that everyone really wants a free and unencumbered codec as opposed to a generous hand-out that could easily be rescinded by a public company that is still subject to the whims of its shareholders. This mechanism also definitely will not work with Apple iOS according to the Cisco openh264.org Q&A…but then again, Apple already licenses H.264 on it’s devices. The downloaded binary module approach may be problematic for other platforms too. It is unclear how this will leverage underlying hardware acceleration, if at all. It is also not available today. I think it is safe to assume Cisco will put the necessary resources behind this to get a quality module to market quickly, but there is risk there.
The ugliest part is that this does not end the debate. It does not bring the community together. Based on the IETF mailing list discussion today, opponents to H.264 are not phased by the move, claiming it does not go far enough. Certainly it does not remove the lawyers from the equation. It leaves open new issues – some have already cited security concerns about relying on an unauditable binary (i.e. what if the NSA inserted something between in the compilation process?). It also does nothing to establish precedent for future codec battles – namely VP9 vs. H.265. We will have to wait until next week to see what this really means.
The MTI codec outcome that may be strengthened the most by a move where VP8 and H.264 are both chosen as mandatory. H.264 proponents will have done just enough to alleviate IPR concerns, while achieving interoperablity. VP8 proponents will still have leverage over any patent trolls wanting to stir up trouble through future use of H.26x. In the future, any side that proves to be an un-trustworthy citizen of the open web can be “voted off the island”, so to speak.
As is typical with debates that are so polarizing, it can be challenging to remain objective and pragmatic when it revolves around matters which are qualitative instead of quantitative. It is ironic that perhaps this battle is less about what happens in the immediate future on the web, and more about jockeying for future position, and RTC that is outside the web. What happens within WebRTC will undoubtedly shape the future of RTC in the existing telephony domain. Surely larger forces are in play here.
Many questions are going to remain open:
Are the large H.264 proponents unfairly protecting their interests and keeping a stranglehold on RTC outside the web going forward?
Is Google attempting to undermine and weaken other tech giants under the guise of open and “free” technology by sealing off interoperability to those that are already significantly invested?
What if Cisco’s generosity runs dry, providing MPEG-LA some way to corner future licensing fee gotchas H.264?
What if Google’s generosity runs dry, and they start charging for VP8 (9)?
What about H.265 vs VP9?
Who do I trust for an unknown future?
We’ll try to leave you to answer those for yourselves. If you already have your mind made up about the integrity and motivation of the major players involved in this debate, this move may do little to dissuade. At a minimum however, this news serves as a potential major contribution to WebRTC from Cisco and those supporting H.264. As the MTI debate plays out, certainly this makes H.264 a much more attractive option for inclusion as a WebRTC component. Simply focusing on the here-and-now, this aims to mitigate concerns about down-the-road licensing fees against implementors of WebRTC, while ensuring interoperability outside the web.
Lawrence Byrd says
In the scheme of things where it is probably impossible for MPEG LA to just give up and “go free” on behalf of their complex web of patent pool members, this seems to be he biggest move possible in opening up H.264 – so I think Cisco is to be congratulated regardless of any “is this enough” opinions! A bold step, it seems to me, and also perhaps a challenge to Apple (in particular) and Microsoft to make H.264 more open to WebRTC developers using their platform – it shouldn’t all be on Cisco, after all, to solve this dilemma of balancing integration with existing video systems versus open license-unencumbered codecs.
Chad Hart says
Thanks for the comment Lawrence. I agree some kind of positive statement from the MPEG LA on this would be ideal, but that will never work in the litigious patent system we have today. It is impossible to get assurances on anything. The next best thing is to get a major patent holder on your side to provide some cover – that is exactly what Open H.264 with Cisco does.
I am eagerly waiting to see how this all plays out in Vancouver…
Lorenzo Miniero says
That’s exactly the point. If the MPEG LA doesn’t “free” H.264, then it is indeed not enough. Certainly not to impose a codec as MTI. I’ve said many times why I don’t think so, so I won’t state them again here. I’ve tried to raise my concerns on the RTCWEB ML as well,
just to be ignored so far: I guess that I’ll either have to be hired by a big company, or wait for my tiny startup to be acquired from one, before my questions will be deemed worthy and answered. May take a while…
Reid Stidolph says
In this case (since it is RTC-Web), it seems “Manditory to Implement” really only applies to browser vendors, of which there are just a few. In other words, in reality there will be a relatively small number of “Implementations”, but much of the concerns are from non-browser implementors (big and small) that want to inter-operate with browsers. I assume you fall into this category Lorenzo? Do you think it might be feasible to have both VP8 and H.264 implemented in browsers, but allow constraints on them via the API? So that the actual services and applications could enforce what exactly they use, and the browser would inter-operate with non-browser implementations of either…and after all, it’s MTI, not “Manditory to Use”. I’m curious what your take on this approach would be.
Lorenzo Miniero says
Yes, that’s the category I fall in, and what a lot of other small developers and companies have been doing while H.264 advocates were busy doing something else.
MTI applies to WebRTC-compliant applications in general. While it is of course much more important for browsers, it inevitably falls back on applications that do interact with them anyway. If H.264 is MTI and VP8 or whatever else is not, while Google and (maybe, I don’t know what to think about them anymore) Firefox will probably support VP8, Microsoft and Apple will definitely not deign other codecs of their consideration. Which means that your server side application that does streaming, conferencing, or whatever else may happen to generate content will need to talk H.264 if it wants to talk to ALL browsers. Just implementing VP8 would mean not support browsers that decided not to bother with it. If you’re only passing media through, instead, that’s not really an issue.
Lorenzo Miniero says
Reading your question again, maybe you were actually asking whether or not I’d be ok with both MTI. I guess I could be, as it would allow me to neglect H.264 entirely and still be confident that VP8 would be available: I don’t think that will ever happen, sadly, though.
Reid Stidolph says
Good point Lorenzo. If the browser folks (Appl, MS, Moz, Goog) don’t completely adhere to MTI (whether VP8, H.264, or both) and remain divided, then interoperablity is broke for everyone, even the “web”. Regardless, at least one of the browser vendors are going to have to implement a codec they don’t want, to achieve basic web-web interoperability. If one side or the other (or both) can budge on that…then it becomes about web-nonweb interop, and if you satisfy the larger establishments or the emergent ones (or both).
Victor Pascual says
Google: “Congratulations on the Cisco announcement – but we still prefer VP8” http://t.co/JJKkZrUmn9
From: Harald Alvestrand
To: “rtcweb at ietf.org”
Date: Thu, 31 Oct 2013 19:47:31 +0100
List-id: Real-Time Communication in WEB-browsers working group list
We congratulate Cisco on their intention to make an open source H.264 codec available and usable by the community. We look forward to seeing the result of this effort.
Google still believes that VP8 – a freely available, fully open, high-quality video codec that you can download, compile for your platform, include in your binary, distribute and put into production today – is the best choice of a Mandatory to Implement video codec for the WebRTC effort.
Harald (sending this from my Google address)
Lawrence Byrd says
Well Google is absolutely right in the sense that a completely open, licence-unencumbered good quality video codec is ideal for the open web – which is VP8 becoming VP9 :)! And it is backed by a large vendor willing to fight ll the extraneous lawsuits against it. However the real world is full of vendors with IETF votes and sizable current and future H.264 (going to H.265) investments and lots of existing enterprise video systems much of which will want to be integrated with WebRTC endpoints, at least in near-term paid-for use-cases before WebRTC takes over the planet. Hence the exciting upcoming real-world “Mixed Media Arts” (MMA) fight in Vancouver! Get your tickets…
Robert Welbourn says
Or should we just give up on mandatory-to-implement? Let’s play out the scenarios:
1. W3C sticks with MTI, and no agreement emerges on the choice of video codec. Roll-out of WebRTC is slowed, until non-Google browser vendors go ahead, ignoring MTI, and bring out WebRTC implementations with only H.264. Mozilla straddles the fence and supports both codecs. Mobile SDK vendors vote with their feet and leverage native chipset support for H.264.
2. W3C sticks with MTI, and VP8 is chosen. Enterprises wishing to deploy WebRTC have to stock up on VP8-to-H.264 transcoders, increasing cost and reducing the take-up of the technology.
3. W3C sticks with MTI, and H.264 is chosen. This, I might suggest, would encourage Microsoft and Apple to come down off the fence and embrace WebRTC, despite Microsoft’s misgivings over the browser API.
4. W3C sticks with MTI, and both codecs are chosen. Developers of new services can embrace VP8 and forget about royalty payments — until some non-member of MPEG-LA comes along asserting a patent claim. Some mobile SDK vendors may implement only H.264, but no big deal, because their apps are doubtless tied to legacy infrastructure in any case.
5. MTI is waived, and the market chooses H.264.
From where I stand (working for a Cisco partner, I should disclose) any solution that does not support H.264 would be a disappointment. While we do support video transcoding, that’s a cost that our customers have to bear.
Robert Welbourn says
My last post should, of course, have referenced the MTI discussions as taking place within the IETF rather than the W3C.
Victor Pascual says
MPEG Liaison Statement to IETF RTCWeb (Liaison on Video Coding for Browsers) http://webrtchacks.com/wp-content/uploads/2013/11/29n138201.doc
Dean Bubley says
“the objective of developing a standard where patent owners are prepared to grant a free-of-charge license (Type 1 license in ISO parlance) to an unrestricted number of applicants on a worldwide, non-discriminatory basis”
That sounds promising, at least in the medium term….
Victor Pascual says
VP8 acceleration hits the mobile phone in time for IETF88 http://disruptivewireless.blogspot.se/2013/11/vp8-acceleration-hits-mobile-phone-in.html?spref=tw
Victor Pascual says
Brief 5 minute video explaining the open source and binary module distribution of OpenH264.org