Well, that title is clickbait for Hacker News. The honest title would have been “using AI to tackle long-standing WebRTC issues.”
Posts on webrtcHacks are rare these days, and in German we would say there are Unkenrufe that WebRTC is dead. It is not. It is the infrastructure for meetings, game streaming, and voice AI, as OpenAI laid out recently when they walked through how their stack handles real-time audio at their scale.
Posting less often here does not mean I stopped being involved in WebRTC. Over the past years I have gotten involved more deeply in libWebRTC than ever before, while also doing work across the wider ecosystem. These days I work on rtcstats.com, which does monitoring for WebRTC, with two friends, taking a break from corporate life.
I just fixed webrtc-internals gets throttled while in the background, an issue I had filed in 2022. We had been running into it more often from rtcstats.com users and it was becoming annoying. A “setInterval” in Javascript requesting WebRTC getStats got throttled after minutes of the page being in the background, which caused gaps in timeseries and graphs.
So I changed my attitude and said: “hey Claude, can you move this polling to C++?”
I reviewed the change, tweaked it (not much, it was solid), uploaded it, and reviewers liked it. It landed in Chrome 150.
I started using Claude and Gemini to be more efficient. Going deeper with libWebRTC work, some bigger projects like cryptex, taking what I built into Go, and chipping away at Firefox. While making more steps on my daily walks.
Keep out of my codebase, mate
The initial results of genAI on large codebases were “meh”. libWebRTC and Chromium have conventions, internal abstractions, and reviewers who will (correctly) push back on anything that looks like “slop” or as if the author does not understand the current state and the long-term impact and responsibility. Google’s WebRTC team only started to work (publically) on Improving AI agent support for libwebrtc development at the end of 2025.
WebRTC and libWebRTC also have a lot of oral lore. Things you do and don’t do, and why things are the way they are, that nobody wrote down. People who have been around for years know it. New contributors do not. This is bad and we should get better at it, not because LLMs need it as input, but because humans do. The fact that LLMs would benefit too is a side effect (or a convenient excuse to sell AI to management).
So I continued the work I had been pushing for, which was consistency. I had picked up IWYU adoption in libWebRTC and followed up with clang-tidy. Operating on the gut feeling that we need to be more consistent and strict (and the reviewers felt the same). This paid off, honing those tools makes life easier for every tool that comes after, AI included.
Something changed in 2026. The models got better. Finally things worked as advertised.
AI coding works in libWebRTC and Chromium because both have a ton of guardrails. The culture encouraged more tests, including test-only changes, which I land quite often. Using AI works on top of decades of test discipline and the cultural expectation that you add to it rather than just consume it. And we finally got back release notes, a job nobody wanted to do in a long time!
cryptex: who can say “no” to more encryption
Some work is about timing. I have always been drawn to the networking parts of libWebRTC, like SRTP and DTLS, so when RFC 9335 shipped in 2023, it was on my radar. It closes a small but important gap in WebRTC’s cryptography by extending encryption to RTP header extensions, and the more crypto the merrier. One of the things this protects is the ssrc-audio-level extension, which is widely used in conferencing. It is a measure of how much audio energy is in this RTP packet with 20ms of audio data, and Jitsi uses it to determine who is speaking right now. It is one byte of information, sent unencrypted but authenticated. No known attacks exist, but we all are a bit uncomfortable doing this. OpenAI does not negotiate this extension at all.
libSRTP shipped support recently. We just got unblocked, why wait longer. I have been doing the libWebRTC side in SRTP cryptex support in libWebRTC. Some of it was IETF list questions where the right answer was not fully clear to me or my favorite reviewer. We both prefer a paper trail for accountability. Those came out of a pion conversation, and the answers fed back into the libSRTP work.
What it actually takes to deliver this kind of change is driving in multiple venues with different stakeholders, taking lessons from one codebase to another, and improving the specification along the way. The skill being applied across all of those venues is the same one: knowing what should change, why, and what “good” looks like in each codebase. The thing that changed is that I am now in a position to do the implementation in each of them, not just file the bug and hope.
Claude was happy to split up the patches on reviewer feedback, something I always pushed back on slightly because it cost me time. Now… it is on Claude’s.
SNAP/SPED in an alien codebase
WebRTC is old. It was the latest and greatest on how to do things in 2011, and we have learned a bit since. Call setup time has always been an issue, and we even voted in a way that increased it in 2013. This always bugged me. Then I had an idea. 250 lines of code written by me later, I had a prototype.
I proposed the work upstream in libWebRTC: massively reducing call setup time using SPED and SNAP and. I poked Justin Uberti who is libWebRTC Urgestein, now at OpenAI, and call setup time is a shared passion – see here for how I took his Google Duo app apart on stage at KrankyGeek 2016). OpenAI uses pion, a Go WebRTC implementation as we just learned from the blog post.
Which led to the question: “So we need it in pion…” I do not write Go and I remain a skeptic of the language. My C++ jam is libWebRTC/Chromium C++.
But Gemini gave me the confidence to say: I can pull this off. The work is tracked in this pion issue. Getting a prototype was easy. It works, yay. Splitting it up into reviewable patches and getting it through review is, unsurprisingly, the hard part of the job. I got pulled into pion as a community, not just as a codebase, and I even joined their Discord. Welcoming bunch and totally made Sean‘s day.
Gemini helped me write idiomatic Go when I told it what I wanted in C++ as my language. It let me apply taste I had developed over a decade in libWebRTC, knowing what the network packets should look like, to an ecosystem where I do not have the Go muscle memory.
I said yes and it was a great experience. Pion’s repositories were unfamiliar codebases for me. I knew the domain, but recognizing it in the code required effort. And then there is the manual work: splitting changes up based on reviewer feedback, rebasing, retrying. That used to be on my time. Now it is on Gemini’s.
Advancing Firefox, and retiring adapter.js
adapter.js is an old ugly hack that ended up in my lap. Originally created in 2013, I took over maintenance some time in 2015. It is downloaded 2 million-plus times a week from npm, which is great and sad in equal measure. It has been called an “ugly hack” on stage, which was odd since it did the job: letting developers write WebRTC code that adheres to the spec.
Two recent Firefox changes that retire pieces of it: do not throw when calling getStats on a closed peerconnection, and implement (some of) RTCTransportStats which makes a decade old Firefox workaround in the samples unnecessary.
Firefox sits on top of libWebRTC, so fixing an issue requires a slightly different skillset and contribution mechanics. The cost for me to fix the bug and then provide some backward compatibility in adapter.js is now much lower. But a lot of the responsibility remains on the reviewers, some of whom I have known for a decade. Happy belated anniversary, Andreas!
Minions to the rescue
What is cool about these new minions is that they come without the responsibility of being an engineering manager. Used to be that if you wanted to assign work to someone, you had to first earn the right to do that by becoming their manager. Now I just open a tab. I can give them a task and review the output later. They make me ship more.
The constraint that used to bind me was implementation cost in unfamiliar territory. libWebRTC and Chromium idioms are bread and butter for me, which is why I work in both daily. Firefox’ WebRTC, while C++, is different from Chromium/libWebRTC (despite using the latter). Go was completely alien.
The cost of context-switching across the unfamiliar parts was always high, high enough that most cross-stack issues stayed on someone’s mental TODO list forever. The constraint now is taste. I have been good at doing the work that needs to be done and impatient enough to drive it myself when I did not want to wait. I wait less now.
None of this is “AI replaces the engineer.” Models change too. All of it is “an engineer with context and a clear sense of what should change can now operate across more of the stack than before.”
We knew what to do
We knew WebRTC had problems. Some we could solve, others we could not. The bottleneck has been “the person who knows what to do does not have a week to do what is necessary to solve the problem.”
That problem is meaningfully smaller than it was two years ago. The bar for implementation has lowered. And yet shipping is still hard and takes time.
{“author”: “Philipp Hancke“, “copyminion”: “Claude”}
Leave a Reply