There are a lot of options for reading and changing the pixels inside a video frame. In this post, W3C specialists François Daoust and Dominique Hazaël-Massieux (Dom) review every web-based option for processing video frames on the web available today – JavaScript, WebAssembly (wasm), WebGPU, WebGL, WebCodecs, Web Neural Networks (WebNN), and WebTransport.
Technology
Real-Time Video Processing with WebCodecs and Streams: Processing Pipelines (Part 1)
WebRTC used to be about capturing some media and sending it from Point A to Point B. Machine Learning has changed this. Now it is common to use ML to analyze and manipulate media in real time for things like virtual backgrounds, augmented reality, noise suppression, intelligent cropping, and much more. To better accommodate this […]
coturn: No Time to Die – Q&A with new project leads
New coturn project leads Gustavo Garcia and Pavel Punsky give an update on the popular TURN server project, what’s new in STUN and TURN standards, and the roadmap for the project
Revealing mediasoup’s core ingredients: Q&A with Iñaki Baz Castillo
I interviewed mediasoup’s co-founder, Iñaki Baz Castillo, about how the project got started, what makes it different, their recent Rust support, and how he maintains a developer community there despite the project’s relative unapproachability. mediasoup was one of the second-generation Selective Forwarding Units (SFUs). This second generation emerged to incorporate different approaches or address different use cases a few years after the first generation of SFUs came to market. mediasoup was and is different. It is node.js-based, built as a library to be part of a serve app, and incorporated the Object-oriented approaches used by ORTC – the alternative spec to WebRTC at the time. Today, mediasoup is a popular SFU choice among skilled WebRTC developers. mediasoup’s low-level native means this skill is required.
Calculating True End-to-End RTT (Balázs Kreith)
Balázs Kreith of the open-source WebRTC monitoring project, ObserveRTC shows how to calculate WebRTC latency – aka Round Trip Time (RTT) – in p2p scenarios and end-to-end across one or more with SFUs. WebRTC’s getStats provides relatively easy access to RTT values, bu using those values in a real-world environment for accurate results is more difficult. He provides a step-by-step guide using some simple Docke examples that compute end-to-end RTT with a single SFU and in cascaded SFU environments.