Earlier this month, OpenAI released the GA version of its realtime API. This includes many capabilities that the Beta didn’t have, including video support. I started out doing an update to the The Unofficial Guide to OpenAI’s Realtime WebRTC API I made for the Beta release last November. I discovered there were enough WebRTC updates […]
openai
OpenAI & WebRTC Q&A with Sean DuBois
OpenAI is utilizing WebRTC for its Realtime API! Even better, webrtcHacks friend and Pion founder Sean DuBois helped to develop it and agreed to a Q&A about the implementation. It is not often a massive WebRTC use case like this emerges so rapidly. In addition, Sean was extremely transparent about his work at OpenAI. In […]
Measuring the response latency of OpenAIs WebRTC-based Realtime API
As Chad mentioned in his post last week, we have been diving into what OpenAI is doing with WebRTC. Over the last months, we actually did a full teardown and compared OpenAI’s Realtime API to what powers chatgpt.com. What intrigued us most was how to measure response latency. One of the key metrics for any […]
The Unofficial Guide to OpenAI’s (Beta) Realtime WebRTC API
EDITOR NOTE: OpenAI release the GA version of the Realtime API with WebRTC. This post covers the Beta version-only. The GA version is fully backward compatible and everything here is still accurate – just don’t mix approaches. I will have an gpt-realtime GA version update / version soon! OpenAI using WebRTC in its Realtime API […]