What is a realtime voice API?

A realtime voice API holds a live, two-way audio session over one WebSocket connection: your app streams microphone audio to the model and plays its spoken reply as it arrives. The user can interrupt the reply at any time, and the same endpoint also serves live transcription.

Model types

What a realtime voice API is

Most AI APIs follow a request and response pattern: you send a complete input and wait for a complete output. A realtime voice API keeps a connection open instead. Audio flows in both directions at once over a WebSocket, so your application can stream a user's microphone to the model while the model's spoken reply streams back. Either side can speak at any time, so the user can interrupt the model mid-reply.

The model you connect to decides what kind of session you get:

  • Speech-to-speech. A spoken conversation with a model that listens and answers in its own voice.
  • Live transcription. Text returned while the speaker is still talking.
  • Speech translation. The model answers in a target language, with text and audio.
  • Text to speech. You stream text in, and speech comes back while the text is still arriving.

How a realtime session works

You open a WebSocket to wss://api.empiriolabs.ai/v1/realtime?model=MODEL_ID and authenticate with your API key in the Authorization header of the handshake. Once the session starts, you set the voice, the instructions and the turn detection mode, then stream audio in small chunks. The model sends back audio chunks to play, a transcript of what it says, and an event that marks the end of each turn.

Turn detection decides when the model answers. With server-side voice activity detection, the model notices when the speaker stops and responds on its own, and speaking during its reply interrupts it. With turn detection off, your application ends each turn and asks for the reply. Many speech-to-speech models can also call functions you declare during the conversation, and some accept live video frames or search the web.

Browser code must never hold your API key, so a web page connects with a short-lived client secret that your server creates with POST /v1/realtime/client_secrets. The endpoint follows the OpenAI Realtime protocol, so OpenAI SDKs can connect to it.

What realtime voice is used for

  • Voice agents. Support, booking and intake lines that answer questions and call your functions to look up an order or schedule an appointment.
  • In-app assistants. Hands-free help inside a product or a device.
  • Tutoring and language practice. Spoken lessons that respond while the learner speaks.
  • Live captions. Text for meetings, streams and events while people are talking.
  • Interpretation. Speech translated into a target language during a conversation.

Choosing a model and how billing works

Compare response latency, voices and languages, how the model handles interruptions and background noise, and whether it supports function calling, video input or web search. Then check the billing unit on each model page. Speech-to-speech models bill each completed turn from the tokens it used, and many price audio and text tokens separately, so a spoken turn and a typed turn of the same length cost different amounts. A model priced per minute bills the length of the session instead. Live transcription bills by the duration of the audio you send, or by tokens on some models.

Try a realtime model on EmpirioLabs AI

Open the Playground, select a realtime model, choose a voice and start speaking. To build your own client, create an API key in the dashboard and follow the Realtime Voice API guide, which lists every realtime model with its languages and voices, the events, and a complete example. Realtime models appear among the audio models and the transcription models.

Pay as you go, or choose a plan with weekly allowances. Current rates are on the pricing page.

The models on EmpirioLabs AI

20 models, cheapest first where they share a billing unit. Each model's page has its full rate card, parameters and code.

Endpoints

POST /v1/audio/transcriptions WEBSOCKET /v1/realtime POST /v1/audio/speech POST /v1/audio/speech:stream GET /v1/voices

Priced per minute or hour of audio

ModelLowest rateApplies to
StepAudio 2.5 ASR Stream StepFun$0.40 per hour of audioTranscription
Qwen Audio 3.0 ASR Stream Alibaba Cloud$0.648 per hour of audioTranscription
Deepgram Nova-3 Stream Deepgram$0.025 per minute of audioTranscription
OpenAI Whisper 1 OpenAI$0.030 per minutePer Minute of Audio

Also available: StepAudio 3 Realtime, StepAudio 2.5 Realtime, Qwen3.5 Omni Flash Realtime, Qwen3.5 Omni Plus Realtime, Qwen3.8 LiveTranslate Flash Realtime, Qwen3.8 Omni Flash Realtime, Qwen Audio 3.1 Realtime Plus, Gemini 3.8 Live, Gemini 3.8 Live Extended Thinking, Qwen Audio 3.1 ASR Stream, Qwen Audio 3.1 ASR Message, Qwen Audio 3.1 TTS, Qwen Audio 3.1 TTS Plus, Qwen Audio 3.1 Realtime Flash, Qwen Audio 3.0 Realtime Plus, Qwen Audio 3.0 Realtime Flash

Frequently asked questions

What is the difference between a realtime voice API and a text-to-speech API?

A text-to-speech API turns text into audio, one request at a time. A realtime voice API keeps a two-way connection open, so the model can listen, answer in speech and be interrupted within one session.

Why does a realtime voice API use a WebSocket?

A WebSocket carries data in both directions at once over one connection, so microphone audio can flow to the model while its reply flows back. A streamed HTTP response only carries output.

Can users interrupt the AI while it is speaking?

Yes. With server-side voice activity detection, speaking while the model is replying interrupts it, and the model responds to the new input.

How is a realtime voice session billed?

Speech-to-speech models bill each completed turn by its tokens, or the session length on models priced per minute. Live transcription bills by the duration of audio sent, or by tokens on some models. On EmpirioLabs AI you can pay as you go or choose a plan with weekly allowances.

Can I connect to a realtime voice API from a browser?

Yes. Your server creates a short-lived client secret with your API key, and the web page opens the WebSocket with that secret, so the API key never reaches the browser.

Ready to use better endpoints?

Check out our pricing or reach out if you want your own model deployed on our stack.