Ways to connect a call
Not supported:
- Native SIP. Izwi has no SIP endpoint and registers with no trunk.
- WebRTC.
/v1/realtime/callsis not served, and/v1/realtime?call_id=returns400. - DTMF. Keypad digits are not detected in audio, and
dtmfframes of a media stream are ignored. Handle menus in the telephony layer.
IZWI_MAX_REALTIME_SESSIONS).
Through a media server or agent framework
Anything with an OpenAI Realtime client can be pointed at Izwi. The repository’s end-to-end suite (scripts/e2e/openai-realtime/) runs the unmodified Realtime clients of openai-python, LiveKit Agents and Pipecat against /v1/realtime; with LiveKit or Pipecat, the framework’s own SIP or telephony transport carries the call and its Realtime client talks to Izwi.
Configure the framework’s OpenAI Realtime client with:
Through a media stream
GET /v1/realtime/media-stream speaks the WebSocket framing carriers and call-centre platforms use to hand a call’s audio to an application: JSON text frames carrying base64 8 kHz μ-law audio. Give the platform the stream URL; one socket carries one call.
Some platforms drop the query string of a stream URL. Send the same names as the
start frame’s custom parameters instead; a custom parameter overrides the query string.
Authentication is the server’s own: an Authorization: Bearer header carrying IZWI_API_KEY or a client secret, or the client secret as the openai-insecure-api-key.<key> WebSocket subprotocol. A client secret’s session config applies to the call, so an application can mint one per call with the instructions and voice that call needs. Keys are not read from the query string. If your platform can set neither a header nor a subprotocol, put a proxy that adds the header between the platform and Izwi.
The frames, the events Izwi adds, and what is ignored are in the API reference.
Example: live transcription for agent assist
A supervisor or assist tool wants the caller’s words while they are spoken.-
Pick a transcription model that transcribes live.
GET /v1/realtime/capabilitieslists each model undertranscription_modelswithtranscription_mode:livemodels send partial text while the caller speaks;turnmodels send each utterance after the caller pauses. -
Have the platform stream the caller’s audio to a transcription call:
-
Read the
transcriptframes from the same socket:deltaframes add up to the utterance andcompletedcarries it whole. Aturnmodel sendscompletedframes only.
/v1/realtime?intent=transcription, set audio.input.transcription.model and read conversation.item.input_audio_transcription.delta and .completed events.
To transcribe both parties, stream each party’s audio to its own transcription call.
Example: a voice agent answers the call
-
Choose a profile:
izwi-voiceruns a transcription, chat and speech model in sequence;izwi-voice-compactruns one speech-to-speech model.IZWI_REALTIME_PROFILESdefines your own. -
Connect the call to a conversation:
Send
instructionsas a custom parameter, or bind them to a client secret, to give the agent its script. -
Izwi cuts the caller’s turns, answers each one, and sends the speech back as
mediaframes for the platform to play. When the caller talks over the agent, Izwi stops the reply and sendsclearso the platform drops the audio it has not played yet.
/v1/realtime with G.711 audio.
scripts/e2e/openai-realtime/run_media_stream.py plays a WAV file as a call of either kind and saves the agent’s speech.
Post-call transcription beside live calls
One server can transcribe recordings in bulk while it serves live calls.- Send recordings to
POST /v1/audio/transcriptionswithservice_tier: "flex". Flex requests are admitted and scheduled behind live work. See Audio Service Tier. - While a model has live demand, batch work on that model leaves part of its rows free so a live request starts without waiting.
IZWI_REALTIME_ROW_RESERVEsets how many (by default a quarter of the model’s rows, at least one). - A flex request that finds the server full waits up to
IZWI_ADMISSION_QUEUE_TIMEOUT_SECS(30 seconds) and is then refused with429, coderesource_unavailableand aRetry-Afterheader. Retry it later. - Long recordings are transcribed in bounded steps, with live requests served between them. Two models are the exception: VibeVoice-ASR and Nemotron 3.5 Transcribe read a whole recording in one pass and hold the device until it ends, so a live request of the same model waits. If you need those models for recordings and strict live latency, run them on a separate server or device.
scripts/bench/run-mixed-load.py holds a set of live sessions while it raises a batch load and reports what the batch load does to live latency; see scripts/bench/README.md.