A telephone voice agent is often limited by its media transport long before it is limited by its language model. Twilio Media Streams exposes the call-audio rules that a realistic design must follow.
Unidirectional versus bidirectional
Twilio documents Media Streams over WebSockets. A unidirectional stream sends selected call audio to your application for processing; a bidirectional stream can also play application-generated audio into the call. Twilio starts the former with Start/Stream and the latter with Connect/Stream. In a bidirectional stream the application receives the inbound track; it cannot assume access to both tracks as if it were a recording mixer.
Audio format and control events
For media sent back into a bidirectional stream, Twilio specifies base64-encoded audio/x-mulaw at 8000 Hz with no file header bytes. Its protocol documents media, mark and clear events. Implementations should resample synthesized output correctly and purge playback when a caller interrupts. Feeding arbitrary PCM or a WAV header into the socket can sound broken even if TTS itself is perfect.
Track and region limits
Twilio’s overview describes track count limits and one bidirectional stream per call. It also mentions deployment regions such as US1, IE1 and AU1; application location must still be validated against the desired call region and the speech provider processing location. Do not claim that selecting a Twilio region automatically confines third-party ASR or LLM data to that geography.
A safe end-to-end pattern
Authenticate the stream, bind each call to a short-lived server session, enforce request budgets, connect a permitted recognizer and synthesize only approved agent responses. Announce recording and automation as required locally and expose a route to a human operator. Retain event metadata sufficient for debugging without retaining full sensitive audio by default. If ASR confidence deteriorates or the caller requests privacy, stop the AI component.
True cost and latency model
The budget involves carrier minutes, stream charges, ASR or realtime model metering, synthesis and infrastructure. End-to-end latency includes transport jitter, decoding, endpointing, reasoning, first synthesized frame and playback buffering. Request authoritative dated prices from AI10 instead of multiplying a model’s cost by call minutes without accounting for the carrier and silence.
Publication scope, rights and cost caveats
Research cutoff: 25 October 2026. The material above distinguishes provider documentation from independently measured behavior; no hands-on evaluation, legal clearance, voice-owner consent, commercial license purchase or model benchmark is claimed. Production teams must verify the specific API or installed version, account entitlement, input/output rights and destination market before release. Monetary plan amounts and metering tiers are not asserted here; the canonical pricing ledger belongs to AI 10. Missing release dates, exact rates and sample-rate capabilities are intentionally not inferred.