SIP for OpenAI Voice AI: Build Reliable Phone Agents
Learn how to connect phone calls to OpenAI with SIP, handle inbound and outbound calls, secure session controls, and prepare your voice agent for production.
How to Connect Phone Calls to OpenAI with SIP
SIP for OpenAI Voice AI: Build Reliable Phone Agents
A voice agent can sound great in a demo and still fall apart as a phone system. Real calls come with carrier routing, authentication, dropped connections, caller identity, keypad input, transfers, and business rules that a speech model can't handle by itself.
SIP is the bridge between your phone provider and OpenAI's Live API. Your provider routes the call and carries audio; your backend decides what to do with the session, applies business logic, and can monitor or control what happens. That split is helpful - but it also means you have to design the call flow, not just point a phone number at an AI model.
This guide walks through the key choices: how SIP fits into the architecture, how inbound and outbound calls differ, what a sideband connection is for, and which security and network requirements you need to think about before you go live. The goal is a system your team can observe and run reliably - not a call that only works once.
How SIP connects a phone call to an AI agent
SIP, or Session Initiation Protocol, sets up and manages internet-based phone calls. For an OpenAI voice agent, your SIP trunking provider connects a phone number to the Live API. The provider handles the phone-network connection; OpenAI runs the AI conversation; and your backend coordinates configuration, authorization, tools, and call control.
It's easier if you think of it as three responsibilities:
- Telephony: The provider routes calls and negotiates the phone connection.
- Conversation: The Live session handles audio and produces the agent's responses.
- Application logic: Your backend decides who can call, what the agent can do, and when to transfer or end a call.
These parts talk over different channels. SIP signaling uses TLS, while the call audio uses SRTP. Your backend can also open a sideband WebSocket to receive session events and send commands. The WebSocket isn't carrying the audio - SIP is. The sideband is how your backend gets control and visibility.
Choose the call direction first
Inbound and outbound calls follow different setup flows. For an inbound call, your provider routes the call to your OpenAI project, and your backend receives a webhook before deciding what to do. For an outbound call, your backend creates a Live session with SIP transport details and a destination phone number. In that case, you don't wait for an inbound-call webhook or send an accept request.
This difference impacts where you put authorization, routing, and error handling. Inbound webhooks are a decision point. Outbound session creation is effectively an instruction to start dialing.
Decide what the backend owns
The model shouldn't be the only place where business-critical actions are decided. Your backend is where you should check customer status, enforce account permissions, connect tools, and determine whether a transfer is allowed. This matters even more when using client delegation - your application has to handle the delegated work instead of assuming the model will do everything correctly on its own.
Here's a practical question to keep you sane: if the same event shows up twice, who is allowed to trigger the real-world action? Assign an owner for each side effect. For example, the webhook handler might own accept/reject, while a dedicated sideband worker owns a specific tool action. This prevents duplicate deliveries or multiple connections from executing the same operation twice.
How to handle inbound calls safely
An inbound flow starts with a SIP trunk configured to route calls to your OpenAI project. In the OpenAI project settings, create a webhook endpoint subscribed to live.transport.incoming, then point the provider's trunk at the project-specific SIP endpoint. The standard endpoint uses sip:$PROJECT_ID@sip.api.openai.com;transport=tls; for European data residency, use sip:$PROJECT_ID@sip-eu.api.openai.com;transport=tls.
Use the project ID that owns the webhook. Confirm SIP support is enabled for that same project and that the provider routes to it. If you get a mismatch here, it can look like a webhook or call-routing bug when it's really just configuration drift.
Verify the webhook and make one decision
When a call arrives, OpenAI sends a live.transport.incoming event. For SIP calls, the payload sets data.type to sip and includes data.session_id. Keep that session ID unchanged for any subsequent Live API actions. The payload may also include SIP headers like From, To, and Call-ID, but treat those as metadata you don't automatically trust.
Before you act on an event:
- Verify the webhook signature using your server-side secret or a supported verification library.
- Deduplicate delivery attempts using the webhook delivery ID.
- Apply your routing, authorization, and business rules.
- Make the accept-or-reject decision in a single place.
During migration, some older integrations may still receive the deprecated live.call.incoming event, and a pending call may also appear through a Realtime webhook. Don't accept through both paths. If you need to support legacy events, do it only long enough to drain existing deliveries and retries - and still ensure exactly one component owns the decision.
Accept or reject explicitly
To accept a call, send an authenticated POST request to /v1/live/sessions/{session_id}/accept. Include a top-level session object with the model, instructions, output voice, and delegation configuration. Pick the voice and delegation mode at acceptance time. Since SIP negotiates the audio format, omit audio.format.
A successful response is 200 OK with an empty body after session initialization. Don't assume the call is accepted just because you sent the request - check for HTTP errors and timeouts first. To reject, call /v1/live/sessions/{session_id}/reject with a SIP status code (for example, 486 for busy). The status must be an integer in the 300-699 range. The first accept-or-reject decision wins; if another request races you, it can return decision_already_made.
Keep the OpenAI API key on your trusted backend. Don't put it in a browser, a mobile app, or your phone-provider configuration. Treat the accept/reject endpoints as privileged controls: authenticate them, log them, and use idempotency rules that match your application.
How to place outbound calls through a SIP trunk
Outbound calls use the Live API session-creation endpoint with transport.type set to sip. This SIP call-creation flow is in the Live API, not the Realtime API's call-creation endpoint. Your provider connects to the phone network, and the Live session runs the conversation.
Before sending a request, configure a trunk that supports TLS signaling and SDES-SRTP media, and enable SRTP in the provider's settings. Include the trunk configuration in every session-creation request:
| Field | What to provide |
|---|---|
transport.destination |
The phone number to call in E.164 format, such as +14155550123. SIP URI destinations are not supported here. |
transport.trunk.provider_url |
A TLS provider endpoint, such as sips:sip.example.com:5061. |
transport.trunk.auth.type |
digest for SIP Digest authentication. |
transport.trunk.auth.username |
The SIP username issued by your provider. |
transport.trunk.auth.password |
The SIP password issued by your provider. |
transport.trunk.caller_number |
Your outbound caller number in E.164 format. |
The provider URL must use sips:. Keep it limited to the endpoint itself - don't add credentials, paths, URI headers, or unrelated parameters. Local hostnames and literal private or local IP addresses aren't accepted. Store the OpenAI API key and SIP credentials in server-side secrets management, not in source code or client apps.
Create the session and track its status
Send POST /v1/live/sessions with the session configuration and SIP transport details. Set the model, instructions, voice, and delegation mode at creation time. As with inbound calls, omit audio.format since SIP negotiates the audio format.
A 200 OK response means the session initialized. It does not mean the recipient answered. The response gives you the session ID, not SDP or trunk credentials. Save session.id exactly since you'll need it for the sideband connection and later call controls. There's no incoming-call webhook or accept request in this outbound flow.
Attach your sideband connection right after session creation and then watch for call-progress events. transport.ringing indicates ringing or early media; transport.answered means the call was answered and media is established; and transport.failed indicates an asynchronous setup failure. When setup fails, check the error code and message.
One more gotcha: the sideband replays only the events from the preceding three seconds. If you connect late, you can miss early progress events.
Ringing is limited to three minutes, and a connected call is limited to two hours. You can't change those limits via the creation request. Invalid session configuration is returned at creation time; transport setup can return 502, and initialization timeouts can return 504. Keep monitoring for transport.failed even after you get a successful creation response.
How to control calls with a sideband connection
A sideband connection attaches your backend to an existing Live conversation. It doesn't replace the SIP audio path. Instead, it's where you receive events, send commands, handle delegated work, and inspect reflected audio or transcripts where configured.
For inbound calls, connect after acceptance using wss://api.openai.com/v1/live/sessions/{session_id}/attach. For outbound calls, connect using the session ID returned by session creation. Use the right project authentication and connection headers. Don't send session.start again - the session already exists.
Prevent duplicate work across connections
A sideband gives you server-side visibility, but visibility can also create concurrency issues. If two connections observe the same event (or a webhook is redelivered), two handlers might try the same action. That can lead to duplicate tool calls, repeated customer updates, or multiple transfer requests.
Design around ownership, not just event receipt. Pick one handler per side effect and deduplicate using stable identifiers. For outbound call-progress events, use event_id; if an event is replayed, it keeps the original ID. For webhooks, deduplicate delivery using the delivery ID. Also separate "observe" from "act": it's fine for a service to log an event even if it isn't authorized to execute the related business operation.
Send keypad tones, transfer, or hang up
The sideband receives transport.dtmf.received when the caller presses a keypad key. To send a tone to the SIP trunk, send a transport.dtmf.send client event over the WebSocket. After a successful delivery, you'll receive transport.dtmf.sent.
If you attach an event_id to your client event, the result may include it as client_event_id, which helps correlate commands and failures.
To transfer a call, send POST /v1/live/sessions/{session_id}/refer with a target_uri. The target can be a telephone URI like tel:+14155550123 or a SIP URI like sip:agent@example.com. Your provider uses that to populate the SIP Refer-To header and completes the rest of the transfer flow.
To end the call, send POST /v1/live/sessions/{session_id}/hangup. Transfer and hangup both return 200 OK with an empty body on success. Keep the sideband open until you receive session.closed and the final usage information. If you close early, your app may not have the final state it needs for reporting or reconciliation.
What to plan for before production
SIP calls use separate network paths for signaling and media. For signaling, allow outbound TCP/TLS traffic on port 5061 to the DNS-resolved addresses for sip.api.openai.com or sip-eu.api.openai.com. These are GeoIP-routed endpoints, so you shouldn't assume one fixed signaling IP.
For media, the negotiated SDP tells you which IP and UDP port to use. Allow bidirectional SRTP over UDP to and from these CIDR ranges:
13.79.45.80/2823.98.140.64/2840.67.149.176/2840.83.204.240/28
If your firewall allows SIP signaling but blocks UDP media, you can end up with a call that seems connected but has no usable audio. Test signaling and media separately, and verify the actual network path from the environment where your trunk and backend run.
Treat telephony like an operations problem, not just an integration
A phone agent does more than produce a model response. Decide what your system does when the caller isn't eligible, the provider can't establish media, a tool isn't available, or the agent can't resolve the request. Make the policies explicit and testable: when to reject, when to retry, and when to transfer to a human.
Also review legal and operational requirements for your business and regions. Call recording, notices, consent, data retention, and access to caller info can be governed by local rules, including privacy requirements like the GDPR. Regional SIP endpoints can support data residency preferences, but they don't automatically make you compliant. Confirm the full data flow, provider arrangements, and retention policy with your legal and security teams.
Before launch, run test calls for both directions and verify:
- Webhook signature verification, delivery deduplication, and authorization behavior.
- Correct handling of accept/reject races and asynchronous setup failures.
- Two-way SRTP audio, keypad events, transfers, and hangups.
- Sideband reconnection behavior, event deduplication, and final usage capture.
- Human escalation and a clear response when the AI or a downstream tool can't proceed.
Key takeaway: SIP gets the call to the model, but production reliability comes from the system around it: one owner for every action, secure credentials, observable call state, and a clear path to a human.
Conclusion
SIP provides the phone-network connection, but it doesn't remove the need to build a reliable call system. Keep inbound and outbound flows separate, keep credentials on the server, treat caller headers as untrusted, and assign one owner to each action. Use the sideband for control and observability, and test the full path - including SRTP media, failure handling, transfers, and final usage - before you route real customers to the agent.
The next practical step is to build one narrow call flow end to end. Prove it connects, recovers from expected failures, and hands off safely. Once those behaviors are observable and repeatable, you can expand what the agent is responsible for.