OpenAI Realtime

OpenAI's SIP endpoint only accepts TLS signaling with encrypted (SRTP) audio. Vonyx trunks use UDP or TCP without media encryption, so you can't point a Vonyx trunk straight at OpenAI today. Instead, run the OpenAI Realtime model inside a LiveKit agent, and connect LiveKit to Vonyx over SIP.

A direct SIP connection doesn't work

OpenAI's SIP URI is sip:[email protected];transport=tls. OpenAI requires TLS signaling on port 5061 and SRTP for call audio. Vonyx supports neither, so calls sent from a Vonyx trunk to that URI will fail.

How it connects

Leg	Between	Protocol
Phone call	Caller and Vonyx	PSTN
SIP trunk	Vonyx and LiveKit	SIP over UDP or TCP, PCMU / PCMA
Model	LiveKit agent and OpenAI	Realtime API, using your `OPENAI_API_KEY`

The Realtime model still handles speech-to-speech. LiveKit carries the call audio between Vonyx and the model.

Before you start

A Vonyx SIP trunk with a number attached
A LiveKit Cloud project and the LiveKit CLI
An OpenAI API key

Set up the bridge

Connect Vonyx to LiveKit

Follow the LiveKit guide to create the outbound trunk, the inbound trunk and the dispatch rule, and to set the Vonyx inbound address. That guide covers both outbound and inbound calls.

Use the OpenAI Realtime model in your agent

Install the LiveKit Agents OpenAI plugin:

uv add "livekit-agents[openai]~=1.8"

Set OPENAI_API_KEY in your agent's .env, then pass the Realtime model as the session's LLM:

agent.py

from livekit.agents import AgentSession
from livekit.plugins import openai
 
session = AgentSession(
    llm=openai.realtime.RealtimeModel(voice="marin"),
)

OpenAI bills the model usage to your key.

Using Pipecat instead

Pipecat ships an OpenAI Realtime service (OpenAIRealtimeLLMService, installed with uv add "pipecat-ai[openai]"). You still need a SIP transport in front of Pipecat that accepts UDP or TCP from Vonyx.

Test the connection

Start your LiveKit agent.
Place an outbound call as shown in the LiveKit guide. Your phone should ring and the Realtime agent should answer.
Call your Vonyx number from a phone. The call should reach a call- room and the agent should pick up.

Limitations

DTMF and call transfer are not available on Vonyx yet. OpenAI's refer endpoint for transfers doesn't apply on this setup.
New accounts make 1 call at a time (raise limits).
Vonyx can't connect to OpenAI's SIP endpoint directly, because it has no TLS or SRTP support.
The Realtime API's call webhooks (realtime.call.incoming) and the /v1/realtime/calls/{call_id}/accept, /reject and /hangup endpoints are only for OpenAI SIP calls. They aren't used when the call goes through LiveKit.

Troubleshooting

Problem	Fix
Calls to `sip.api.openai.com` fail or time out	Expected: OpenAI needs TLS and SRTP. Use the LiveKit bridge above.
The call connects but the agent is silent	Check that `OPENAI_API_KEY` is set where the agent runs and that the agent joined the room.
Inbound calls don't reach LiveKit	Check the inbound address in the Vonyx console and the inbound trunk's allowed address. See troubleshooting.
Outbound calls are rejected	Make sure the caller ID is one of your Vonyx numbers and the trunk credentials match.

On this page