Wire Protocol

The bridge terminates two protocols: the StandIn media bridge’s worker protocol on one side, and the OpenAI Realtime API WebSocket on the other. This page documents both. The StandIn side is identical to the sibling bridges - the implementations are interchangeable.

The upgrade (StandIn side)

The StandIn media bridge opens one WebSocket per call to {path}/{callId} - the call id is the last path segment of the URL. The upgrade carries two headers:

Header	Value
X-StandIn-Timestamp	Unix epoch milliseconds
X-StandIn-Signature	HMAC-SHA256(secret, "{timestampMs}.{callId}"), lowercase hex

The legacy header names X-OpenClawTeamsBridge-Timestamp / -Signature are still accepted (the bridge checks the new names first); StandIn sends both pairs during the transition.

Verification (401 on failure): the timestamp must be within the freshness window (HMAC_FRESHNESS_MS, default 60 s), the signature must match (constant-time compare), and the (callId, ts, sig) tuple must be single-use (a captured handshake cannot be replayed within the window). The bridge fails closed if the shared secret is unset. The call id is also cross-checked against the session.start body.

Connection guards

Guard	Value
Max concurrent connections	64 (MAX_CONNECTIONS)
Per-IP cap	= total cap (MAX_CONNECTIONS_PER_IP)
Max inbound frame	2 MB
Outbound backpressure cap	1 MB (drops audio frames above it; control frames and one-shot images always pass)
Pre-start timeout	10 s (PRE_START_TIMEOUT_MS) - drops a socket that never sends session.start (only a real session.start clears it)
Worker idle timeout	90 s (WORKER_IDLE_TIMEOUT_MS) - dead-peer detection: ends the call after 3 missed 30 s heartbeats, freeing the call id and the billed Realtime session
Duplicate call id	rejected with 409 - no second billed Realtime session for one call

Audio on the StandIn wire is base64 PCM 16 kHz, 16-bit, mono; toward OpenAI it is PCM 24 kHz (converted at the adapter boundary).

Worker to bridge

Message	Fields	Bridge action
session.start	callId, threadId, caller{aadId?, displayName?, tenantId?}, recordingStatus?, direction?	Open the Realtime session; send session.update with instructions (base + caller name/tenant/direction), voice, VAD, PCM 24k formats and the tools array. All caller fields are nullable and are defaulted, never sent as null.
audio.frame	seq, timestampMs, payloadBase64, speakerName?	Upsample to 24 kHz, forward as input_audio_buffer.append. In group calls, a changed speakerName becomes a rate-limited context item.
video.frame	source (camera/screenshare), ts, width, height, mime, dataBase64, participantId?, participantName?	Buffer the latest frame per source, in memory, for the on-demand look tool. Unknown sources are ignored.
participants	count	System context item (“N humans on the call, stay quiet unless addressed”).
dtmf	digit	System context item (“the caller pressed {digit}”).
ping	ts	Reply pong with the same ts.
recording.status	status	Gate what may be persisted (transcripts, path-1 image attaches).
assistant.say	text	Governor goodbye: speak it (TTS or agent fallback), backstop teardown armed, then StandIn tears the call down.
session.end	reason	Close the Realtime socket, tear down.

Bridge to worker

Message	Fields	Meaning
audio.frame	seq, timestampMs, payloadBase64	Agent audio for the Teams side (Realtime deltas downsampled to 16 kHz).
assistant.cancel	turnId	Barge-in (or goodbye flush): flush queued playback on the Teams side. turnId is always 0 - the worker’s flush ignores the value.
expression	emotion	Avatar emotion cue (from the agent’s express tool).
display.image	dataBase64, mime, mode?, caption?, …	Show an image on the bot’s video tile (from show_image). Never dropped under backpressure.
pong	ts	Reply to a worker ping.
session.end	reason	Ask StandIn to tear the call down (governor, agent end_call, 60-min API ceiling, or fatal error).

OpenAI Realtime side (mapping)

Realtime event	Direction	Bridge behavior
session.update	bridge → OAI	Sent once at call start: type: "realtime", output_modalities: ["audio"], instructions + caller context, voice, audio.input/output.format pinned to {type: "audio/pcm", rate: 24000}, turn detection (server_vad/semantic_vad with interrupt_response: true), optional input transcription, and the tools array (built-ins + custom + MCP).
input_audio_buffer.append	bridge → OAI	Caller audio, upsampled 16k to 24k.
response.output_audio.delta	OAI → bridge	Agent audio, downsampled 24k to 16k → audio.frame. Dropped while the deterministic goodbye plays, if the response was cancelled, or if the response is no longer current (ghost drop).
response.created / response.done	OAI → bridge	Tracks the current response id for the ghost filter.
input_audio_buffer.speech_started	OAI → bridge	Caller barge-in: the server cancels its own response (interrupt_response); the bridge emits assistant.cancel and ghost-drops in-flight deltas from the cancelled response.
response.function_call_arguments.done	OAI → bridge	Tool dispatch: end_call, express, show_image, look, or a registered custom tool. Answered with a function_call_output item + response.create.
conversation.item.create (system message)	bridge → OAI	Non-interrupting context (participants, dtmf, speaker changes) - no response is triggered.
conversation.item.create (user message) + response.create	bridge → OAI	The goodbye fallback when no TTS voice is set, and path-1 vision (input_image + input_text content).
response.cancel	bridge → OAI	Goodbye flush (benign error event if nothing is in flight).
conversation.item.input_audio_transcription.completed / response.output_audio_transcript.done	OAI → bridge	Transcripts, logged only when LOG_TRANSCRIPTS=true and Teams recording is active.
error	OAI → bridge	Logged (benign cancel-nothing errors at debug level); the call survives malformed frames.

MCP tool entries in the tools array (type: "mcp") are executed server-side by the Realtime API - no bridge round-trip; see Extending the Agent’s Tools.