Voice WebSockets | Pinnacle

The SDKs expose generated REST methods for calls and custom voice helpers for WebSocket streams. Use the high-level helper for most interactive voice apps, then drop down to connect(...) or REST when you need finer control.

Generated REST methods:

calls.create(...)
calls.list(...)
calls.get(callId)
calls.update(callId, ...)
calls.cancel(callId)
calls.createStreamToken(callId, ...)
calls.listEvents(callId, ...)
calls.downloadRecording(callId, ...)
calls.deleteRecording(callId)

Start and connect

createAndConnect(...) / create_and_connect(...) is the default for interactive outbound calls. It creates the call, mints a short-lived stream token, opens the WebSocket, attaches the created call to the returned connection, and refreshes stream tokens during reconnects.

TypeScript uses globalThis.WebSocket when available and includes a ws fallback for Node. Python uses websocket-client. Ruby uses websocket-client-simple. All SDKs still accept a custom socket for tests or specialized runtimes.

TypeScript

Python

Ruby

import { PinnacleClient } from "rcs-js";
const client = new PinnacleClient({ apiKey: process.env.PINNACLE_API_KEY });
const voice = await client.voice.createAndConnect({
  from: "+14155559876",
  to: "+14155551234",
  record: true,
  token: {
    commands_enabled: true,
    stream_id: "agent",
    record: true,
  },
});
voice.on("event", (frame) => console.log(frame.type, frame.payload));
voice.on("ack", (frame) => console.log(frame.command_id, frame.status));
await voice.waitUntilOpen();
voice.playAudio({ text: "Thanks for calling." }, "cmd_welcome");

Choose a helper

Helper	Use it for	Reconnect behavior
`createAndConnect(...)` / `create_and_connect(...)`	Starting a new interactive outbound call.	Refreshes stream tokens by default.
`connect(...)` / `connect(...)`	Attaching to an existing call, including inbound calls and observers.	Refreshes stream tokens by default.
`connectStream(...)` / `connect_stream(...)`	Opening a WebSocket from an already minted `stream_url`.	Reuses the same URL. Prefer `connect(...)` when reconnects may need fresh tokens.

Pass reconnect: { enabled: false } only when you want to handle reconnects yourself. Partial reconnect options inherit the default enabled behavior, so { maxAttempts: 2 } still reconnects.

Ruby uses end_call(...) for call.end because end is a reserved keyword.

Frames and commands

The voice socket carries command frames, optional raw audio frames, and server frames. Most apps use the command helpers and listen for event / ack.

Direction	Frame	Purpose
App to Pinnacle	`command`	Control the call: answer, transfer, play audio, gather input, update metadata, and more.
App to Pinnacle	`media`	Send generated outbound audio into the call. Most apps use `audio.play` instead.
Pinnacle to app	`ack`	Confirms whether a command was accepted. It echoes your `command_id`.
Pinnacle to app	`event` / `media`	Delivers call lifecycle events, input results, and inbound audio.

Command frames

All commands use the same envelope. SDK helpers build this for you:

{
  "event": "command",
  "command_id": "cmd_play",
  "action": "audio.play",
  "params": {
    "text": "Thanks for calling.",
    "voice": "female",
    "language": "en-US"
  }
}

The matching acknowledgement echoes command_id and includes a monotonic sequence_number.

Field	Meaning
`event`	Always `ack`.
`command_id`	The ID you sent on the command frame.
`action`	The command action that was acknowledged.
`status`	`ok` when the command was accepted.
`sequence_number`	Server-side ordering value.

Command catalog

Each command lists required and optional fields. Use Show example to reveal a sample payload.

[
  {
    "title": "Call control",
    "description": "Answer, end, transfer, and annotate the active call.",
    "commands": [
      {
        "action": "call.answer",
        "description": "Answer an inbound call or confirm an outbound call is ready for interaction.",
        "optional": ["as"],
        "params": {
          "as": "Support"
        }
      },
      {
        "action": "call.end",
        "description": "End an active call or reject a pre-answer call.",
        "optional": [
          {
            "name": "cause",
            "values": ["user_busy", "call_rejected"]
          }
        ],
        "params": {
          "cause": "call_rejected"
        }
      },
      {
        "action": "call.transfer",
        "description": "Dial and bridge a new number, or bridge to another Pinnacle call.",
        "params": [
          {
            "label": "Dial a new number",
            "required": ["to"],
            "optional": ["from", "as", "custom_headers"],
            "payload": {
              "to": "+14155551234",
              "from": "+14155559876",
              "as": "Support",
              "custom_headers": {
                "X-Customer-ID": "cus_123"
              }
            }
          },
          {
            "label": "Bridge an existing call",
            "required": [
              {
                "name": "call_id",
                "note": "Must be a Pinnacle call ID, not a provider call-control ID."
              }
            ],
            "optional": ["as"],
            "payload": {
              "call_id": "call_123",
              "as": "Support"
            }
          }
        ]
      },
      {
        "action": "call.update_state",
        "description": "Attach metadata that appears on future call events.",
        "required": ["metadata"],
        "params": {
          "metadata": {
            "ticket_id": "tkt_123"
          }
        }
      }
    ]
  },
  {
    "title": "Audio",
    "description": "Play audio, stop playback, and tune the live audio stream.",
    "commands": [
      {
        "action": "audio.play",
        "description": "Play TTS or a public audio URL into the call.",
        "params": [
          {
            "label": "Text-to-speech",
            "required": ["text"],
            "optional": [
              {
                "name": "voice",
                "valueLabel": "examples",
                "values": ["female", "alice"]
              },
              {
                "name": "language",
                "valueLabel": "examples",
                "values": ["en-US"]
              }
            ],
            "payload": {
              "text": "Thanks for calling.",
              "voice": "female",
              "language": "en-US"
            }
          },
          {
            "label": "URL playback",
            "required": [
              {
                "name": "url",
                "note": "Must be a public http or https URL without embedded credentials or private-network hosts."
              }
            ],
            "optional": ["loop", "overlay"],
            "payload": {
              "url": "https://static.pinnacle.sh/examples/hold.wav",
              "loop": 1,
              "overlay": true
            }
          }
        ]
      },
      {
        "action": "audio.stop",
        "description": "Stop active URL playback or TTS.",
        "params": ""
      },
      {
        "action": "audio.clear",
        "description": "Stop audio your app has streamed over the WebSocket and drop what's still queued. Use it for barge-in: audio you send is buffered, so stopping sending doesn't stop the caller hearing it.",
        "params": ""
      },
      {
        "action": "audio.reduce_noise",
        "description": "Enable or disable noise reduction for inbound, outbound, or both tracks.",
        "required": ["enabled"],
        "optional": [
          {
            "name": "engine",
            "values": ["default", "krisp", "aicoustics"]
          },
          {
            "name": "direction",
            "values": ["inbound", "outbound", "both"]
          }
        ],
        "params": {
          "enabled": true,
          "engine": "krisp",
          "direction": "both"
        }
      }
    ]
  },
  {
    "title": "Input",
    "description": "Collect keypad digits, spoken answers, and send DTMF tones.",
    "commands": [
      {
        "action": "input.get",
        "description": "Gather digits, speech, or structured AI extraction after an optional prompt.",
        "optional": [
          "prompt",
          "minDigits",
          "maxDigits",
          {
            "name": "validDigits",
            "valueLabel": "example",
            "values": ["12345"]
          },
          {
            "name": "terminatingDigit",
            "valueLabel": "example",
            "values": ["#"]
          },
          "timeoutMs",
          "interDigitTimeoutMs",
          "ai"
        ],
        "params": {
          "prompt": {
            "text": "Enter your PIN.",
            "voice": "female",
            "language": "en-US"
          },
          "minDigits": 4,
          "maxDigits": 4,
          "validDigits": "0123456789",
          "terminatingDigit": "#",
          "timeoutMs": 10000,
          "interDigitTimeoutMs": 3000,
          "ai": {
            "extract": {
              "type": "object",
              "properties": {
                "pin": {
                  "type": "string"
                }
              },
              "required": ["pin"]
            },
            "instructions": "Extract the caller's PIN."
          }
        }
      },
      {
        "action": "input.cancel",
        "description": "Cancel the active input gathering request.",
        "params": ""
      },
      {
        "action": "dtmf.send",
        "description": "Send keypad tones to the far end of the call.",
        "required": [
          {
            "name": "digits",
            "values": ["0-9", "*", "#", "A-D"]
          }
        ],
        "optional": ["duration_ms"],
        "params": {
          "digits": "1234#",
          "duration_ms": 250
        }
      }
    ]
  },
  {
    "title": "Recording",
    "description": "Control Pinnacle-managed recording during the call.",
    "commands": [
      {
        "action": "recording.start",
        "description": "Start recording future audio frames.",
        "params": ""
      },
      {
        "action": "recording.stop",
        "description": "Stop future capture and flush the current recording segment.",
        "params": ""
      }
    ]
  },
  {
    "title": "Conferences",
    "description": "Add people to a conference with this call in it, and change or remove them. See the Conference Calls guide.",
    "commands": [
      {
        "action": "conference.add_participant",
        "description": "Dial someone (or pull in another of your calls). Without conference_id, uses this call's conference, creating one with this call in it. from defaults to your number on this call.",
        "params": [
          {
            "label": "Dial someone",
            "required": ["to"],
            "optional": ["from", "label", "muted", "on_hold", "whisper_to", "announce", "ring_timeout_secs", "end_conference_on_leave", "record", "metadata", "conference_id"],
            "payload": {
              "to": "+14155551234",
              "label": "supervisor",
              "muted": true,
              "announce": { "text": "You're joining a billing call. You're muted." }
            }
          },
          {
            "label": "Pull in an existing call",
            "required": ["call_id"],
            "optional": ["label", "muted", "on_hold", "whisper_to", "end_conference_on_leave", "conference_id"],
            "payload": {
              "call_id": "call_7cQe4F8xT1yB2mN9pK3L",
              "label": "caller"
            }
          }
        ]
      },
      {
        "action": "conference.update_participant",
        "description": "Mute, hold, whisper, or relabel a participant. participant is a call ID, a label, or self (this call).",
        "required": ["participant"],
        "optional": ["muted", "on_hold", "whisper_to", "label"],
        "params": {
          "participant": "supervisor",
          "muted": false,
          "whisper_to": ["specialist"]
        }
      },
      {
        "action": "conference.remove_participant",
        "description": "Hang up a participant's call; everyone else stays.",
        "required": ["participant"],
        "params": {
          "participant": "specialist"
        }
      },
      {
        "action": "conference.leave",
        "description": "This call leaves the conference (hangs up).",
        "params": ""
      },
      {
        "action": "conference.get",
        "description": "Get this call's conference and its participants in the ack.",
        "params": ""
      },
      {
        "action": "conference.end",
        "description": "End this call's conference and hang up everyone.",
        "params": ""
      }
    ]
  }
]

Audio frames

Most apps should not send audio frames directly.

Use audio.play when you want to play TTS or an audio file from a public URL. Use raw audio frames only when you need to stream audio from your own app in realtime.

For files like MP3, WAV, M4A, MP4, or MOV:

Prefer hosting the file and playing it with audio.play.
If you must stream the bytes yourself, convert the file to raw 8 kHz mono mu-law audio.
Split the converted bytes into small chunks.
Base64 each chunk and send it as media.payload.

Convert almost any local audio or video file with FFmpeg:

ffmpeg -i input.mp3 -vn -ac 1 -ar 8000 -f mulaw output.ulaw

The same command works for formats FFmpeg can read, including .wav, .m4a, .mp4, and .mov. The output.ulaw file is raw audio bytes. It is not a WAV file and does not have a header.

Send each chunk with this frame shape:

{
  "event": "media",
  "media": {
    "track": "outbound",
    "payload": "base64-mulaw-audio",
    "chunk": 1,
    "timestamp": "2026-06-24T00:00:00.000Z"
  }
}

The important field is media.payload: it is one base64-encoded chunk from output.ulaw. A 20 ms chunk is a recommended default, not a protocol requirement. At 8 kHz, 20 ms is 160 bytes before base64 encoding.

Field	What to send
`media.track`	Use `outbound` for audio you send into the call.
`media.payload`	Base64 for one raw 8 kHz mono mu-law chunk.
`media.chunk`	Optional counter. Start at `1` and increment per chunk.
`media.timestamp`	Optional timestamp for the chunk.

Chunk converted audio

These examples assume audio contains the bytes from output.ulaw.

TypeScript

Python

Ruby

function mediaFramesFromMulawBytes(audio) {
  const chunkSize = 160;
  const frames = [];
  for (let offset = 0; offset < audio.length; offset += chunkSize) {
    const chunk = audio.subarray(offset, offset + chunkSize);
    frames.push({
      event: "media",
      media: {
        track: "outbound",
        payload: Buffer.from(chunk).toString("base64"),
        chunk: frames.length + 1,
      },
    });
  }
  return frames;
}
const frames = mediaFramesFromMulawBytes(Buffer.from([255, 127, 0, 127]));

In your app, a good default is to send one frame about every 20 ms with voice.sendMedia(frame.media) in TypeScript, voice.send_media(frame["media"]) in Python, or voice.send_media(frame["media"]) in Ruby. This is recommended, not required. At 8 kHz, 20 ms is 160 audio bytes, which keeps latency low without creating excessive WebSocket traffic.

If you are generating audio live, ask your audio provider for 8 kHz mono mu-law / PCMU output when possible. If it gives you PCM, MP3, or another format, run the same conversion step before chunking.

Receiving audio

Inbound audio from Pinnacle uses the same media.payload format. Server frames include stream_sid and sequence_number so you can keep chunks in order:

{
  "event": "media",
  "stream_sid": "stream_123",
  "sequence_number": 42,
  "media": {
    "track": "inbound",
    "payload": "base64-mulaw-audio",
    "chunk": 7,
    "timestamp": "2026-06-24T00:00:01.400Z"
  }
}

SDK frame types

The SDKs export the WebSocket frame shapes and enum values used by the custom voice helpers.

TypeScript

Python

Ruby

import { VoiceCallEndCause, VoiceClientFrame, VoiceCommandAction, VoiceCommandFrame, VoiceMediaTrack, VoiceNoiseReductionDirection, VoiceNoiseReductionEngine, VoiceServerFrame } from "rcs-js";