> This page is for version v2 (default).
> For other versions, use one of these documentation indexes:
> - v2 (default): https://docs.pinnacle.sh/v-2/llms.txt
> - v1: https://docs.pinnacle.sh/v1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.pinnacle.sh/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.pinnacle.sh/_mcp/server.

# Voice WebSockets

> Use stream tokens to connect to live voice calls.

The SDKs expose generated REST methods for calls and custom voice helpers for WebSocket streams. Use the high-level helper for most interactive voice apps, then drop down to `connect(...)` or REST when you need finer control.

Generated REST methods:

* `calls.create(...)`
* `calls.list(...)`
* `calls.get(callId)`
* `calls.update(callId, ...)`
* `calls.cancel(callId)`
* `calls.createStreamToken(callId, ...)`
* `calls.listEvents(callId, ...)`
* `calls.downloadRecording(callId, ...)`
* `calls.deleteRecording(callId)`

## Start and connect

`createAndConnect(...)` / `create_and_connect(...)` is the default for interactive outbound calls. It creates the call, mints a short-lived stream token, opens the WebSocket, attaches the created call to the returned connection, and refreshes stream tokens during reconnects.

TypeScript uses `globalThis.WebSocket` when available and includes a `ws` fallback for Node. Python uses `websocket-client`. Ruby uses `websocket-client-simple`. All SDKs still accept a custom socket for tests or specialized runtimes.

#### TypeScript

```typescript
import { PinnacleClient } from "rcs-js";

const client = new PinnacleClient({ apiKey: process.env.PINNACLE_API_KEY });

const voice = await client.voice.createAndConnect({
  from: "+14155559876",
  to: "+14155551234",
  record: true,
  token: {
    commands_enabled: true,
    stream_id: "agent",
    record: true,
  },
});

voice.on("event", (frame) => console.log(frame.type, frame.payload));
voice.on("ack", (frame) => console.log(frame.command_id, frame.status));

await voice.waitUntilOpen();
voice.playAudio({ text: "Thanks for calling." }, "cmd_welcome");
```

#### Python

```python
import os

from rcs import Pinnacle

client = Pinnacle(api_key=os.environ["PINNACLE_API_KEY"])

voice = client.voice.create_and_connect(
    from_="+14155559876",
    to="+14155551234",
    record=True,
    token={
        "commands_enabled": True,
        "stream_id": "agent",
        "record": True,
    },
)

voice.on("event", lambda frame: print(frame["type"], frame["payload"]))
voice.on("ack", lambda frame: print(frame["command_id"], frame["status"]))

voice.wait_until_open()
voice.play_audio({"text": "Thanks for calling."}, "cmd_welcome")
```

#### Ruby

```ruby
require "rcs"

client = Pinnacle::Client.new(api_key: ENV["PINNACLE_API_KEY"])

voice = client.voice.create_and_connect(
  from: "+14155559876",
  to: "+14155551234",
  record: true,
  token: {
    commands_enabled: true,
    stream_id: "agent",
    record: true
  }
)

voice.on("event") { |frame| puts "#{frame['type']}: #{frame['payload']}" }
voice.on("ack") { |frame| puts "#{frame['command_id']}: #{frame['status']}" }

voice.wait_until_open
voice.play_audio({ text: "Thanks for calling." }, command_id: "cmd_welcome")
```

## Choose a helper

| Helper                                              | Use it for                                                            | Reconnect behavior                                                                |
| --------------------------------------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| `createAndConnect(...)` / `create_and_connect(...)` | Starting a new interactive outbound call.                             | Refreshes stream tokens by default.                                               |
| `connect(...)` / `connect(...)`                     | Attaching to an existing call, including inbound calls and observers. | Refreshes stream tokens by default.                                               |
| `connectStream(...)` / `connect_stream(...)`        | Opening a WebSocket from an already minted `stream_url`.              | Reuses the same URL. Prefer `connect(...)` when reconnects may need fresh tokens. |

Pass `reconnect: { enabled: false }` only when you want to handle reconnects yourself. Partial reconnect options inherit the default enabled behavior, so `{ maxAttempts: 2 }` still reconnects.

Ruby uses `end_call(...)` for `call.end` because `end` is a reserved keyword.

## Frames and commands

The voice socket carries command frames, optional raw audio frames, and server frames. Most apps use the command helpers and listen for `event` / `ack`.

| Direction       | Frame             | Purpose                                                                                  |
| --------------- | ----------------- | ---------------------------------------------------------------------------------------- |
| App to Pinnacle | `command`         | Control the call: answer, transfer, play audio, gather input, update metadata, and more. |
| App to Pinnacle | `media`           | Send generated outbound audio into the call. Most apps use `audio.play` instead.         |
| Pinnacle to app | `ack`             | Confirms whether a command was accepted. It echoes your `command_id`.                    |
| Pinnacle to app | `event` / `media` | Delivers call lifecycle events, input results, and inbound audio.                        |

### Command frames

All commands use the same envelope. SDK helpers build this for you:

```json
{
  "event": "command",
  "command_id": "cmd_play",
  "action": "audio.play",
  "params": {
    "text": "Thanks for calling.",
    "voice": "female",
    "language": "en-US"
  }
}
```

The matching acknowledgement echoes `command_id` and includes a monotonic `sequence_number`.

| Field             | Meaning                                   |
| ----------------- | ----------------------------------------- |
| `event`           | Always `ack`.                             |
| `command_id`      | The ID you sent on the command frame.     |
| `action`          | The command action that was acknowledged. |
| `status`          | `ok` when the command was accepted.       |
| `sequence_number` | Server-side ordering value.               |

### Command catalog

Each command lists required and optional fields. Use **Show example** to reveal a sample payload.

<pre />

### Audio frames

Most apps should not send audio frames directly.

Use `audio.play` when you want to play TTS or an audio file from a public URL. Use raw audio frames only when you need to stream audio from your own app in realtime.

For files like MP3, WAV, M4A, MP4, or MOV:

1. Prefer hosting the file and playing it with `audio.play`.
2. If you must stream the bytes yourself, convert the file to raw 8 kHz mono mu-law audio.
3. Split the converted bytes into small chunks.
4. Base64 each chunk and send it as `media.payload`.

Convert almost any local audio or video file with [FFmpeg](https://ffmpeg.org/ffmpeg.html):

```bash
ffmpeg -i input.mp3 -vn -ac 1 -ar 8000 -f mulaw output.ulaw
```

The same command works for formats FFmpeg can read, including `.wav`, `.m4a`, `.mp4`, and `.mov`. The `output.ulaw` file is raw audio bytes. It is not a WAV file and does not have a header.

Send each chunk with this frame shape:

```json
{
  "event": "media",
  "media": {
    "track": "outbound",
    "payload": "base64-mulaw-audio",
    "chunk": 1,
    "timestamp": "2026-06-24T00:00:00.000Z"
  }
}
```

The important field is `media.payload`: it is one base64-encoded chunk from `output.ulaw`. A 20 ms chunk is a recommended default, not a protocol requirement. At 8 kHz, 20 ms is 160 bytes before base64 encoding.

| Field             | What to send                                            |
| ----------------- | ------------------------------------------------------- |
| `media.track`     | Use `outbound` for audio you send into the call.        |
| `media.payload`   | Base64 for one raw 8 kHz mono mu-law chunk.             |
| `media.chunk`     | Optional counter. Start at `1` and increment per chunk. |
| `media.timestamp` | Optional timestamp for the chunk.                       |

#### Chunk converted audio

These examples assume `audio` contains the bytes from `output.ulaw`.

#### TypeScript

```typescript
function mediaFramesFromMulawBytes(audio) {
  const chunkSize = 160;
  const frames = [];

  for (let offset = 0; offset < audio.length; offset += chunkSize) {
    const chunk = audio.subarray(offset, offset + chunkSize);
    frames.push({
      event: "media",
      media: {
        track: "outbound",
        payload: Buffer.from(chunk).toString("base64"),
        chunk: frames.length + 1,
      },
    });
  }

  return frames;
}

const frames = mediaFramesFromMulawBytes(Buffer.from([255, 127, 0, 127]));
```

#### Python

```python
import base64

def media_frames_from_mulaw_bytes(audio):
    chunk_size = 160
    frames = []

    for offset in range(0, len(audio), chunk_size):
        chunk = audio[offset : offset + chunk_size]
        frames.append({
            "event": "media",
            "media": {
                "track": "outbound",
                "payload": base64.b64encode(chunk).decode("ascii"),
                "chunk": len(frames) + 1,
            },
        })

    return frames

frames = media_frames_from_mulaw_bytes(bytes([255, 127, 0, 127]))
```

#### Ruby

```ruby
require "base64"

def media_frames_from_mulaw_bytes(audio)
  chunk_size = 160

  audio.bytes.each_slice(chunk_size).with_index.map do |chunk, index|
    {
      "event" => "media",
      "media" => {
        "track" => "outbound",
        "payload" => Base64.strict_encode64(chunk.pack("C*")),
        "chunk" => index + 1
      }
    }
  end
end

frames = media_frames_from_mulaw_bytes([255, 127, 0, 127].pack("C*"))
```

In your app, a good default is to send one frame about every 20 ms with `voice.sendMedia(frame.media)` in TypeScript, `voice.send_media(frame["media"])` in Python, or `voice.send_media(frame["media"])` in Ruby. This is recommended, not required. At 8 kHz, 20 ms is 160 audio bytes, which keeps latency low without creating excessive WebSocket traffic.

If you are generating audio live, ask your audio provider for 8 kHz mono mu-law / PCMU output when possible. If it gives you PCM, MP3, or another format, run the same conversion step before chunking.

#### Receiving audio

Inbound audio from Pinnacle uses the same `media.payload` format. Server frames include `stream_sid` and `sequence_number` so you can keep chunks in order:

```json
{
  "event": "media",
  "stream_sid": "stream_123",
  "sequence_number": 42,
  "media": {
    "track": "inbound",
    "payload": "base64-mulaw-audio",
    "chunk": 7,
    "timestamp": "2026-06-24T00:00:01.400Z"
  }
}
```

#### SDK frame types

> **Note**
>
> The SDKs export the WebSocket frame shapes and enum values used by the custom voice helpers.
>
> #### TypeScript
>
> ```typescript
> import { VoiceCallEndCause, VoiceClientFrame, VoiceCommandAction, VoiceCommandFrame, VoiceMediaTrack, VoiceNoiseReductionDirection, VoiceNoiseReductionEngine, VoiceServerFrame } from "rcs-js";
> ```
>
> #### Python
>
> ```python
> from rcs.wrapper.voice import (
>     VOICE_COMMAND_ACTIONS,
>     VoiceCallEndCause,
>     VoiceClientFrame,
>     VoiceCommandAction,
>     VoiceCommandFrame,
>     VoiceMediaTrack,
>     VoiceNoiseReductionDirection,
>     VoiceNoiseReductionEngine,
>     VoiceServerFrame,
> )
> ```
>
> #### Ruby
>
> ```ruby
> require "rcs"
>
> voice_types = Pinnacle::Wrapper::Voice::Types
>
> action = voice_types::AUDIO_PLAY
> track = voice_types::TRACK_OUTBOUND
> end_cause = voice_types::CALL_REJECTED
> noise_engine = voice_types::NOISE_KRISP
> noise_direction = voice_types::DIRECTION_OUTBOUND
> ```