Streaming TTS

Streaming lets playback start before the whole clip is synthesized. Higgs TTS 3 offers two ways to stream, depending on whether you have all of the text up front.

Which mode to use

Mode	When to use it	Endpoint
Stream a response	You have the whole text up front, such as a narration script, an article, or an audiobook chapter.	`POST https://api.boson.ai/v1/audio/speech` with `stream: true`
Stream text in over a WebSocket	The text arrives in pieces, such as an LLM reply in a voice agent or live captions.	`wss://api.boson.ai/v1/audio/speech/live`

Both return the same audio: raw 16-bit / 24 kHz / mono PCM.

Stream a response

Send a normal POST /v1/audio/speech request with stream: true and response_format: "pcm". This approach streams the audio out but still needs the whole text up front. The audio streams back as it’s generated, so playback can start long before the whole text is spoken. Streaming also raises the input limit to 50,000 characters, which makes it the best way to synthesize long text. The response body is the raw PCM, with no framing around it. With timestamps: true, it is Server-Sent Events carrying the audio and the timestamps instead, as described in Stream timestamps. Read the raw PCM in chunks as it arrives:

curl -N https://api.boson.ai/v1/audio/speech \
  -H "Authorization: Bearer $BOSON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "higgs-tts-3",
    "input": "Hello, this is a streaming PCM test.",
    "response_format": "pcm",
    "stream": true
  }'

import os
import requests

BASE_URL = "https://api.boson.ai/v1"
API_KEY = os.environ["BOSON_API_KEY"]

payload = {
    "model": "higgs-tts-3",
    "input": "Hello, this is a streaming PCM test.",
    "voice": "default",
    "response_format": "pcm",
    "stream": True,
}

# The response body is a raw 16-bit / 24kHz / mono PCM byte stream — collect chunks directly.
pcm = bytearray()
with requests.post(
    f"{BASE_URL}/audio/speech",
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    },
    json=payload,
    stream=True,
    timeout=180,
) as r:
    r.raise_for_status()
    for chunk in r.iter_content(chunk_size=4096):
        if chunk:                       # first non-empty chunk == time-to-first-audio
            pcm.extend(chunk)

with open("out.pcm", "wb") as f:
    f.write(pcm)

import { writeFile } from "node:fs/promises";

const res = await fetch("https://api.boson.ai/v1/audio/speech", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.BOSON_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "higgs-tts-3",
    input: "Hello, this is a streaming PCM test.",
    response_format: "pcm",
    stream: true,
  }),
});

if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

// Raw 16-bit / 24kHz / mono PCM byte stream — no "data:" lines, just collect the bytes.
const reader = res.body.getReader();
const pcmChunks: Uint8Array[] = [];
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  if (value) pcmChunks.push(value);     // first chunk == time-to-first-audio
}

await writeFile("out.pcm", Buffer.concat(pcmChunks));

Each sample saves the audio to out.pcm. To play it as it streams instead, write each chunk to a speaker, as the WebSocket sample below does. If you close the connection before the stream ends, you’re billed only for the text whose audio was sent in full: long input is synthesized in pieces, and billing stops at the end of the last complete piece.

Stream text in over a WebSocket

When the text arrives a few words at a time, such as an LLM reply in a voice agent, open the speech WebSocket and send the text as you get it. Audio starts streaming back while you’re still sending text, so listeners don’t have to wait for the full reply. The session takes the same settings as a POST body, minus input. Send them in start, then send text as it arrives, and stop to speak what’s left and end the session. The audio comes back in audio events, as base64-encoded PCM in the same format as a streamed response.

Text without a natural ending waits for more text. End each LLM reply with flush, or with stop if it’s the last message of the session.

You need Python 3.10 or later with the websockets (14 or later) and sounddevice packages, and speakers or headphones, since the reply plays as audio:

export BOSON_API_KEY="bai-xxxx"
pip install "websockets>=14" sounddevice

Send a reply a few words at a time and play the audio as it comes back:

Python

import asyncio
import base64
import json
import os
import time

import sounddevice as sd
import websockets

REPLY = ["Sure thing. ", "I can move that ", "appointment to ", "Thursday afternoon. ", "Does that work?"]


async def send_reply(ws, t0):
    # Stream the reply in a few words at a time, as an LLM would
    for piece in REPLY:
        await asyncio.sleep(0.2)
        await ws.send(json.dumps({"type": "text", "text": piece}))
        print(f"{time.monotonic() - t0:.2f}s sent {piece!r}")

    # Speak whatever is left, then end the session
    await ws.send(json.dumps({"type": "stop"}))


async def speak():
    # Speaker output matching the audio format: 24 kHz, 16-bit, mono PCM
    speaker = sd.RawOutputStream(samplerate=24000, channels=1, dtype="int16")
    speaker.start()

    # max_size=None: audio events have no fixed size
    async with websockets.connect(
        "wss://api.boson.ai/v1/audio/speech/live",
        additional_headers={"Authorization": f"Bearer {os.environ['BOSON_API_KEY']}"},
        max_size=None,
    ) as ws:
        # Start the session
        await ws.send(json.dumps({
            "type": "start",
            "request": {"model": "higgs-tts-3", "voice": "default"},
        }))

        # Send the text while receiving the audio
        t0 = time.monotonic()
        sender = asyncio.create_task(send_reply(ws, t0))

        # Play audio as it streams in, until the server closes after `finish`
        first_audio = True
        async for msg in ws:
            event = json.loads(msg)
            if event["type"] == "audio":
                if first_audio:
                    print(f"{time.monotonic() - t0:.2f}s first audio")
                    first_audio = False
                # Write in a thread so playback doesn't block sending
                await asyncio.to_thread(speaker.write, base64.b64decode(event["data"]))
            elif event["type"] == "error":
                raise RuntimeError(f"{event['code']}: {event['message']}")
        await sender

    speaker.stop()  # returns once the last audio has played

asyncio.run(speak())

You should hear the reply start before all of it has been sent: the log shows the first audio arriving while text is still going out. To keep the connection open for more replies, see Multiple turns and interruptions. For every event and its fields, see the Speech WebSocket reference.

Multiple turns and interruptions

One connection can carry a whole conversation. Instead of stop, send flush at the end of each reply and wait for flushed, which the server sends once that reply’s audio is out; the session stays open for the next turn. When the user interrupts, send cancel: the server drops the text not yet spoken, stops the audio in progress, and answers with cancelled. Every audio event after cancelled belongs to new text.

Errors

Errors arrive as a flat error event, with no nested error object and no param:

{
  "type": "error",
  "code": "session_not_started",
  "message": "the first message must be start"
}

Not every error ends the session. The sample above stops at the first error; in production, check the code:

`code`	Session	Meaning
`session_not_started`	Closes (`1008`)	The first message wasn’t `start`.
`invalid_field_value`	Closes (`1008`) for a bad `start`; goes on for a bad `text`	A bad `start.request` (including the refused fields `input`, `stream`, and `stream_options`), a `text` whose `text` isn’t a string, or a piece of text with an unknown control tag, which is skipped.
`unsupported_response_format`	Closes (`1008`)	`response_format` other than `pcm`.
`session_limit_exceeded`	Closes (`1008`)	The API key already has as many open speech sessions as its limit allows. See Limits and billing.
`input_too_long`	Closes (`1008`)	More than 50,000 characters of text waiting to be spoken.
`invalid_message`	Goes on; closes (`1008`) if it is the first message	Not a JSON object in a text frame, or an unknown `type`.
`session_already_started`	Goes on	A second `start`.
`session_stopping`	Goes on	A message after `stop`.
`session_idle_timeout`	Closes (`1000`)	No message and nothing being spoken for 5 minutes, or no `start` within 5 minutes of connecting.
`internal_error`	Closes (`1011`)	Generating audio failed. A backend error may carry its own `code` instead.

Refusals shared with the REST API use the same codes as REST errors (or the error’s type when it has no code), followed by a close with 1008:

invalid_api_key: a missing or invalid key. The connection opens first, then the server sends the error and closes.
rate_limit_exceeded: the key is over its rate limit.
no_billing_account, insufficient_quota, monthly_cap_reached, contract_ended: billing refusals, at start or mid-session. The key is checked again before each piece of text is synthesized.
Problems with the start settings that a POST would also reject, such as a model the key can’t use or an unknown voice, get the same code as on the POST.

At start, rate-limit and session-limit refusals are checked before billing, so a key over either limit gets that error even if its account also can’t pay. The server closes with standard WebSocket close codes, each preceded by an error event except the normal close after finish:

Close code	Meaning
`1000`	Normal: after `finish`, or after `session_idle_timeout`.
`1008`	Refused or stopped by policy: a bad first message or `start`, an authentication, billing, rate-limit, or session-limit refusal, or too much text waiting.
`1011`	A server error while generating audio.

Limits and billing

Sessions: an API key can hold as many open sessions as its rate limit for the model allows per second, rounded down, with a minimum of 1. For example, a limit of 10 requests per second or 600 per minute allows 10 open sessions. A connection over the cap is refused with session_limit_exceeded and isn’t charged against the rate limit.
Rate limit: each session counts as one request against the key’s speech rate limit, at start, not per text message. It’s the same limit POST /v1/audio/speech uses, so sessions and POST requests draw on one budget.
Text waiting: at most 50,000 characters received but not yet spoken, the same limit as a streamed POST’s input. More closes the session with input_too_long.
Idle timeout: a session with no client message and nothing being spoken for 5 minutes is closed (session_idle_timeout). Text waiting for more input doesn’t count as activity, so send flush.
Billing: the same as the POST: per character of text that is spoken, with control tags not counted. These aren’t billed:
- text dropped by cancel or by closing the connection
- a piece of text whose audio was cut off partway
- a piece of text with nothing to speak, such as ... on its own
- a piece of text refused for an unknown control tag
Features: text normalization, voices, reference audio, control tags, and timestamps work as on the POST. Response formats other than pcm are not available.

Speech WebSocket reference

Every client and server event, with its fields.

Create a speech

The POST endpoint and every request field.

Common error types

The REST error codes, reused by the WebSocket for authentication and billing refusals.

Pricing

TTS pricing per character.

Get started

Account & billing

Higgs Realtime

Higgs TTS 3

Higgs Avatar

Help

Streaming TTS

Which mode to use

Stream a response

Stream text in over a WebSocket

Multiple turns and interruptions

Errors

Limits and billing

Speech WebSocket reference

Create a speech

Common error types

Pricing

​Which mode to use

​Stream a response

​Stream text in over a WebSocket

​Multiple turns and interruptions

​Errors

​Limits and billing

​Related

Speech WebSocket reference

Create a speech

Common error types

Pricing

Which mode to use

Stream a response

Stream text in over a WebSocket

Multiple turns and interruptions

Errors

Limits and billing

Related