> ## Documentation Index > Fetch the complete documentation index at: https://docs.boson.ai/llms.txt > Use this file to discover all available pages before exploring further. # Speech WebSocket > Send text as it arrives and receive PCM speech while you are still sending, over one WebSocket connection. Messages are JSON text frames, each with a `type` field, processed in the order they are sent. Ignore event types you don't recognize: new ones may be added. For a complete client, errors, close codes, and limits, see [Streaming TTS](https://docs.boson.ai/models/higgs-tts/streaming.md#stream-text-in-over-a-websocket). > [!NOTE] > > The speech WebSocket is TTS only and separate from [Higgs Realtime](https://docs.boson.ai/api-reference/realtime/overview.md) (`/v1/realtime`). Its events, error shape, and close codes are different. ## AsyncAPI ````yaml asyncapi-speech.json speechLive id: speechLive title: Speech live description: Client and server events of a speech WebSocket session. servers: - id: production protocol: wss host: api.boson.ai bindings: [] variables: [] address: /v1/audio/speech/live parameters: [] bindings: [] operations: - &ref_3 id: receiveClientEvents title: Receive client events description: Receive client events type: receive messages: - &ref_5 id: start payload: - name: Start TTS Session description: >- Opens the session with its speech settings. It must be the first message, and it is sent once. An accepted `start` gets no reply: the next thing the server sends is audio. A refused `start` gets an `error`, and the connection closes with `1008`. A second `start` gets an `error` (`session_already_started`), and the session goes on. type: object properties: - name: type type: string description: Always `start`. required: true - name: request type: object description: >- Speech settings: the fields of the `POST /v1/audio/speech` body listed here, with the same defaults. `input`, `stream`, and `stream_options` are refused, and `response_format` may only be `pcm` or omitted. required: true properties: - name: model type: string description: TTS model ID / public alias. enumValues: - higgs-tts-3 required: false - name: voice type: string description: >- Preset voice name or custom voice ID. Mutually exclusive with `ref_audio` / `ref_text` when explicitly provided. required: false - name: ref_audio type: string description: >- Inline reference audio for one-off cloning: an http(s) URL, data URI, or base64-encoded raw audio bytes. Supported formats: AAC, WAV, MP3, FLAC, OPUS. Inline (base64 / data-URI) payloads: max 10 MB. required: false - name: ref_text type: string description: Recommended transcript of `ref_audio`. required: false - name: normalize type: boolean description: >- Expand non-standard words (numbers, dates, currency, URLs, and so on) into spoken form before synthesis. Set to false to synthesize the text exactly as written. required: false - name: language type: string description: >- ISO 639-3 language code of the text, for example `eng` or `cmn`, used for text normalization. See `POST /v1/audio/speech` for details. required: false - name: timestamps type: boolean description: >- Send a `timestamps` event with the token timestamps after each `audio` event they came with. Entries count per turn. See [Timestamps](/models/higgs-tts/timestamps#over-the-websocket). required: false - name: response_format type: string description: 'Only `pcm`: 16-bit signed little-endian, mono, 24 kHz.' enumValues: - pcm required: false headers: [] jsonPayloadSchema: type: object required: - type - request properties: type: type: string const: start description: Always `start`. x-parser-schema-id: request: type: object description: >- Speech settings: the fields of the `POST /v1/audio/speech` body listed here, with the same defaults. `input`, `stream`, and `stream_options` are refused, and `response_format` may only be `pcm` or omitted. properties: model: type: string default: higgs-tts-3 enum: - higgs-tts-3 description: TTS model ID / public alias. x-parser-schema-id: voice: type: string default: default description: >- Preset voice name or custom voice ID. Mutually exclusive with `ref_audio` / `ref_text` when explicitly provided. x-parser-schema-id: ref_audio: type: string description: >- Inline reference audio for one-off cloning: an http(s) URL, data URI, or base64-encoded raw audio bytes. Supported formats: AAC, WAV, MP3, FLAC, OPUS. Inline (base64 / data-URI) payloads: max 10 MB. x-parser-schema-id: ref_text: type: string description: Recommended transcript of `ref_audio`. x-parser-schema-id: normalize: type: boolean default: true description: >- Expand non-standard words (numbers, dates, currency, URLs, and so on) into spoken form before synthesis. Set to false to synthesize the text exactly as written. x-parser-schema-id: language: type: string description: >- ISO 639-3 language code of the text, for example `eng` or `cmn`, used for text normalization. See `POST /v1/audio/speech` for details. x-parser-schema-id: timestamps: type: boolean default: false description: >- Send a `timestamps` event with the token timestamps after each `audio` event they came with. Entries count per turn. See [Timestamps](/models/higgs-tts/timestamps#over-the-websocket). x-parser-schema-id: response_format: type: string enum: - pcm default: pcm description: 'Only `pcm`: 16-bit signed little-endian, mono, 24 kHz.' x-parser-schema-id: x-parser-schema-id: additionalProperties: false x-parser-schema-id: startPayload title: Start TTS Session description: >- Opens the session with its speech settings. It must be the first message, and it is sent once. An accepted `start` gets no reply: the next thing the server sends is audio. A refused `start` gets an `error`, and the connection closes with `1008`. A second `start` gets an `error` (`session_already_started`), and the session goes on. example: |- { "type": "start", "request": { "model": "higgs-tts-3", "voice": "default", "normalize": true, "language": "eng" } } bindings: [] extensions: - id: x-parser-unique-object-id value: start - &ref_6 id: text payload: - name: Send Text Chunk description: >- Adds text to speak. It can be sent at any time, even while audio is playing, and can be any size, from a single token to a paragraph. Text may wait for more input before it is spoken. `flush` or `stop` speaks everything received so far. type: object properties: - name: type type: string description: Always `text`. required: true - name: text type: string description: >- The text to append. A non-string value gets an `error` (`invalid_field_value`) and the session goes on. required: true headers: [] jsonPayloadSchema: type: object required: - type - text properties: type: type: string const: text description: Always `text`. x-parser-schema-id: text: type: string description: >- The text to append. A non-string value gets an `error` (`invalid_field_value`) and the session goes on. x-parser-schema-id: additionalProperties: false x-parser-schema-id: textPayload title: Send Text Chunk description: >- Adds text to speak. It can be sent at any time, even while audio is playing, and can be any size, from a single token to a paragraph. Text may wait for more input before it is spoken. `flush` or `stop` speaks everything received so far. example: |- { "type": "text", "text": "Sure thing. I can move " } bindings: [] extensions: - id: x-parser-unique-object-id value: text - &ref_7 id: flush payload: - name: Flush Buffered Text description: >- Ends a turn and speaks everything received so far. Once that audio has been sent, the server sends `flushed`. The session stays open for the next turn. Send it at the end of every reply, since text without a natural ending waits for more text. type: object properties: - name: type type: string description: Always `flush`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: flush description: Always `flush`. x-parser-schema-id: additionalProperties: false x-parser-schema-id: flushPayload title: Flush Buffered Text description: >- Ends a turn and speaks everything received so far. Once that audio has been sent, the server sends `flushed`. The session stays open for the next turn. Send it at the end of every reply, since text without a natural ending waits for more text. example: |- { "type": "flush" } bindings: [] extensions: - id: x-parser-unique-object-id value: flush - &ref_8 id: stop payload: - name: End TTS Session description: >- Ends the session. The server speaks whatever is left, sends `finish`, and closes the connection with `1000`. Any message sent after `stop` gets an `error` (`session_stopping`), and the stop still completes. Closing the connection yourself without `stop` drops any text not yet spoken. type: object properties: - name: type type: string description: Always `stop`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: stop description: Always `stop`. x-parser-schema-id: additionalProperties: false x-parser-schema-id: stopPayload title: End TTS Session description: >- Ends the session. The server speaks whatever is left, sends `finish`, and closes the connection with `1000`. Any message sent after `stop` gets an `error` (`session_stopping`), and the stop still completes. Closing the connection yourself without `stop` drops any text not yet spoken. example: |- { "type": "stop" } bindings: [] extensions: - id: x-parser-unique-object-id value: stop - &ref_9 id: cancel payload: - name: Cancel Pending Speech description: >- Drops all text not yet spoken, and any pending `flush`, and stops the audio being generated, for example when the user interrupts. The server answers with `cancelled`. No audio for the dropped text is sent after it. The session stays open. type: object properties: - name: type type: string description: Always `cancel`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: cancel description: Always `cancel`. x-parser-schema-id: additionalProperties: false x-parser-schema-id: cancelPayload title: Cancel Pending Speech description: >- Drops all text not yet spoken, and any pending `flush`, and stops the audio being generated, for example when the user interrupts. The server answers with `cancelled`. No audio for the dropped text is sent after it. The session stays open. example: |- { "type": "cancel" } bindings: [] extensions: - id: x-parser-unique-object-id value: cancel bindings: [] extensions: &ref_2 - id: x-parser-unique-object-id value: speechLive - &ref_4 id: sendServerEvents title: Send server events description: Send server events type: send messages: - &ref_10 id: audio payload: - name: Audio Chunk description: >- A piece of speech. Decode the `data` of every `audio` event and join them in order to get the full audio: the same PCM as a streamed `POST /v1/audio/speech` response. type: object properties: - name: type type: string description: Always `audio`. required: true - name: data type: string description: 'Base64-encoded PCM: 16-bit signed little-endian, mono, 24 kHz.' required: true headers: [] jsonPayloadSchema: type: object required: - type - data properties: type: type: string const: audio description: Always `audio`. x-parser-schema-id: data: type: string description: 'Base64-encoded PCM: 16-bit signed little-endian, mono, 24 kHz.' x-parser-schema-id: x-parser-schema-id: audioPayload title: Audio Chunk description: >- A piece of speech. Decode the `data` of every `audio` event and join them in order to get the full audio: the same PCM as a streamed `POST /v1/audio/speech` response. example: |- { "type": "audio", "data": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: audio - &ref_11 id: timestamps payload: - name: Token Timestamps description: >- With `timestamps: true` in `start.request`: the timing of the tokens spoken so far, sent after the `audio` event they came with. Entries count per turn: a turn is the text sent before a `flush`, and the next turn starts after each `flush` or `cancel`. type: object properties: - name: type type: string description: Always `timestamps`. required: true - name: entries type: array description: >- Entries that changed since the last `timestamps` event. A token is usually sent twice, first with `end_ms: null`, then with its end: merge entries by `index`, a later copy replacing an earlier one. required: true properties: - name: index type: integer description: The entry's position, counting from 0 in each turn. required: false - name: text type: string description: >- The token as it appears in the turn's text: letters, numbers, and marks only. For a span that normalization rewrote, the whole span. required: false - name: start_ms type: integer description: >- When the token starts, in milliseconds from the start of the turn's audio. required: false - name: end_ms type: &ref_0 - integer - 'null' description: >- When the token ends. `null` until the end is known; a later copy with the same `index` carries it. required: false - name: start_idx type: integer description: >- Start of the token in the turn's text (all its `text` messages joined), in Unicode code points. required: false - name: end_idx type: integer description: >- End of the token in the turn's text, in Unicode code points (exclusive). required: false - name: normalized type: array description: >- Present when `normalize` is true. Empty for a token normalization left alone; for a rewritten span, the tokens spoken for it. required: false properties: - name: index type: integer description: Position within the span, counting from 0. required: false - name: text type: string description: The spoken token. required: false - name: start_ms type: integer description: >- When the token starts, in milliseconds from the start of the turn's audio. required: false - name: end_ms type: &ref_1 - integer - 'null' description: When the token ends, or `null` until known. required: false headers: [] jsonPayloadSchema: type: object required: - type - entries properties: type: type: string const: timestamps description: Always `timestamps`. x-parser-schema-id: entries: type: array description: >- Entries that changed since the last `timestamps` event. A token is usually sent twice, first with `end_ms: null`, then with its end: merge entries by `index`, a later copy replacing an earlier one. items: type: object properties: index: type: integer description: The entry's position, counting from 0 in each turn. x-parser-schema-id: text: type: string description: >- The token as it appears in the turn's text: letters, numbers, and marks only. For a span that normalization rewrote, the whole span. x-parser-schema-id: start_ms: type: integer description: >- When the token starts, in milliseconds from the start of the turn's audio. x-parser-schema-id: end_ms: type: *ref_0 description: >- When the token ends. `null` until the end is known; a later copy with the same `index` carries it. x-parser-schema-id: start_idx: type: integer description: >- Start of the token in the turn's text (all its `text` messages joined), in Unicode code points. x-parser-schema-id: end_idx: type: integer description: >- End of the token in the turn's text, in Unicode code points (exclusive). x-parser-schema-id: normalized: type: array description: >- Present when `normalize` is true. Empty for a token normalization left alone; for a rewritten span, the tokens spoken for it. items: type: object properties: index: type: integer description: Position within the span, counting from 0. x-parser-schema-id: text: type: string description: The spoken token. x-parser-schema-id: start_ms: type: integer description: >- When the token starts, in milliseconds from the start of the turn's audio. x-parser-schema-id: end_ms: type: *ref_1 description: When the token ends, or `null` until known. x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: timestampsPayload title: Token Timestamps description: >- With `timestamps: true` in `start.request`: the timing of the tokens spoken so far, sent after the `audio` event they came with. Entries count per turn: a turn is the text sent before a `flush`, and the next turn starts after each `flush` or `cancel`. example: |- { "type": "timestamps", "entries": [ { "index": 0, "text": "Hi", "start_ms": 0, "end_ms": 240, "start_idx": 0, "end_idx": 2, "normalized": [] }, { "index": 1, "text": "there", "start_ms": 240, "end_ms": null, "start_idx": 3, "end_idx": 8, "normalized": [] } ] } bindings: [] extensions: - id: x-parser-unique-object-id value: timestamps - &ref_12 id: flushed payload: - name: Flush Complete description: All audio for the turn ended by a `flush` has been sent. type: object properties: - name: type type: string description: Always `flushed`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: flushed description: Always `flushed`. x-parser-schema-id: x-parser-schema-id: flushedPayload title: Flush Complete description: All audio for the turn ended by a `flush` has been sent. example: |- { "type": "flushed" } bindings: [] extensions: - id: x-parser-unique-object-id value: flushed - &ref_13 id: cancelled payload: - name: Cancel Complete description: >- The `cancel` is done. Every `audio` event after it belongs to text sent after the `cancel`. type: object properties: - name: type type: string description: Always `cancelled`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: cancelled description: Always `cancelled`. x-parser-schema-id: x-parser-schema-id: cancelledPayload title: Cancel Complete description: >- The `cancel` is done. Every `audio` event after it belongs to text sent after the `cancel`. example: |- { "type": "cancelled" } bindings: [] extensions: - id: x-parser-unique-object-id value: cancelled - &ref_14 id: finish payload: - name: Session Finished description: >- Everything has been spoken after a `stop`. The server closes the connection next, with code `1000`. type: object properties: - name: type type: string description: Always `finish`. required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: finish description: Always `finish`. x-parser-schema-id: x-parser-schema-id: finishPayload title: Session Finished description: >- Everything has been spoken after a `stop`. The server closes the connection next, with code `1000`. example: |- { "type": "finish" } bindings: [] extensions: - id: x-parser-unique-object-id value: finish - &ref_15 id: error payload: - name: Error description: >- Something went wrong. Depending on the `code`, either the session goes on, or the server closes the connection right after this event. See [Errors](/models/higgs-tts/streaming#errors) for every code and close code. type: object properties: - name: type type: string description: Always `error`. required: true - name: code type: string description: >- Machine-readable cause, such as `session_not_started` or `invalid_api_key`. See [Errors](/models/higgs-tts/streaming#errors). required: true - name: message type: string description: Human-readable description. required: true headers: [] jsonPayloadSchema: type: object required: - type - code - message properties: type: type: string const: error description: Always `error`. x-parser-schema-id: code: type: string description: >- Machine-readable cause, such as `session_not_started` or `invalid_api_key`. See [Errors](/models/higgs-tts/streaming#errors). x-parser-schema-id: message: type: string description: Human-readable description. x-parser-schema-id: x-parser-schema-id: errorPayload title: Error description: >- Something went wrong. Depending on the `code`, either the session goes on, or the server closes the connection right after this event. See [Errors](/models/higgs-tts/streaming#errors) for every code and close code. example: |- { "type": "error", "code": "session_not_started", "message": "the first message must be start" } bindings: [] extensions: - id: x-parser-unique-object-id value: error bindings: [] extensions: *ref_2 sendOperations: - *ref_3 receiveOperations: - *ref_4 sendMessages: - *ref_5 - *ref_6 - *ref_7 - *ref_8 - *ref_9 receiveMessages: - *ref_10 - *ref_11 - *ref_12 - *ref_13 - *ref_14 - *ref_15 extensions: - id: x-parser-unique-object-id value: speechLive securitySchemes: - id: bearerAuth name: bearerAuth type: http description: >- Your Boson API key, sent as `Authorization: Bearer ` on the WebSocket handshake. An invalid key doesn't fail the handshake: the connection opens, then the server sends an `error` event and closes with `1008`. See [Authentication](/authentication). scheme: bearer extensions: [] ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.