> ## Documentation Index
> Fetch the complete documentation index at: https://docs.boson.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Timestamps

> Get the time each token of your input is spoken, along with the generated audio.

Set `timestamps: true` on a [`POST /v1/audio/speech`](https://docs.boson.ai/api-reference/audio/create-a-speech) request to get, along with the audio, the time each token of your input is spoken and where that token sits in your text. Use it to highlight text as it is read, show captions in sync, or drive an avatar's mouth. Timestamps work on long input, up to the usual limits of 5,000 characters or 50,000 when streamed. They also work on a [streamed response](#stream-timestamps) and over the [speech WebSocket](#over-the-websocket), and they don't change the price.

> [!WARNING]
>
> `timestamps: true` changes the response. Without `stream`, it is JSON with the audio base64-encoded inside it; with `stream: true`, it is Server-Sent Events. Writing the response body straight to a file produces an unplayable file. Decode the audio first.

## Get timestamps

Without `stream`, the response is JSON. `response_format` still selects the audio format, and the encoded audio is returned in that format in the `audio` field:

```json theme={null}
{
  "audio": "<base64-encoded audio in the requested response_format>",
  "response_format": "wav",
  "input": "Hi there.",
  "timestamps": [
    { "index": 0, "text": "Hi", "start_ms": 0, "end_ms": 240, "start_idx": 0, "end_idx": 2, "normalized": [] },
    { "index": 1, "text": "there", "start_ms": 240, "end_ms": 610, "start_idx": 3, "end_idx": 8, "normalized": [] }
  ]
}
```

The times are illustrative. `input` is your text as you sent it, and `timestamps` lists one [entry](#entries) per token, in order.

```bash cURL theme={null}
curl https://api.boson.ai/v1/audio/speech \
  -H "Authorization: Bearer $BOSON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "higgs-tts-3",
    "input": "The quick brown fox jumps over the lazy dog.",
    "response_format": "wav",
    "timestamps": true
  }' > response.json

jq -r '.audio' response.json | base64 -d > out.wav
jq -r '.timestamps[] | "\(.start_ms)-\(.end_ms) ms  \(.text)"' response.json
```

```python Python theme={null}
import base64
import os
import requests

resp = requests.post(
    "https://api.boson.ai/v1/audio/speech",
    headers={"Authorization": f"Bearer {os.environ['BOSON_API_KEY']}"},
    json={
        "model": "higgs-tts-3",
        "input": "The quick brown fox jumps over the lazy dog.",
        "response_format": "wav",
        "timestamps": True,
    },
)
resp.raise_for_status()
body = resp.json()

with open("out.wav", "wb") as f:
    f.write(base64.b64decode(body["audio"]))

for entry in body["timestamps"]:
    print(f"{entry['start_ms']}-{entry['end_ms']} ms  {entry['text']}")
```

```typescript TypeScript theme={null}
import { writeFile } from "node:fs/promises";

const res = await fetch("https://api.boson.ai/v1/audio/speech", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.BOSON_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "higgs-tts-3",
    input: "The quick brown fox jumps over the lazy dog.",
    response_format: "wav",
    timestamps: true,
  }),
});
if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`);

const body = await res.json();
await writeFile("out.wav", Buffer.from(body.audio, "base64"));

for (const entry of body.timestamps) {
  console.log(`${entry.start_ms}-${entry.end_ms} ms  ${entry.text}`);
}
```

## Entries

Each entry is one token of your input:

| Field | Type | Description |
| - | - | - |
| `index` | integer | The entry's position, counting from 0. |
| `text` | string | The token as it appears in your input: letters, numbers, and marks only. |
| `start_ms` | integer | When the token starts, in milliseconds from the start of the audio. |
| `end_ms` | integer or null | When the token ends. `null` only in a streamed entry whose end isn't known yet. |
| `start_idx`, `end_idx` | integer | Where the token is in your input: `input[start_idx:end_idx]` is `text`. |
| `normalized` | array | The tokens actually spoken for this entry. Present when `normalize` is on, the default. See [With text normalization](#with-text-normalization). |

* **Tokens, not words.** Entries follow the model's tokenizer, so a token can be part of a word: "Honestly" may come back as `H` and `onestly`. To work with words, group entries by their input positions, for example joining entries with no space between one's `end_idx` and the next one's `start_idx`.
* **Only tokens with a letter or number.** Punctuation, symbols, and [control tags](https://docs.boson.ai/models/higgs-tts/tags.md) get no entry, so there can be gaps between entries, both in the text and in time.
* **Positions count Unicode code points.** Python string indexing matches them. JavaScript strings index UTF-16 code units, which differ for characters such as emoji, so slice `Array.from(input)` instead of `input`.

## Stream timestamps

With `stream: true` (and `response_format: "pcm"`, as for any streamed request), the response is Server-Sent Events (`text/event-stream`) instead of raw PCM. Each event is a `data:` line holding a JSON object, and the stream ends with `data: [DONE]`. An event with audio looks like this:

```json theme={null}
{
  "audio": { "data": "<base64-encoded PCM>", "sample_rate": 24000 },
  "usage": { "input_chars": 44, "audio_bytes": 48000, "sample_rate": 24000, "output_duration_s": 1.0 },
  "timestamps": [
    { "index": 0, "text": "The", "start_ms": 0, "end_ms": null, "start_idx": 0, "end_idx": 3, "normalized": [] }
  ]
}
```

* **`audio.data`** is 16-bit signed little-endian mono PCM at `audio.sample_rate`. Join the chunks in order to get the full audio.
* **`usage`** holds running totals for the request so far: the characters of input, not counting control tags, and the audio sent.
* **`timestamps`** is present only on events that carry entries.

Entries arrive as the audio is generated. Each token is usually sent twice: first when it starts, with `"end_ms": null`, then again with its end. If both are known at once, it is sent once. Entries that arrive after the last audio come on a final event with `usage` and `timestamps` but no `audio`, just before `[DONE]`.

**Merge entries by `index`**: a later copy of an entry replaces the earlier one.

The samples below save the audio to `out.pcm` as it arrives, keep the latest copy of each entry, and print the timestamps once the stream ends.

```python Python theme={null}
import base64
import json
import os
import requests

entries = {}  # index -> latest copy of the entry

with requests.post(
    "https://api.boson.ai/v1/audio/speech",
    headers={"Authorization": f"Bearer {os.environ['BOSON_API_KEY']}"},
    json={
        "model": "higgs-tts-3",
        "input": "The quick brown fox jumps over the lazy dog.",
        "response_format": "pcm",
        "stream": True,
        "timestamps": True,
    },
    stream=True,
) as resp, open("out.pcm", "wb") as f:
    resp.raise_for_status()
    for line in resp.iter_lines(decode_unicode=True):
        if not line.startswith("data: "):
            continue
        payload = line[len("data: "):]
        if payload == "[DONE]":
            break
        event = json.loads(payload)
        if "audio" in event:
            f.write(base64.b64decode(event["audio"]["data"]))
        for entry in event.get("timestamps", []):
            entries[entry["index"]] = entry

for entry in sorted(entries.values(), key=lambda e: e["index"]):
    print(f"{entry['start_ms']}-{entry['end_ms']} ms  {entry['text']}")
```

```typescript TypeScript theme={null}
import { createWriteStream } from "node:fs";

const res = await fetch("https://api.boson.ai/v1/audio/speech", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.BOSON_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "higgs-tts-3",
    input: "The quick brown fox jumps over the lazy dog.",
    response_format: "pcm",
    stream: true,
    timestamps: true,
  }),
});
if (!res.ok || !res.body) throw new Error(`${res.status}: ${await res.text()}`);

const out = createWriteStream("out.pcm");
const entries = new Map<number, any>(); // index -> latest copy of the entry
const decoder = new TextDecoder();
let buffer = "";

for await (const chunk of res.body) {
  buffer += decoder.decode(chunk, { stream: true });
  let end;
  while ((end = buffer.indexOf("\n\n")) !== -1) {
    const line = buffer.slice(0, end);
    buffer = buffer.slice(end + 2);
    if (!line.startsWith("data: ")) continue;
    const payload = line.slice("data: ".length);
    if (payload === "[DONE]") continue;
    const event = JSON.parse(payload);
    if (event.audio) out.write(Buffer.from(event.audio.data, "base64"));
    for (const entry of event.timestamps ?? []) entries.set(entry.index, entry);
  }
}
out.end();

for (const entry of [...entries.values()].sort((a, b) => a.index - b.index)) {
  console.log(`${entry.start_ms}-${entry.end_ms} ms  ${entry.text}`);
}
```

## With text normalization

[Text normalization](https://docs.boson.ai/models/higgs-tts/overview.md#text-normalization) is on by default, so the model speaks the normalized text: `$42.50` is read as "forty-two dollars and fifty cents". The entries are still on your input, and each one gains a `normalized` list:

* **A token normalization left alone** gets its entry as usual, with `"normalized": []`.
* **A span normalization rewrote** becomes one entry. Its `text` is the span of your input, and it is timed from its first spoken token to its last. `normalized` lists the spoken tokens, each with `index` (counting from 0 within the span), `text`, `start_ms`, and `end_ms`.

For the input `The total is $42.50.`, the entry for `$42.50` is:

```json theme={null}
{
  "index": 3,
  "text": "$42.50",
  "start_ms": 900,
  "end_ms": 2700,
  "start_idx": 13,
  "end_idx": 19,
  "normalized": [
    { "index": 0, "text": "forty", "start_ms": 900, "end_ms": 1150 },
    { "index": 1, "text": "two", "start_ms": 1150, "end_ms": 1320 },
    { "index": 2, "text": "dollars", "start_ms": 1320, "end_ms": 1750 },
    { "index": 3, "text": "and", "start_ms": 1750, "end_ms": 1880 },
    { "index": 4, "text": "fifty", "start_ms": 1880, "end_ms": 2250 },
    { "index": 5, "text": "cents", "start_ms": 2250, "end_ms": 2700 }
  ]
}
```

The times are illustrative. When streamed, a rewritten entry is sent again as more of its tokens are spoken: merge by `index` as usual.

To time the text exactly as written, set `normalize: false`. Entries then have no `normalized` field.

## Over the WebSocket

On the [speech WebSocket](https://docs.boson.ai/models/higgs-tts/streaming.md#stream-text-in-over-a-websocket), set `timestamps: true` in `start.request`. The server then sends a `timestamps` event after the `audio` event its entries came with:

```
→ {"type": "start", "request": {"model": "higgs-tts-3", "voice": "default", "timestamps": true}}
→ {"type": "text", "text": "Hi there. "}
← {"type": "audio", "data": "..."}
← {"type": "timestamps", "entries": [{"index": 0, "text": "Hi", "start_ms": 0, "end_ms": null, "start_idx": 0, "end_idx": 2, "normalized": []}]}
← {"type": "audio", "data": "..."}
← {"type": "timestamps", "entries": [{"index": 0, "text": "Hi", "start_ms": 0, "end_ms": 240, "start_idx": 0, "end_idx": 2, "normalized": []},
                                     {"index": 1, "text": "there", "start_ms": 240, "end_ms": null, "start_idx": 3, "end_idx": 8, "normalized": []}]}
```

The entries are the same as on a streamed POST: merge them by `index`.

Each turn is timed as if it were its own POST. A turn is the text you send before a `flush`, and the next turn starts after each `flush` or `cancel`:

* `index` counts from 0.
* `start_ms` and `end_ms` count from the start of the turn's audio.
* `start_idx` and `end_idx` are positions in the turn's text: its `text` messages joined, as sent.

## Related

> [card] [Create a speech](https://docs.boson.ai/api-reference/audio/create-a-speech.md)
>
> The POST endpoint, with every request field and the timestamps response schema.

> [card] [Streaming TTS](https://docs.boson.ai/models/higgs-tts/streaming.md)
>
> Stream audio from a request, or stream text in over a WebSocket.

> [card] [Text normalization](https://docs.boson.ai/models/higgs-tts/overview.md#text-normalization)
>
> How the input is rewritten into spoken form before synthesis.

> [card] [Speech WebSocket reference](https://docs.boson.ai/api-reference/tts/speech-live.md)
>
> Every client and server event, including `timestamps`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.