Streaming

Set stream: true on any chat completion request and Onlist returns the response as Server-Sent Events (SSE) instead of a single JSON body. The wire format is identical to OpenAI and OpenRouter, so your existing streaming client works unchanged after you point it at https://onlist.io.

The SSE wire format

When stream: true, the response Content-Type is text/event-stream. The body is a sequence of data: lines, each carrying one ChatCompletionChunk JSON object. The stream ends with a final usage-only chunk followed by the literal sentinel data: [DONE]:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":11,"completion_tokens":2,"total_tokens":13}}

data: [DONE]

Two details matter:

• The final usage chunk is the last data: frame before [DONE]. It carries choices: [] (empty) and a populated usage object. Content chunks before it do not include usage.
• data: [DONE] is a plain string sentinel, not JSON. Stop reading when you see it; do not try to parse it.

The official OpenAI SDKs handle both of these for you. If you parse the stream by hand, guard against the [DONE] sentinel and accumulate choices[].delta.content across chunks.

Examples

The request body is a standard chat completion plus stream: true. The optional provider object works the same way as on non-streaming requests.

from openai import OpenAI

client = OpenAI(
    base_url="https://onlist.io/v1",
    api_key="YOUR_ONLIST_TOKEN",
)

stream = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Stream a haiku about routing."}],
    stream=True,
)

for chunk in stream:
    # The terminal usage chunk has an empty choices list.
    if chunk.choices:
        delta = chunk.choices[0].delta
        if delta.content:
            print(delta.content, end="", flush=True)
    elif chunk.usage:
        print(f"\n[tokens: {chunk.usage.total_tokens}]")

The -N flag disables curl's output buffering so you see chunks as they arrive.

The terminal usage chunk is always emitted

Onlist rewrites only three things in your request in place — the model, the provider pin, and stream_options.include_usage — and otherwise forwards it to the upstream provider byte-faithfully, relaying the chunks back unchanged. The usage object in the terminal chunk carries prompt_tokens, completion_tokens, and total_tokens. Note that usage.cost is not included anywhere in the stream — cost is reported out of band. See Usage & cost for how to retrieve it.

Route id is written before the first byte

X-Onlist-Route-Id identifies the routing decision for the request. On a streaming response it is written to the response headers before the first SSE byte is sent, so you can read it as soon as the connection is established — you do not have to wait for the stream to finish.

This is the same opaque UUID returned on non-streaming responses, and it is your key for correlating a stream to its settled cost and metadata after the fact. It is not the body id field (that is the upstream completion id).

For the full header reference and rate-limit behavior, see the pages below.

Next steps

• Usage & cost — Read token counts and correlate a stream to its out-of-band cost via the route id.
• Limits & headers — X-Onlist-Route-Id, X-Onlist-Warnings, and Retry-After behavior in full.