Chat Service#

class saia_python.chat.ChatService(session: requests.Session, base_url: str, *, retry: RetryPolicy | bool | None = None)[source]#

Access the /chat/completions endpoint.

Parameters:

  • session – A requests.Session with auth headers configured.

  • base_url – The SAIA API base URL.

completions(model: str, messages: list[dict], *, temperature: float | None = None, top_p: float | None = None, max_tokens: int | None = None, stream: bool = False, retry: RetryPolicy | bool | None = None, **kwargs) → dict | SSEStream[source]#

Send a chat completion request.

Parameters:

  • model – Model identifier (e.g. "meta-llama-3.1-8b-instruct").

  • messages – List of message dicts with "role" and "content" keys.

  • temperature – Sampling temperature (0–2).

  • top_p – Nucleus sampling parameter (0–1).

  • max_tokens – Maximum tokens to generate.

  • stream – If True, return a generator yielding chunks.

  • **kwargs – Additional parameters forwarded to the API.

Returns:

the API response dict, with an extra "_rate_limits" key — a JSON-serializable dict of the current rate-limit headers (see RateLimitInfo). When stream=True: an SSEStream — iterate it for the response chunks; its rate_limits attribute exposes the same dict (available immediately, from the response headers).

Return type:

When stream=False

completions_structured(model: str, messages: list[dict], response_model: type[ModelT], *, temperature: float | None = None, top_p: float | None = None, max_tokens: int | None = None, retry: RetryPolicy | bool | None = None, **kwargs) → ModelT[source]#

Send a chat completion and return the answer as a validated model.

Sends response_model’s JSON Schema as the response_format (see response_format_for()), which SAIA enforces on the server, then validates the answer into an instance (see parse_structured()). Non-streaming only. To keep the raw response too (usage, _rate_limits), call those two helpers around completions() yourself.

Parameters:

  • model – Model identifier (e.g. "meta-llama-3.1-8b-instruct").

  • messages – List of message dicts with "role" and "content" keys. SAIA uses the schema only to constrain the output and does not add it to the prompt, so say what each field should hold.

  • response_model – The Pydantic v2 model class the answer must match.

  • temperature – Sampling temperature (0–2).

  • top_p – Nucleus sampling parameter (0–1).

  • max_tokens – Maximum tokens to generate. Reasoning models think before they answer, so leave room, or turn thinking off where the chat template allows it (e.g. Qwen: chat_template_kwargs={"enable_thinking": False}).

  • retry – Overrides the service’s rate-limit retry policy for this call, as in completions().

  • **kwargs – Additional parameters forwarded to the API.

Returns:

An instance of response_model.

Raises:

StructuredOutputError – The response holds no answer that validates, e.g. because the token budget ran out (finish_reason is "length"). The error keeps the full response, usage included.