Chat Service#

class saia_python.chat.ChatService(session: requests.Session, base_url: str, *, retry: RetryPolicy | bool | None = None)[source]#

Access the /chat/completions endpoint.

Parameters:

  • session – A requests.Session with auth headers configured.

  • base_url – The SAIA API base URL.

completions(model: str, messages: list[dict], *, temperature: float | None = None, top_p: float | None = None, max_tokens: int | None = None, stream: bool = False, retry: RetryPolicy | bool | None = None, **kwargs) → dict | SSEStream[source]#

Send a chat completion request.

Parameters:

  • model – Model identifier (e.g. "meta-llama-3.1-8b-instruct").

  • messages – List of message dicts with "role" and "content" keys.

  • temperature – Sampling temperature (0–2).

  • top_p – Nucleus sampling parameter (0–1).

  • max_tokens – Maximum tokens to generate.

  • stream – If True, return a generator yielding chunks.

  • **kwargs – Additional parameters forwarded to the API.

Returns:

the API response dict, with an extra "_rate_limits" key — a JSON-serializable dict of the current rate-limit headers (see RateLimitInfo). When stream=True: an SSEStream — iterate it for the response chunks; its rate_limits attribute exposes the same dict (available immediately, from the response headers).

Return type:

When stream=False