Skip to content
agentling

OpenAIModel

The built-in provider adapter, its retry policy, and the Model protocol for custom adapters.

OpenAIModel is an adapter for any OpenAI-compatible chat-completions endpoint. It is the only place in agentling that knows OpenAI's wire format: it converts framework-neutral ChatMessage lists into provider payloads on the way out and converts responses back on the way in.

OpenAIModel(...)

from agentling import OpenAIModel
 
model = OpenAIModel(
    "llama-3.1-70b",
    base_url="http://localhost:8000/v1",
    api_key="not-needed-locally",
)
Parameter Default Description
model required The model name to request.
api_key env Falls back to the OpenAI SDK's environment configuration, typically OPENAI_API_KEY.
base_url None Point at any OpenAI-compatible endpoint.
context_window 128_000 Advertised context window for this model.
max_retries 2 Retries after the initial request for transient errors.
retry_base_delay 0.5 Initial backoff delay in seconds. It doubles on each retry.

Retry policy

The adapter classifies failures before deciding whether to retry:

  • Transient failures are retried with exponential backoff: rate limits, connection errors, timeouts, and 5xx responses. With the defaults, a request is attempted up to three times in total, waiting 0.5 seconds and then 1 second between attempts.
  • Permanent errors fail fast without retrying: a bad request or bad auth returns immediately, because retrying them cannot succeed.

See Errors for how model failures surface to your code.

Compatible endpoints

Because the adapter speaks the OpenAI chat-completions protocol, base_url lets you use a local server, a gateway, or another vendor's compatible API without touching the rest of your agent. See Using other providers for worked examples.

The Model protocol

Any object implementing the Model protocol works as an agent's model, so you can write your own adapter for a provider with a different wire format:

class Model(Protocol):
    async def generate(self, messages, tools=None) -> ChatMessage: ...
    def stream(self, messages, tools=None) -> AsyncIterator[Delta]: ...

generate returns one complete ChatMessage. stream yields Delta objects: small chunks of content or fragments of a tool call. The module-level agglomerate_deltas function reassembles a delta stream back into a single ChatMessage, which is how the loop can stream text to the user and still execute tools from a complete message. Swapping providers means writing one adapter, not touching the loop.

Edit this page on GitHub
Share