LLM providers

Chat and tools go through BaseLLMProvider. First-party adapters cover major Chinese clouds; everything else can use LiteLLMProvider (OpenAI-compatible or LiteLLM model IDs).

A provider is configured when an API key or a custom API base is non-empty. Leaving only the official default base does not count. Local hosts such as Ollama need a reachable API address.

Worked example: mark a provider configured

Scene. You added a key for one cloud and a custom base for a local host.

1. Settings → provider list. Sidebar dots and the detail ON/OFF must agree: green only when a key or custom base is non-empty. Official default base alone stays red.
2. Ollama: type a reachable API address. Empty address is not configured.
3. Click 测试连通性 (one field, no extra env-var box). Fetch models opens a picker with per-row +/- visibility — it must not dump every model into the chat picker.
4. Clearing the key and custom base should flip the provider off.

What you should see. Red / green only. After save, a new pane’s model pill can pick a visible model. Attaching an image to MiniMax M2 should show the in-pane warning, not a provider 400.

Supported providers

ProviderResolver.PROVIDER_MAP wires config keys openai, anthropic, ollama, zhipu, volcengine / ark, bailian, qianfan, kimi, minimax to the classes above.

Usage

OpenAI

1from agenticx.llms import OpenAIProvider
2
3llm = OpenAIProvider(
4    model="gpt-4o",
5    api_key="sk-...",  # or rely on env / config
6)
7resp = llm.invoke("Summarize this in one line.")

Anthropic

1from agenticx.llms import AnthropicProvider
2
3llm = AnthropicProvider(
4    model="anthropic/claude-sonnet-4-20250514",
5    api_key="sk-ant-...",
6)
7resp = llm.invoke([{"role": "user", "content": "Hello"}])

Ollama (local)

1from agenticx.llms import OllamaProvider
2
3llm = OllamaProvider(
4    model="ollama/qwen2.5:7b",
5    base_url="http://127.0.0.1:11434",
6)
7resp = llm.invoke("Ping")

MiniMax

1from agenticx.llms import MinimaxProvider
2
3llm = MinimaxProvider(
4    model="MiniMax-M2.5",
5    api_key="...",
6    # base_url defaults to https://api.minimax.chat/v1
7)
8resp = llm.invoke("Reply with OK.")

LiteLLM

1from agenticx.llms import LiteLLMProvider
2
3# Example: third-party OpenAI-compatible endpoint (e.g. SiliconFlow)
4llm = LiteLLMProvider(
5    model="openai/Qwen/Qwen2.5-7B-Instruct",
6    api_key="...",
7    base_url="https://api.siliconflow.cn/v1",
8)
9resp = llm.invoke("Hello")

Auth profile rotation

AuthProfileManager (agenticx.llms.auth_profile) rotates multiple API keys (or logical profiles) for the same provider. It persists cooldown metadata to a JSON file (atomic write via .tmp then replace).

• get_current() — picks the next usable profile (available first, ordered by last_used; cooling profiles queued by cooldown_until).
• mark_success(profile_name) — clears error state and cooldown timestamps for that profile.
• mark_failure(profile_name, failure_type) — increments error_count, sets failure_type, and applies exponential backoff to cooldown_until.
• classify_failure(exc) — maps exceptions to billing, auth, rate_limit, or other using message heuristics.

Backoff (implemented in _compute_cooldown_ms):

BaseLLMProvider.invoke_with_profile(messages, api_key=...) forwards to invoke(..., api_key=api_key) so callers can inject the rotated secret without replacing the provider instance.

Failover routing

FailoverProvider wraps two providers: a primary and a fallback. For each of invoke, ainvoke, stream, astream, and stream_with_tools, it tries the primary unless the primary is in cooldown.

• failure_threshold (default 3) — consecutive primary failures before entering cooldown.
• cooldown_duration (default 60 seconds) — primary bypass window after the threshold is hit.
• A successful primary call resets the failure counter and clears cooldown.

1from agenticx.llms import FailoverProvider, OpenAIProvider, AnthropicProvider
2
3llm = FailoverProvider(
4    primary=OpenAIProvider(model="gpt-4o", api_key="..."),
5    fallback=AnthropicProvider(model="anthropic/claude-sonnet-4-20250514", api_key="..."),
6    failure_threshold=3,
7    cooldown_duration=120.0,
8)

Response cache

ResponseCache is an in-memory store keyed by SHA-256 (truncated) of the string prompt. Entries carry a TTL (ttl_seconds, default 300) and LRU eviction (max_entries, default 100). It is not wired automatically into LiteLLMProvider; wrap calls when you want cheaper dev loops.

1from agenticx.llms import OpenAIProvider, ResponseCache
2
3llm = OpenAIProvider(model="gpt-4o-mini", api_key="...")
4cache = ResponseCache(ttl_seconds=300, max_entries=100)
5
6def cached_invoke(text: str):
7    hit = cache.get(text)
8    if hit is not None:
9        return hit
10    out = llm.invoke(text)
11    cache.put(text, out)
12    return out

stats() exposes hits, misses, size, and hit rate.

Transcript sanitizer

Before model calls, agent_runtime runs _sanitize_context_messages on chat history. The pipeline is provider-aware in the broader sense that it enforces valid assistant / tool message chains so upstream APIs do not see orphaned tool rows or dangling tool_calls.

Behavior (simplified):

• tool messages are kept only when their tool_call_id appears on a preceding assistant tool_calls list that is fully satisfied by contiguous tool responses.
• Assistant messages with tool_calls are kept only when every call id has a matching tool response in history; otherwise tool_calls are stripped and text content is preserved.

This reduces provider 400 errors from broken tool loops after edits, retries, or partial persistence.

Vision and image input

Multimodal content is honored when the backend and model support it. MiniMax's M2 chat family (including M2, M2.1, M2.5, M2.7 and *-highspeed SKUs; excluding ids containing vl / vision) does not accept image or audio input per vendor constraints.

Studio strips image_inputs for those models before the completion request. Prefer a vision-capable model if attachments must reach the LLM.

Provider configuration (environment variables)

Values below are the ones AgenticX's config loader commonly pairs with providers.<name> in ~/.agenticx/config.yaml. LiteLLM may read additional variables depending on the model id.

Ollama is usually configured with base_url in YAML (for example http://localhost:11434); keys are not required for local inference.