Clapilot-Agent Providers and Models

The provider layer is how the native runtime turns "run this agent turn" into a concrete upstream model request. It owns the provider catalog, secret resolution, model selection and fallback, per-model context/output limits, and the choice of execution adapter (direct HTTP vs a subscription bridge). Configuration lives in Postgres (agent_provider_configs plus app_settings.native_model_routing); env-based fallback applies only when the DB configuration is empty.

Implementation:

• services/clapilot-agent/src/providers/index.mjs
• services/clapilot-agent/src/config.mjs

Supported provider types

agent_provider_configs.provider_type currently supports:

• openai
• anthropic
• aws_bedrock
• azure_openai
• openai_compatible
• google_gemini
• xai
• ollama

The UI now exposes providers only after an admin explicitly adds them from ClapilotAICore -> Provider & Modelle -> Provider. Older seeded placeholder rows are hidden unless they already carry real config or models.

Provider config shape

Each provider row stores:

• slug
• label
• enabled
• is_default
• base_url
• encrypted API key material
• model_defaults
• models
• fallback_order
• metadata

The runtime reads these rows, decrypts the stored secret, merges inherited app-level secrets, and builds an executable provider catalog.

Provider rows are now fully user-managed from the settings UI:

• add a supported provider explicitly via Provider
• configure secrets / base URLs / models only for the providers you actually want visible
• use the explicit Use provider for routing switch to include or exclude a provider from model selection and fallback routing without deleting its credentials or model configuration
• remove providers again from the same panel; save deletes the removed rows from agent_provider_configs

Adding the first model to a newly created provider enables that provider automatically. After that, the saved enabled value is authoritative: saving the provider form never silently derives or changes it from the model list. Both the UI and POST /api/agent-runtime/config reject an enabled provider with no configured models and ask the admin to add a model or disable the provider.

Every persisted provider enable/disable transition is written in the same database transaction to agent_provider_config_audit_events. The event records the provider slug, previous and next state, authenticated admin when available, change source, and model/label context so a provider disappearing from routing can be traced without relying on the provider row's mutable updated_at timestamp.

OpenAI Compatible is intentionally multi-instance now. Admins can add several compatible backends, give each row its own label, and route models against those individual provider slugs instead of one shared openai_compatible/* namespace.

Secret resolution

Secret priority is:

1. provider-specific encrypted secret in agent_provider_configs
2. inherited secret from app_settings
3. environment fallback when DB config is missing

Special case for OpenAI:

• codex_auth_json is still synchronized into the native Codex auth store so Codex app-server / orchestrator features can use it
• new Codex subscription connections use Codex app-server device-code authentication: Clapilot displays OpenAI's one-time code and verification page, polls the managed login, and persists the resulting refreshable credential automatically instead of asking the administrator to copy a localhost redirect URL
• device-code login must be enabled in the personal ChatGPT security settings or by the ChatGPT workspace administrator; OpenAI currently labels this login mode beta
• OpenAI-Codex provider rows remain configurable and can still use Codex OAuth for model discovery and Codex-brokered execution paths
• standard native chat model routing can include OpenAI-Codex rows again, but those rows execute through the Codex bridge instead of the generic OpenAI HTTP adapter
• the Codex bridge now registers a Clapilot MCP stdio server inside the private Codex CODEX_HOME, so Codex-backed GPT runs can call the same Clapilot-native app tools as the direct native runtime path as long as they pass the current session_key
• embeddings still exclude OpenAI-Codex rows, so embedding calls only use direct API-key-backed OpenAI providers

Supported auth modes

In practice the runtime currently supports:

OpenAI-compatible providers are intentionally tolerant of missing API keys so local or internal compatible endpoints can be used without bearer auth. During streaming, the native OpenAI-compatible transport recognizes the commonly used reasoning_content, reasoning, and thinking delta fields (plus typed reasoning content parts). It forwards them through the internal stream as a temporary <think>...</think> boundary while keeping the completed assistant text clean. The web and iOS/macOS chats use that boundary only for an unboxed, animated in-progress reasoning line. Explicit reasoning tags emitted inside ordinary model content remain supported, and repeated orphan </think> markers produced by local chat templates are normalized so their intermediate text cannot leak into the final answer or completion notifications. Ollama is a first-class provider rather than an OpenAI-compatible alias. Local servers use a base URL such as http://host.docker.internal:11434 without authentication; Ollama Cloud uses https://ollama.com with a bearer API key. API keys cover inference and per-request metrics, but Ollama does not expose 5-hour or weekly plan windows through the API-key API. The same provider can therefore retain an encrypted ollama.com browser Cookie header for Subscription Usage. That usage session remains eligible even if no models are routed through the provider, and it is never used for model inference. For Bedrock, Clapilot stores the encrypted provider secret in the normal secret slot as the raw bearer token. The region stays in provider metadata. Older SigV4-based Bedrock rows are still read for compatibility, but the supported admin configuration path is now bearer token only.

Model catalogs

Each provider has its own models array plus model_defaults.

Providers no longer auto-populate chat models when a row exists but no models were selected. A provider only contributes model refs to routing after models were explicitly added in the UI.

The runtime builds a model catalog of fully qualified model refs keyed by provider slug:

• openai-default/<model>
• openai/gpt-* for GPT models backed by Codex OAuth; legacy openai-codex-default/<model> refs still resolve for existing saved settings
• anthropic-default/<model>
• claude-default/<model>
• aws-bedrock-default/<model>
• azure-openai-default/<model>
• openai-compatible-default/<model>
• google-gemini-default/<model>
• xai-default/<model>
• ollama-default/<model>
• additional OpenAI-compatible rows use their own slug, for example openai-compatible-2/<model>

The global model-routing list stored in app_settings.native_model_routing is then used as a cross-provider priority chain for the standard native chat path. This list may contain direct API-backed rows, Google Gemini text-chat models, and subscription-backed rows such as OpenAI-Codex or Anthropic-Claude. The settings UI now keeps those rows in the same ordered list but annotates each entry with its access mode (API vs Subscription) and execution route (for example Codex Bridge or Anthropic setup-token). The same native_model_routing payload also stores the default Clapilot-code profile for non-specialized providers plus separate global provider/model defaults for TTS, STT, and image generation. The unchanged wire value non_specialized_agent_core=native selects Clapilot-code (Assistant), while embedded_pi selects Clapilot-code (Coding) without changing the underlying provider/model transport. A provider row can override that global default through agent_provider_configs.metadata.agent_core; accepted stored values remain native and embedded_pi, while absent, null, or inherit inherits the global setting. Resolution order is explicit request agentCore, explicit request runtimePath, provider metadata.agent_core, global non_specialized_agent_core, then native.

Adaptive model and profile routing

native_model_routing.adaptive_routing adds a policy layer in front of the existing provider resolver. It only considers configured entries from native_model_routing.priority; it does not create a second provider catalog or bypass provider health, authentication, rate-limit cooldowns, fallback rules, or public-channel subscription restrictions.

• mode=off disables decisions.
• mode=shadow is the default. ClapilotAICore scores and records the recommendation, but executes the normal base model and profile.
• mode=apply may select another candidate and, when route_harness=true, choose Clapilot-code (Coding) for coding/repository work or Clapilot-code (Assistant) for other work.
• An explicit request/session model, explicit agentCore/runtimePath, specialized agent, or Mixture-of-Agents preset is a hard override. The router never changes provider or profile inside an active tool loop.
• candidate_limit, exploration_rate, min_samples, and switch_margin bound selection and adaptation. Exploration is capped at 25 percent by the configuration normalizer.

Every shadow or applied decision is stored in agent_adaptive_route_decisions with the task class, candidate scores, protection/decision reason, base/recommended/selected model and profile, and a terminal technical outcome. The first policy version learns from completion/failure, retry/fallback count, tool errors, token counts, and duration over the prior 90 days. It deliberately does not infer user satisfaction from prose. Recent successful session routing is sticky when its score remains within the configured switch margin.

Adaptive routing is configured under Settings → ClapilotAICore → Model Routing, in the "Adaptive model and profile routing (learning)" section below the Routing Models editor. The same page shows the collected decision data: summary counts, aggregated per-task-class outcomes from the 90-day learning window, and the most recent decisions with base model, recommendation, protection/decision reason, and outcome. The former section in the OpenCore models panel now only links to this page.

Adaptive routing and Routing Models are connected: when a route/<slug> Routing Model resolves the base selection, the learned-outcome layer runs inside the member ranking instead of on the global priority list. The static capability/property score stays the primary signal; recorded outcomes may only reorder members whose static scores are within a fixed tie band (1.5 points — less than one capability match, so a configured capability tag can never be overridden). An alternative member needs at least min_samples recorded outcomes for the detected task class and must beat the current member's learned score by switch_margin. In shadow mode the would-be reorder is only recorded; in apply mode the member order (selection plus fallback chain) is actually reordered. Exploration is intentionally not applied inside Routing Models.

Every routing-model request also writes a decision row with routing_model_slug, the static tag-based pick as the base model, and the outcome-based pick as the recommendation, so the settings data view can report per-route agreement between the configured ranking and learned outcomes. The sessions-level adaptive layer never re-switches a routing-model selection; global candidate scoring applies only to non-routing requests.

Model Routing / Routing Models

Routing Models are administrator-defined virtual models with references in the form route/<slug>. Each Routing Model contains an ordered list of member models. A member can be annotated with capabilities and properties so ClapilotAICore can select a suitable model for each request while retaining a deterministic fallback chain.

Model Routing is off by default. Administrators configure it under Settings → ClapilotAICore → Model Routing, where they can enable the feature and optionally choose a default Routing Model. When a default is set, requests without an explicitly selected model are sent through that route. Enabled route/<slug> entries are also available in regular model pickers for explicit selection.

The capability taxonomy is coding, frontend_design, law, taxes, finance, writing, translation, research, math, data_analysis, design_3d, image_understanding, summarization, and general. The property taxonomy is fast, cheap, expensive, high_reasoning, long_context, vision, local_private, tool_use, and creative.

At execution time, ClapilotAICore classifies the request, scores the configured members against the detected capabilities and properties, and picks the highest-scoring member. Member order breaks score ties. The remaining ranked members become the fallback chain, so a failed first choice can continue through the same Routing Model without discarding the classification result. When adaptive routing is not off, recorded technical outcomes can additionally refine this ranking within a bounded tie band; see the adaptive-routing section above for the exact rules.

Management through agent/chat tools is intentionally not exposed in this iteration; Routing Models are managed through the admin web UI only.

Google Gemini provider rows support both media routing and regular text chat. The chat model list filters out obvious Gemini media-only entries such as TTS, Live/Realtime, image, video, embedding, Imagen, and Lyria model ids, while Gemini text/chat models execute through the native generateContent transport with Clapilot tool-calling support.

xAI Grok provider rows are subscription-backed by default. Admins start the xAI OAuth flow from the provider detail dialog, paste the callback URL, and Clapilot stores the refreshable OAuth JSON in agent_provider_configs. The default catalog seeds grok-4.3 for general chat, grok-build-0.1 for coding-oriented runs, and Grok Imagine image/video model ids for media routing. Chat routing filters grok-imagine-* models out of normal conversation priority, while image/video media settings can select them explicitly. The runtime calls xAI's OpenAI-family Responses endpoint (/responses) for chat and refreshes the xAI OAuth token in place when it expires; if xAI rotates the refresh token, the new OAuth JSON is persisted back to the provider row. Direct xAI API keys remain possible by saving the provider with authMode=api_key, but the built-in catalog entry is intended for SuperGrok / X Premium-style subscription access rather than pay-as-you-go API billing.

Ollama provider rows support local and Cloud execution through Ollama's native API. /api/tags supplies the model catalog, /api/chat handles streaming chat, images, structured output, and the native Clapilot tool loop, and /api/embed handles memory and RAG embeddings. Local rows are considered configured when they have a valid base URL; Cloud rows additionally require an API key. Because the web and agent services normally run in Docker, a host-installed Ollama server should use http://host.docker.internal:11434 rather than localhost.

Provider/account capabilities are enforced before chat routing. A Codex OAuth provider backed by a ChatGPT account accepts only OpenAI GPT/Codex model ids; Grok ids must belong to an xai provider. The admin save endpoint rejects incompatible enabled configurations, the settings UI omits them from chat priority, and the native runtime warns about stale incompatible rows at configuration load while excluding those models from its catalog. Normal configured-priority fallbacks remain available, so a stale invalid model reference cannot consume the user turn.

For Anthropic-Claude with a Claude setup-token, model discovery is intentionally catalog-based instead of live-validated. Clapilot now exposes the shipped Claude subscription catalog (latest, claude-opus-5, claude-fable-5, claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5-20251001) without probing /v1/models or /v1/messages first, because subscription-backed runs no longer execute through direct Anthropic HTTP transport.

At runtime, those Anthropic subscription rows execute through the local Claude CLI bridge:

• the stored setup-token is injected as runtime auth for the claude CLI
• chat streaming uses Claude CLI stream-json output instead of synthetic word-chunk replay
• each Clapilot session_key now keeps a hidden resumable Claude bridge session id, so follow-up turns continue the same Claude Code conversation instead of rebuilding a fully stateless CLI run every time
• built-in Claude CLI tools such as Bash, Read, Grep, Glob, WebFetch, and WebSearch can run when the model chooses them
• Clapilot-native tools are exposed to Claude through an MCP stdio bridge, so app tools such as documents_list, documents_get, calendar_*, aufgaben_*, notizen_*, and other native tool-proxy contracts remain reachable from the same subscription-backed chat run
• normal native chat runs do not apply a hard default kill timeout to the Claude CLI bridge anymore; callers may still pass an explicit run timeout when they need one, and CLAPILOT_AGENT_CLAUDE_CLI_TIMEOUT_MS can restore a bridge-wide default limit
• when the Clapilot session model is changed or cleared, that hidden Claude bridge binding is invalidated so the next turn starts a fresh Claude session with the new model context

The dropdown is therefore a shipped subscription catalog, while actual execution happens through the local Claude CLI bridge plus Clapilot MCP exposure.

Selection algorithm

When a run starts, provider selection works like this:

1. if no explicit model is requested and native_model_routing.priority exists, use its first model
2. otherwise parse the requested model for provider and model hints
3. otherwise fall back to the default provider
4. otherwise fall back in order: OpenAI with credentials, Azure OpenAI with credentials, Anthropic with credentials, AWS Bedrock with credentials, then first enabled provider

If a selected model later fails, the runtime retries using fallback model refs:

• first the remaining entries in the global priority list, if the selected model came from that list
• otherwise the selected provider’s own fallback_order

Context windows and output limits

The native runtime supports per-model limits, mainly to manage history replay and output caps.

Resolved limits come from:

• model-specific metadata maps
• per-model limits embedded in the provider’s saved models catalog entries
• model_defaults.context_window
• model_defaults.max_input_tokens
• model_defaults.max_output_tokens
• model_defaults.max_completion_tokens
• Clapilot’s shipped fallback model-limit catalog for current OpenAI + Anthropic top-tier models when provider config does not include explicit limits
• only if none of the above exist, the generic native emergency fallback is used (32,000 context / 4,096 reserved output)

These values are used by:

• prompt budget estimation
• memory flush triggering
• session compaction
• max token values for provider calls

This is the part the Settings UI exposes as per-model context window and output cap. If no explicit override is stored for a known OpenAI/Anthropic model, the provider settings UI still shows the effective shipped fallback value and labels it as such, so admins can see the real runtime limit without having to save a manual override first. For OpenAI providers with Codex OAuth, the UI also exposes a per-model 1M-Kontextfenster aktivieren toggle. It is disabled by default and only affects shipped fallback resolution for eligible models; explicit token overrides still take precedence.

For OpenAI providers with stored Codex OAuth, the shipped fallback catalog now distinguishes between:

• the plain OpenAI API fallback profile for raw model IDs such as gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, and gpt-5.4
• Codex-auth chat defaults such as openai/latest; the settings dialog loads the live Codex app-server model catalog from the installed Codex CLI so unavailable rollout models do not get added from stale fallbacks.

latest is a Clapilot-owned moving alias, not an upstream model ID. openai/latest resolves to the first concrete model in the enabled Codex OAuth provider catalog, currently gpt-5.6-sol after migration 191, and Claude subscription providers may use <claude-provider-slug>/latest for the first concrete model in the shipped Claude subscription catalog, currently claude-opus-5 after migration 224. Concrete refs such as openai/gpt-5.6-terra, openai/gpt-5.5, claude-default/claude-opus-5, or claude-default/claude-fable-5 remain supported as explicit pins.

The current OpenAI family is role-aware rather than a single replacement slug:

• gpt-5.6 is the direct-API family alias and routes to gpt-5.6-sol upstream.
• gpt-5.6-sol is the frontier model.
• gpt-5.6-terra is the balanced everyday model.
• gpt-5.6-luna is the efficient high-volume model.
• gpt-5.3-codex-spark is Codex-subscription-only and is intentionally excluded from OpenAI API provider defaults.

Direct OpenAI and Azure OpenAI GPT-5.6 runs use the Responses endpoint, including text-only runs, so native reasoning and function tools do not fall into the incompatible Chat Completions default-reasoning combination. Codex OAuth runs continue through the Codex app-server bridge.

Codex OAuth model cards in Settings -> ClapilotAICore -> Model providers expose a per-model reasoning-effort selector. The available levels and the displayed default come from the installed Codex app-server model/list catalog, because support differs by model. Clapilot stores explicit choices in provider metadata as modelReasoningEfforts and applies them to new threads and subsequent turns for standard native chat, channel sessions, interactive Agent Orchestrator sessions, and Codex-backed reference runs. Leaving a model on Codex default lets the installed Codex runtime choose its advertised default. An explicit Agent Orchestrator runtime effort still takes precedence for that session.

This matters because the public GPT-5.4 API docs advertise a larger maximum context window, while Codex-auth chat runs currently use a smaller effective default budget in practice. The native runtime therefore resolves Codex-auth OpenAI chat against its own shipped fallback profile instead of blindly reusing the public API maximums. If the per-model 1M-Kontextfenster toggle is enabled, the runtime switches that model’s fallback from the Codex-auth profile back to the larger public OpenAI profile.

Current shipped fallback coverage includes the native aliases currently used in this repo, such as:

• latest
• gpt-pro
• gpt-5.6
• gpt-5.6-sol
• gpt-5.6-terra
• gpt-5.6-luna
• gpt-5.5
• gpt-5.4
• gpt-5.4-mini
• gpt-5.3-codex-spark (Codex OAuth only)
• claude-opus-5
• claude-fable-5
• claude-opus-4-8
• claude-opus-4-7
• claude-opus-4-6
• claude-sonnet-4-6
• claude-haiku-4-5-20251001
• grok-4.3
• grok-build-0.1
• grok-imagine-image-quality
• grok-imagine-image
• grok-imagine-video-1.5
• grok-imagine-video

Remote model discovery

The runtime can fetch upstream model lists for admin configuration:

• Anthropic via its models endpoint
• AWS Bedrock via the regional foundation-models control-plane API
• Azure OpenAI via its v1 models endpoint
• OpenAI and OpenAI-compatible providers via OpenAI-style models endpoints
• xAI Grok via OpenAI-style /models; subscription OAuth rows fall back to the saved Grok catalog when model-list scope is unavailable
• Ollama via native /api/tags for local servers and Ollama Cloud

Internal API:

• GET /internal/provider-models?slug=<provider-slug>
• GET /internal/provider-status?slug=<provider-slug>

Enabled OpenAI API-key and Codex OAuth rows are write-preflighted when saved or activated. The probe uses the actual Chat Completions and/or Responses endpoint selected by the configured models, requests only one output token, and stores no credential or response content. A 401/403 scope response records the explicit required scopes (model.request, and api.responses.write for Responses), marks the provider unhealthy, and removes it from default/model routing. Runtime auth failures are not replayed against the same provider: the provider is quarantined for the current runtime configuration and the next eligible cross-provider fallback is selected.

Enabled OpenAI-compatible rows also run a one-token Chat Completions capability probe for every configured model when the row is saved or activated. Token-free results are stored in metadata.modelAvailability. Definitive load errors such as Failed to load model mark only the affected model unavailable: the settings UI labels it as not ready, and native model catalogs and default selection exclude it while leaving healthy sibling models usable. Inconclusive network or server failures do not disable a model. If a provider later returns a definitive model-load error during execution, the shared classifier records model_not_found, proceeds through the configured fallback chain, and returns a clear user-facing error when no fallback succeeds.

Clapilot proxies this through the settings UI so admins can populate the provider model list from the upstream API instead of maintaining it manually. For OpenAI providers, remote model discovery now prefers the stored Codex OAuth access token when present and only falls back to the saved API key if OAuth is unavailable or unusable. For Codex OAuth providers, each returned model can also include reasoningEfforts / reasoning_efforts and defaultReasoningEffort / default_reasoning_effort, normalized from the model-specific Codex app-server capability record. The settings UI uses these fields to avoid offering unsupported effort levels. For AWS Bedrock providers, the supported setup path is the Bedrock bearer token / API key. Clapilot tries to load both Bedrock foundation-model ids and available inference profiles from the regional control plane, and stores the profile ARN when Bedrock returns one. If that discovery path is unavailable for the current account/auth scope, manual model entry remains the fallback for Converse model ids, inference profiles, or provisioned-throughput ARNs. For Azure OpenAI providers, Clapilot normalizes the configured endpoint to /openai/v1 and keeps all Azure data-plane requests under that path. If apiVersion is left at v1, no extra query parameter is added; only preview or date-style Azure API versions are appended as ?api-version=.... For authentication, Clapilot first tries Azure API-key auth and also tolerates OpenAI-style bearer auth with the same stored key for Azure v1 compatibility. Depending on how the Azure resource is set up, the remote list may expose model ids, deployment ids, or both, so manual deployment entry remains available in the settings UI. For xAI Grok providers, Clapilot uses the current xAI OAuth discovery document under https://auth.x.ai/.well-known/openid-configuration for token refresh and uses https://api.x.ai/v1 for model and response traffic. The provider status card treats a parseable xAI OAuth credential as connected even when the live model list cannot be fetched with the current subscription scope. For openai_compatible providers, the settings UI also stores a per-model Visionfähig flag in provider metadata. That flag is used by downstream features such as document vision fallback so custom backends only appear in vision selectors when the admin marked them as multimodal-capable.

Provider request building also reads that flag, per attempt, because capability differs per candidate:

• A vision-capable model keeps the attached image inline as image content, unchanged.
• A text-only model does not receive the image binary. Sending it makes the backend reject the whole request (for example litellm.BadRequestError: ... is not a multimodal model). Instead the image part is dropped and replaced with a note stating that the image is not visible to the model and that the attachment is listed in the turn with its file name and path. The turn's own attachment summary (Anhänge: - Bild: <name> (<size>; Originalpfad-Metadaten: <path>)) is what makes this actionable, so the model can still store the file or hand its path to an image-to-video job rather than failing outright.

The recorded flag is authoritative — gateway aliases such as spark-cluster/deepseek-v4-flash carry no reliable capability signal in the id — and the id heuristic applies only to models with no recorded flag. The selected model is never swapped out because of an attachment; only the form the attachment takes changes.

When the fallback budget runs out, the surfaced error keeps the provider's own message and appends the budget note, so the actionable cause stays visible instead of being replaced by the timer. The same per-model metadata area now also supports a four-stage Kontextmodus:

• Voll: normaler Webchat-Kontext; der native Runtime-Pfad startet trotzdem mit einem kompakten Tool-Familien-Katalog, direkt gerouteten Funktionsschemas und On-Demand-Erweiterung, damit der volle konkrete Tool-Katalog nicht mehr in jeden ersten Request geht
• Voll - Compact Tools: expliziter Alias fuer dasselbe kompakte Tool-Catalog-Verhalten mit vollem Chat-/UI-/RAG-Kontext
• Low: weniger injizierter User/UI-Kontext, kleinere Dokument- und RAG-Blöcke, lightContext im nativen Runtime-Pfad und kein doppelt serialisierter Verlauf im appseitigen Extra-Systemprompt
• Ultra Low: wie Low, aber im nativen Runtime-Pfad zusätzlich minimalContext, ohne Bootstrap-, Retrieval-, Compaction- und Stored-History-Replay und mit stark reduziertem Tool-Katalog

The compact catalog path is now the default for normal full-context native runs. It exposes core memory/context tools, directly routed concrete schemas for the active request/page, and tool_catalog_search / tool_catalog_expand / tool_execute for on-demand discovery. Bridge-backed Claude/Codex runtimes expose the same idea through lean MCP tools instead of registering every concrete Clapilot tool as a separate MCP function. This is mainly intended for slower local or self-hosted models where prompt prefill dominates latency; the tradeoff is that long multi-step conversations may retain less context as you move from Voll to Ultra Low. The same settings UI now also shows two provider-state signals per card:

• Konfiguriert: whether the provider has the required stored setup (for example API key, Codex OAuth, or compatible base URL)
• Verbunden / Auth ok: whether the runtime could validate the saved connection live against the upstream models endpoint, or for OpenAI Codex setups whether a usable OAuth credential is present even when a stale API key would otherwise fail

Provider cards are collapsed by default so the overview stays scannable even with multiple configured backends.

Execution behavior

The native runtime chooses the execution adapter by provider type:

• anthropic -> Anthropic Messages API path for direct API-key rows, Claude CLI bridge for setup-token subscription rows
• aws_bedrock -> Bedrock Converse API path using the stored Bedrock bearer token
• azure_openai -> Azure OpenAI v1 OpenAI-compatible paths under /openai/v1; preview/date-style apiVersion values are added only when configured
• openai / openai_compatible -> OpenAI-like path
• xai -> xAI Responses path under /v1/responses, using either refreshed xAI OAuth or a direct xAI API key
• ollama -> native Ollama /api/chat with NDJSON streaming, tool calling, vision images, and structured output; embeddings use /api/embed
• openai-codex -> Codex-brokered path for subscription-backed Codex chat selection
• OpenAI-compatible provider rows can be selected as the global image-generation provider when their model list contains the intended image model. /api/generated-images/generate calls the row's base_url at /images/generations; /api/generated-images/edit calls /images/edits. If the row has an API key Clapilot sends it as a bearer token, and local compatible endpoints can also run without one.
• OpenAI-Codex can also be selected as the global image-generation provider. In that media path Clapilot keeps the normal provider row (provider_type = openai, authMode = codex_oauth) but calls chatgpt.com/backend-api/codex/responses with the hosted image_generation tool, defaulting to gpt-image-2 and the stored Codex OAuth credential instead of OPENAI_API_KEY.
• xAI Grok can also be selected as the global image-generation provider. /api/generated-images/generate calls xAI /images/generations with grok-imagine-image-quality or grok-imagine-image; /api/generated-images/edit calls /images/edits with JSON data-URI source images. The same OAuth/API credential is reused, refreshed, and persisted through the xai provider row.
• AI media video generation is separate from image-generation routing. Settings -> ClapilotAICore -> AI media -> Video generation selects an enabled media_generation_provider_configs video row such as xai-grok-video. Enabled OpenAI-compatible runtime providers with a configured base URL appear there automatically. Their model field uses the native provider's live /models discovery catalog rather than only the models already added to the chat-provider row, while remaining editable for exact video aliases that a gateway does not advertise. xAI video rows use the authenticated /video-generation-models catalog so newly enabled model IDs appear without a code or migration update. Discovery failures retain the saved list and current selection instead of clearing the default. Compatible providers use multipart POST /videos, GET /videos/{id} status polling, and GET /videos/{id}/content download. Stored bearer credentials are reused when present and authless local gateways omit authorization. Normal chat video requests use videos_generate/videos_status, persist jobs in generated_videos, and stream ready files from /api/generated-videos/[id]; Live Stream Studio continues to use livestream_generate_video for stream assets. videos_generate.image_id and source_image_path enable image-to-video for xAI and OpenAI-compatible providers. Grok sends the owned/imported source as image.url; compatible providers send it as input_reference. grok-imagine-video-1.5 is used with an image; text-only requests fall back to the configured grok-imagine-video entry instead of submitting an unsupported 1.5 text-only request. xAI durations retain the requested whole-second value within the supported 1–15 second range.

Non-specialized agent-core split:

• non_specialized_agent_core=native selects Clapilot-code (Assistant) for all providers without a specialized bridge
• non_specialized_agent_core=embedded_pi selects Clapilot-code (Coding), the persistent coding-agent profile with a larger tool-loop budget, on top of the existing provider transport
• agent_provider_configs.metadata.agent_core can override that global choice per provider with the same native or embedded_pi wire values; absent, null, and inherit inherit the global setting
• openai-codex and Claude subscription / CLI paths remain specialized runtimes and are not part of this switch
• the selected provider row still owns the actual model request transport; Clapilot-code (Coding) (embedded_pi) changes the session/core profile, prompt contract, and tool-loop budget rather than swapping the provider to Codex

The same provider layer also resolves:

• tool schema translation
• streaming vs non-streaming response normalization
• usage aggregation
• retry/fallback attempts

Request audit logs

Clapilot now persists a dedicated per-request audit ledger in agent_model_request_logs.

This is intentionally separate from agent_runs:

• one native run can fan out into multiple upstream model requests because of tool loops, retries, or embeddings
• bridge-backed execution paths such as Claude CLI and Codex bridge can be logged alongside direct HTTP providers in one table
• the admin ClapilotAICore -> Logs page can therefore show the real provider traffic instead of only session-level outcomes

Each row records:

• provider label / type / slug and auth mode
• requested and effective model
• request kind (conversation or embedding)
• transport (http, cli_bridge, orchestrator_bridge)
• endpoint family such as openai.responses, anthropic.messages, aws_bedrock.converse, openai.embeddings, anthropic.claude_cli_bridge, or openai.codex_bridge
• derived input / output / total token counts plus raw usage_json
• estimated prompt-layer token attribution in prompt_layer_tokens_json, including runtime system prompt, compaction summary, bootstrap prompt files, retrieved memory, approved learning objects, extra system prompt, stored history, current request, tool schemas, message prompt estimate, and provider input estimate
• request duration in milliseconds
• success vs failure status and the captured error message when the request failed
• optional links back to session_key and agent_runs.id

The metadata JSON also carries robustness breadcrumbs without requiring schema-specific columns:

• assistantSanitization appears only when output normalization changed a completed response and records reasoningStripped, salvagedCount, argsRepaired, and surrogatesStripped
• failed native provider requests record providerErrorReason, providerErrorAction, and providerErrorRetryable from the shared typed error classifier
• a compressed-context retry records contextCompressionRetry with the provider type and original, compressed, and dropped message counts on the retried request
• run-level provider-attempt summaries carry the same classifier reason/action so timeout replay, rate-limit cooldown, quota reset/fallback, configured fallback, and abort decisions can be audited together; quota attempts use reason: quota_exhausted, status: quota_limited, and include quotaResetAt/retryAfterMs when known

Provider metadata also accepts tool_schema_compat. Its default "off" value leaves OpenAI, Azure OpenAI, Anthropic, Bedrock, xAI, and generic OpenAI-compatible schemas unchanged. Set it to "strict" for local or llama.cpp-style backends whose JSON-schema-to-grammar converter rejects keywords such as pattern, format, $ref, or combinators. Gemini applies its narrower schema coercion unconditionally because the Gemini function-calling API accepts only a restricted schema subset.

The current instrumentation covers:

• OpenAI Responses requests
• xAI Responses requests
• OpenAI Chat Completions requests
• Anthropic Messages requests
• AWS Bedrock Converse requests
• provider embedding requests
• Claude subscription CLI bridge runs
• Codex bridge runs

Embeddings

Memory embeddings and the document RAG index are resolved through the same provider layer.

Current behavior:

• admins choose the embedding provider and model explicitly in ClapilotAICore -> Provider & Modelle
• supported embedding backends are:
  • openai via direct API-key-backed OpenAI-API rows
  • azure_openai
  • openai_compatible
  • aws_bedrock
• uses modelDefaults.embedding_model when the provider row defines one and no explicit model-routing override is stored
• otherwise defaults to text-embedding-3-small

Clapilot stores the selected provider slug and model in native model routing, and both the native memory store and the web-side RAG retrieval/indexer use that same selection.

If no embedding-capable provider is configured, memory falls back to deterministic local hash embeddings instead of blocking the runtime. The document RAG retrieval/indexer will skip provider embeddings until a provider/model is configured.

Realtime

Live/call Realtime routing is stored alongside embeddings, but with its own provider/model pair.

Current behavior:

• admins choose the Realtime provider and model explicitly in ClapilotAICore -> Live Voice
• supported Realtime provider rows are currently:
  • openai via direct API-key-backed OpenAI-API rows
  • azure_openai
  • openai_compatible
• google_gemini
• the selection is persisted in native model routing under its own realtime_provider_slug / realtime_model keys
• Realtime-capable provider rows are available in the standard cross-provider chat priority when they also support the regular text-chat runtime path; Gemini Live/media model ids remain filtered from chat routing

Speech-to-text

Server-side STT routing supports direct OpenAI API-key rows, Gemini rows, and OpenAI-compatible rows. Compatible providers use the configured base URL plus /audio/transcriptions; administrators can enter an exact gateway model alias, and local endpoints may run without a bearer key. Image, TTS, and STT agent tools accept an exact runtime provider_slug plus optional model; omitted routing uses the capability default, while an explicit slug resolves only that enabled provider and never silently falls back. Video generation already follows the same explicit-slug contract through media_generation_provider_configs.

Split provider rows

OpenAI and Anthropic are now intentionally split into separate provider rows by auth mode:

• OpenAI-API uses a direct API key
• OpenAI-Codex uses Codex OAuth JSON / stored OAuth credentials
• Anthropic-API uses a direct API key
• Anthropic-Claude uses a Claude setup-token flow; the stored setup-token is persisted as the provider secret and then injected into the local Claude CLI bridge for runtime execution

The old app-level OpenAI / Anthropic key fields remain only as legacy mirrors for compatibility and migration. The source of truth is now ClapilotAICore -> Provider & Modelle.

Environment fallback

If the database has no usable enabled providers, clapilot-agent falls back to env configuration from services/clapilot-agent/src/config.mjs.

Relevant env vars:

• CLAPILOT_AGENT_OPENAI_ENABLED
• CLAPILOT_AGENT_OPENAI_MODELS
• CLAPILOT_AGENT_ANTHROPIC_ENABLED
• CLAPILOT_AGENT_ANTHROPIC_MODELS
• CLAPILOT_AGENT_BEDROCK_ENABLED
• CLAPILOT_AGENT_BEDROCK_REGION
• CLAPILOT_AGENT_BEDROCK_MODELS
• CLAPILOT_AGENT_BEDROCK_ACCESS_KEY_ID
• CLAPILOT_AGENT_BEDROCK_SECRET_ACCESS_KEY
• CLAPILOT_AGENT_BEDROCK_SESSION_TOKEN
• CLAPILOT_AGENT_BEDROCK_BASE_URL
• CLAPILOT_AGENT_AZURE_OPENAI_ENABLED
• CLAPILOT_AGENT_AZURE_OPENAI_BASE_URL
• CLAPILOT_AGENT_AZURE_OPENAI_MODELS
• CLAPILOT_AGENT_AZURE_OPENAI_API_KEY
• CLAPILOT_AGENT_AZURE_OPENAI_API_VERSION
• CLAPILOT_AGENT_COMPAT_BASE_URL
• CLAPILOT_AGENT_COMPAT_MODELS
• CLAPILOT_AGENT_GEMINI_ENABLED
• CLAPILOT_AGENT_GEMINI_MODELS
• CLAPILOT_AGENT_GEMINI_BASE_URL
• CLAPILOT_AGENT_GEMINI_API_KEY (falls back to GEMINI_API_KEY / GOOGLE_AI_API_KEY)

This fallback is mainly for bootstrap and recovery. The intended source of truth in normal operation is Postgres.

Operational recommendations

• Keep provider model lists explicit instead of relying on stale defaults.
• Use the global model priority list for cross-provider failover.
• Set context windows and output caps for OpenAI-compatible backends that do not expose limits automatically.
• Treat Codex OAuth as the preferred OpenAI chat path when available, but remember that embedded runtime runs own their own pre-compaction memory flush and session compaction, so the native Clapilot maintenance path is bypassed for those runs.

Related docs:

• Memory
• Sessions
• Configuration