Clapilot Hub

Clapilot Hub is the control plane for a fleet of Clapilot instances. This page is for admins who operate several tenants and want central issue intake, health checks, telemetry dashboards, and a shared module/skill/widget/agent catalog.

Fleet instances keep the full agent tool surface by default. For an intentionally restricted tester or tenant, an administrator can turn off Shell tools on that instance's detail page. The Hub stores this as a per-instance policy and redeploys only the selected instance; other instances remain unrestricted. Turning shell tools off requires the instance to be running so the Hub can verify and drain remote-runner work; start a stopped instance, apply the restriction, and then stop it again. Re-enabling shell tools on an already-restricted stopped tenant does not start its services when Fleet connector 1.1.0 or newer is installed; the Hub rejects older connectors before queueing that stopped-instance redeploy and tells the operator to re-run the served installer.

Standalone Fleet connectors version 1.2.0 and newer compare their version with the Hub after each successful heartbeat. When the Hub serves a newer connector, an idle standalone connector downloads and validates the connector plus its event-outbox sibling, replaces both files, and exits so launchd or systemd restarts the new version. Set CLAPILOT_FLEET_SELF_UPDATE=0 to opt out. The connector embedded in Clapilot Remote Runner.app does not self-update; runner-app connector updates arrive with the signed app through Sparkle.

Each Fleet machine has an opt-in Remote agent (Codex) toggle. A standalone connector 1.2.0 or newer starts the bundled Node remote-runner process when the toggle is enabled and the machine has an installed, authenticated Codex CLI. The connector reports the detected Codex version, keeps the runner current from the Hub, and restarts it with bounded backoff. The runner-app variant leaves this responsibility to the Mac app's native runner. The toggle may be enabled before Codex is installed; the connector activates it after Codex becomes available. Set CLAPILOT_FLEET_REMOTE_AGENT=0 to suppress the standalone child locally.

Hub admins can also manage the deployment fleet through chat or the generated command-line family. fleet_list / clapilot-cli fleet list returns a redacted instance and machine overview; available capacity counts only online machines with a deployable architecture, while configured capacity is reported separately. fleet_create and fleet_destroy are two-step operations bound to the persisted personal admin session: the first call performs no mutation and returns a signed token plus an exact phrase, the human admin sends that phrase as a new chat message, and only an unchanged retry with approval_token may provision or destroy. New Fleet instances enable main-agent tool restrictions by default; Hub admins can switch that bootstrap setting off in the create form or through fleet_create.main_agent_tool_restrictions_enabled. The value is adopted once on the new instance so later changes in its own settings survive restarts and redeploys. Destroy also requires the resolved instance name as confirm_name. Caller-supplied user ids never grant Fleet access, database lookup outages are not presented as absence, and create preserves the Hub public origin derived from forwarded request headers. Tool output never includes environment or override values, credentials, or machine secrets.

Hub mode is not a separate product: it is a mode of a normal Clapilot instance. When hub mode is enabled, that instance accepts signed issue reports from other Clapilot instances and runs health checks against linked tenants.

Configure the destination instance in Settings -> Admin -> Admin Hub:

• set Hub Modus to Lokal auf dieser Instanz
• set a shared secret

On another instance, configure Settings -> Admin -> Admin Hub as:

• Remote Hub verwenden
• the remote Clapilot base URL
• the same shared secret

Then open Settings -> Issue Reporter and select the available hub target instead of GitHub.

The hub-mode instance exposes:

The page currently combines:

• hub-mode configuration for this instance (disabled, local, remote)
• a signed connection test for the configured local or remote hub target
• a compact fleet overview for saved and auto-discovered instances, including the monitoring toggle and per-instance tracking summaries (events, active users, and tokens over the last 7 days, plus the latest event); classification, detailed health context, and deletion live in the instance detail view to keep overview rows compact
• automatic Fleet placement ranks online, image-compatible machines by relative instance-slot utilization, normalized CPU load per core, and relative free memory; an explicitly selected offline machine is rejected before an instance or provision job is created
• per-instance tracking dashboards under /admin/hub/instances/[id] with classification, detailed health context, instance deletion, monitoring thresholds, activity charts, usage/error rankings, and a user filter (see the Product Telemetry doc)
• a compact Subscription Usage page under Settings -> Hub that shows the Hub's local Codex, Claude Code, Grok, and Ollama accounts plus the latest report from every connected instance
• direct health checks against remote Clapilot deployments
• signed inbound issue intake from linked Clapilot instances
• review actions for inbound issue reports in Settings -> Issue Reporter: approve forwards the report to the configured Issue Reporter target, either GitHub or the Agent Orchestrator task board, while deny closes the report without forwarding
• inbound issue report detail now shows the optional affected-platform classification selected by the reporter when present
• optional admin-login verification when credentials are stored
• add/delete/refresh actions for monitored instances
• product-owned customer monitoring runs every five minutes on the hub. It checks every active monitored host and persists the HTTP result, check time, summary, and structured payload. HTTP 2xx/3xx and authentication gates (401/403) count as reachable; 5xx responses (including Cloudflare 52x/530) and timeouts count as down.
• each host has a persisted classification (kunde, eigen, dev, intern) and monitoring settings. localhost:3000 and clapilot:3000 are classified as development and excluded from customer monitoring.
• telemetry is never, fresh, or stale. The default stale threshold is 26 hours. A host that never sent telemetry remains explicitly visible but does not open an incident; a host that previously sent and then becomes stale is degraded.
• the default 24-hour error threshold is greater than 15 percent with at least 20 operations. Both values and the stale threshold can be changed in the corresponding instance detail view.
• two consecutive observations confirm incidents and recoveries. Each confirmed transition creates one Team Chat alert, preventing flapping from producing repeated notifications while retaining a matching recovery.
• the worker re-checks local hub mode before every cycle and remains dormant on disabled or remote/spoke installations. Monitoring configuration is intentionally restricted to authenticated admins and is not exposed as an agent tool; this prevents chat-driven changes to customer alerting policy.
• auto-discovered instances remain visible but start with monitoring disabled. An authenticated hub admin must explicitly enable the current target before unattended probes begin, and later telemetry cannot replace an approved URL.
• a hub admin issues or resets a random per-instance telemetry credential only after inspecting the discovered host and sender identity. The hub stores only its SHA-256 hash; the admin copies the one-time plaintext value into that customer instance, where it is stored with the local secret-encryption key. Product and model telemetry must present the credential and match the immutable admin-approved sender identity. Issuing or replacing a credential disables monitoring until re-approved, and reissuing is the recovery path after a spoke loses its local token. The fleet-wide shared secret cannot create or replace this per-instance binding.
• periodic monitoring stores its result separately from manually triggered login, auth-API, credential, and session checks. A shallow reachability probe therefore cannot overwrite the latest deep health snapshot.
• target and policy edits invalidate in-flight probe results. Changing a host also clears its previous incident, confirmation, and periodic snapshot state so the replacement starts with a clean baseline.
• telemetry freshness and the 24-hour error window use the hub-controlled receipt timestamp rather than the sender clock. HTTP probes use bounded concurrency so slow hosts do not serialize the five-minute fleet cycle.
• remote instances report only a normalized, credential-free Subscription Usage snapshot. A persisted sender lease limits successful reports to once every 15 minutes and failed attempts to once every five minutes. The Hub stores only the latest snapshot per instance and marks it stale after 45 minutes. Opening or refreshing the Hub page reads these rows and never fans out to customer instances or subscription providers.
• GET /api/v1/subscription-usage continues to return only the API-serving Hub's own accounts by default. include_connected=1 explicitly opts into the per-instance aggregate on a local Hub.
• alert copy, sender labels, stored health summaries, and validation errors use the installation admin's selected German, English, or Italian UI language.
• a detail view for inbound issue reports including transcript and attachments
• the local module, skill, widget, and specialized-agent registries used by Store tabs for Modules, Skills, Widgets, Special Agents, and Settings -> Agent -> Spezialisierte Agenten

Managed instance contract

Each monitored row stores:

• host
  • expected shape: tenant.clapilot.com
  • host only, no path
• source_instance_id
• instance_url
• discovery_source
  • manual
  • validate
  • issue_report
  • telemetry
• last_seen_at
• optional admin_email
• optional admin_password
  • stored encrypted at rest (AES-256-GCM keyed from AUTH_SECRET, format enc:v1:<iv>:<tag>:<ciphertext>)
  • rows created before encryption existed are re-encrypted automatically the next time they are loaded
• cached health snapshot from the most recent server-side check

Instances that call the signed hub API are now auto-discovered:

1. POST /api/hub/validate
2. POST /api/hub/issues/report
3. POST /api/hub/telemetry/ingest
4. POST /api/hub/subscription-usage/ingest

When the sender includes a valid instance_url, the hub upserts that instance into the monitored list automatically and refreshes its last_seen_at timestamp. Manual rows remain editable and keep their stored admin credentials.

Stored admin credentials are used server-side for the optional authenticated health check; the password is decrypted only at that point. The cached health snapshot records only a credentials_configured boolean; the hub UI does not display stored passwords and only receives booleans such as has_admin_password and has_admin_credentials. All monitored-instance endpoints are admin-only.

Health check behavior

The current health worker performs server-to-server checks against the stored instance_url when present, otherwise against https://{host}:

1. GET /login
   • verifies the login surface responds
2. GET /api/auth/me
   • expects a reachable auth API (typically 401 when unauthenticated)
3. if credentials exist:
   • POST /api/auth/login
   • extract session cookie
   • GET /api/auth/me with that cookie

Status meanings:

• healthy
  • login page and auth API respond
  • if credentials exist, authenticated session verification also succeeds
• degraded
  • host responds partially, but one or more checks fail
• down
  • login page and auth API are both unreachable
• unknown
  • no health snapshot has been stored yet

Sender-side settings

On the reporting instance:

• configure Admin Hub as local or remote
• set the shared secret there
• if remote, also configure the remote base URL
• in Issue Reporter, choose GitHub, Task Board, Laufender Hub, or Remote Hub depending on the desired destination and what Admin Hub exposes

Chat transcript sync and training export

Connected instances can sync their chat transcripts to the hub so hub admins can export all collected chat sessions as JSONL for model training/fine-tuning.

Sync behavior (spoke side):

• opt-in via the Send chat transcripts to hub toggle in the Admin Hub panel (hub_chat_transcript_sync_enabled, default off); it only applies in hub_mode=remote
• covers three sources: personal chats (chat_nachrichten + chat_sessions), group chats (chat_group_messages), and completed agent runs (agent_runs joined with agent_session_state; heartbeat runs are excluded)
• runs on the same background scheduler as telemetry sync, with per-source cursors in chat_transcript_hub_sync_state, batched and HMAC-signed like telemetry ingest
• transcript fidelity: attachments are reduced to { name, mimeType } metadata, inline data: base64 URIs in message text are replaced with [inline-data removed], and message content is capped at 100k characters. Tool calls, their arguments, and persisted result previews are retained for training. Other PII is intentionally not scrubbed or pseudonymized; this is an admin-only export of the source workspace data.

Hub behavior:

• POST /api/hub/chat-transcripts/ingest (hub instances only, HMAC-signed) stores records idempotently in hub_instance_chat_transcripts keyed by (instance_host, source, source_record_id)
• GET /api/hub/chat-transcripts/stats (admin only) returns per-instance/per-source session and record counts, including the hub's own local data
• GET /api/hub/chat-transcripts/export (admin only) streams JSONL aggregating hub-synced spoke data plus the hub's own local chats; filters: instances (csv of hosts, or local), sources (csv), from/to (ISO dates), format. dryRun=1 returns counts from the same training pipeline instead of a download.
• export formats: native is the full metadata archive and never drops records; its messages now include tool calls/results and the row carries toolTrace. messages is the UI default and emits system/user/assistant/tool training conversations with full JSON-Schema tool definitions as a top-level tools field when needed.
• messages samples are compatible with strict chat-template trainers such as Unsloth/TRL: tool arguments remain JSON objects, tool-result content is always a string, call/result IDs match, and every row ends with a non-empty assistant response. Windows split only at complete-turn boundaries; a single turn may exceed the normal 32-message limit.
• untraceable records are not training examples. Missing run linkage, incomplete event pairs, invalid names/arguments, and old hub rows without metadata.toolTrace create a boundary: that turn is dropped while traceable turns before and after it survive in separate samples. Ordinary no-tool conversations are retained when their run trace is known to be complete.
• tool result content comes from agent_events.event_data.resultPreview. It is a truncated preview (typically 600, 1,200, 8,000, or 20,000 characters depending on the provider path), not always the exact byte-for-byte result seen by the model. Full tool outputs are not persisted, so this is the highest available fidelity and the exporter does not reconstruct missing text.

Example native line:

{"instance":"spoke.example.com","source":"personal_chat","sessionKey":"personal:chat-1","sessionTitle":"Steuerfragen","channelType":"webchat","model":"claude-fable-5","user":{"id":"…","email":"…","name":"…"},"firstMessageAt":"2026-07-17T10:00:00.000Z","lastMessageAt":"2026-07-17T10:05:00.000Z","recordCount":3,"messages":[{"role":"user","content":"…"},{"role":"assistant","content":"…"}],"toolTrace":"complete"}

Example messages training sample:

{"tools":[{"type":"function","function":{"name":"calendar_list_events","parameters":{"type":"object","properties":{},"required":[]}}}],"messages":[{"role":"system","content":"Du bist Angela, der Clapilot-Copilot. …"},{"role":"user","content":"…"},{"role":"assistant","content":"","tool_calls":[{"id":"call_1","type":"function","function":{"name":"calendar_list_events","arguments":{}}}]},{"role":"tool","tool_call_id":"call_1","name":"calendar_list_events","content":"[]"},{"role":"assistant","content":"…"}]}

The training export is an admin-only bulk data workflow and intentionally not exposed as a chat/live agent tool.

Channel bridges (hub ↔ instance)

Channel bridges mirror a Team-Chat room bidirectionally between two Clapilot instances, typically the hub and a connected or fleet instance. A bridge behaves exactly like the existing WhatsApp/Telegram ↔ Team-Chat channel mapping: instance_bridge is a normal channel type backed by one channel-approval row per side, and it reuses the same approval, mirror, and forward machinery. The bridge API routes exist on every Clapilot instance (both sides can serve them), not only on hub-mode installations.

Pairing flow (hub-initiated in v1):

• an admin on the initiating instance creates the bridge from the ClapilotAICore Channels settings: pick a connected instance (monitored instances with a usable instance_url, plus running fleet instances), load the peer's mappable rooms through the signed POST /api/hub/channel-bridge/rooms contract, and select a local room
• the initiating instance generates a shared bridge_id and calls the peer's POST /api/hub/channel-bridge/register (HMAC-signed with the fleet shared secret); only after the peer registration succeeds is the local bridge row created. The initiating side stores role initiator, the peer stores role peer
• each side stores the bridge as one approved instance_bridge channel-approval row mapped to its local room; the normal deny/revoke flow still shuts a bridge off locally
• removal deletes the local row and sends a best-effort signed remove to the peer; a peer failure is logged and the local delete still succeeds

Message flow:

• a message posted in a bridged room is forwarded through the standard channel-forward path as a signed POST to the peer's /api/agent-runtime/channels/instance_bridge/inbound endpoint
• the peer verifies the HMAC signature on the raw body, rejects unknown bridges (no auto-created pending approvals, unlike WhatsApp), dedupes by event id, and mirrors the message into its mapped room stamped with a channelOrigin marker; participant messages keep the original sender name, agent messages render as agent messages with the remote agent's display name
• bridge forwards carry raw text plus structured sender fields instead of the [Name via Clapilot] / [AgentName] text framing used by external providers

Agent behavior:

• mirrored participant messages may trigger the local main agent, subject to the room's normal reply-mode settings
• mirrored remote agent messages never trigger a local agent run

Loop guards:

• messages carrying channelOrigin (anything mirrored in from a channel) are never forwarded out again
• inbound events are deduplicated by event id
• combined with the never-trigger rule for remote agent messages, agent↔agent loops are impossible for point-to-point bridges

Security note: bridge authentication uses the fleet-wide hub shared secret (hub_shared_secret / CLAPILOT_HUB_SHARED_SECRET). Any instance holding the secret can address any other instance's bridge endpoints; per-bridge secrets are a possible future hardening.

v1 limitations:

• text only; attachments degrade to 📎 <name> placeholders on the far side
• point-to-point only, no 3+-instance relays
• hub-initiated pairing only, no spoke-initiated pairing UI
• no edit/delete propagation

Module, skill, widget, and agent catalog behavior

/modules, /skills, /modules?tab=mini-apps, and specialized-agent settings now resolve their hub target from Admin Hub:

• hub_mode=local
  • the instance serves its own catalog directly from /api/v1/modules, /api/v1/skills, /api/v1/widgets, and /api/v1/agents
• hub_mode=remote
  • the instance proxies catalog, publish, and download operations to the configured remote hub URL
• if the remote hub is unreachable
  • the UI receives a short summarized hub error instead of the raw HTML response body

Specialized agents are published as a portable JSON snapshot. The snapshot includes only the reusable definition (handle, name, description, prompt, default_model_ref, skill_keys, allowed_tool_names, allowed_auth_resource_keys, and include_core_memory_tools). Runtime permission bypass, public embed deployments, API keys, and secrets are intentionally not exported and are never enabled automatically when a hub agent is installed.