Clapilot Docs
Docs

Specialized Agents: Editor & Permission Modes

The specialized-agent editor (Einstellungen → Spezial-Agenten → "Neuen Agenten erstellen" / edit dialog) configures a specialist's identity, prompt, model, permissions, skills, and external channels.

Team Chat settings shortcut

Direct replies to an agent's Team Chat message address that agent even when its channel reply mode is Mention only. This applies both to answers following an @handle mention and to messages the agent posts independently. Web, iOS, and macOS use the same server routing. The recipient must still be enabled and invited to the channel. Additional explicit mentions address those agents too, without triggering the same agent twice. Replies to human messages do not implicitly address an agent.

Administrators can open a specialist's settings directly from the circular settings button beside its Team Chat agent row. The web app opens the existing full editor for that agent without changing the active conversation and refreshes the agent directory after saving. iOS and macOS open the existing native specialist detail and execution settings (full screen on iPhone); the compact channel switcher also includes a named settings action per specialist. Existing administrator permissions still apply.

This shortcut reuses the existing specialist settings APIs and agent tools; it introduces no new mutation or tool contract. Chat/live agents continue to use the specialized-agent tools documented in the agent tool inventory.

Permission modes: Einfach vs. Experte

The permissions area of the editor has two views, switched with a segmented control. Einfach (Simple) is the default every time the dialog opens.

Einfach (default)

Tool permissions are bundled into user-facing areas backed by the tool-catalog categories. Each area is one on/off toggle:

  • Essentials — Aufgaben, Kalender, Web Search, Browser, Canvas, Channels, Kontakte, Team Chat, Wiki, Navigation, Profile, Issue Reporter, CopilotKit UI, Task Planning, Clapilot Context, and media generation (Image Generation, Video Generation, Audio & Speech). Video Studio and Model Training are separate tool-catalog categories and therefore appear as modules.
  • Dokumente — Dokumente, Notizen, Whiteboard, Word Editor, Excel Editor.
  • Kommunikation — E-Mail plus Call & Fax.
  • Module — every remaining tool-catalog category (Social Media, Buchhaltung, Website Canvas, Automationen, …) gets its own toggle, so newly added categories appear automatically.

Behavior in the simple view:

  • Toggling an area on selects all specialist-selectable tools of the underlying categories and automatically grants the matching Auth & API access resources; the Auth section itself is hidden in this view (a hint explains that auth follows the enabled areas).
  • Areas are disjoint: every selectable tool belongs to exactly one area (Aufgaben and Kalender live in Essentials, not in Dokumente), so toggling one area never changes another. If a group is only partially selected (for example after fine-tuning in expert mode), the row shows an x/y Tools aktiv badge; the switch turns the whole group on or off.
  • Core-Memory tools are hidden in the simple view. New agents are created with Core-Memory enabled by default; the toggle to disable it (for public or strictly scoped agents) lives in the expert view.
  • The personal agent memory toggle stays visible in both views.

Experte

The expert view is the full-detail editor: the complete tool catalog with search and per-tool checkboxes grouped by category, the Core-Memory switch with its tool list, the skill auto-dependency summary, and the explicit Auth & API-Zugriffe selection. The list of currently selected tools is collapsed by default and can be expanded via "Ausgewählte Tools anzeigen".

Defaults for new agents

Creating a new (internal) agent starts with:

  • all Essentials tools pre-selected, including the auth resources they require,
  • Core-Memory tools enabled (include_core_memory_tools = true),
  • personal agent memory enabled.

Public embed exposure remains off by default; the public-safe embed tool allowlist stays fully independent of these internal defaults.

Skills and external channels

  • Skill rows show only the skill name and a short description. Declared tool/auth dependencies are still applied automatically when a skill is selected; the dependency summary is visible in the expert view.
  • The Externe Kanäle section (web embed, Telegram, WhatsApp) is collapsed by default and expands via its header.

Client coverage

The full identity and permission editor is in the web app. iOS and macOS also let administrators set execution location, pinned runner, Codex model, and desktop/browser/search capabilities in the specialist detail. All three clients expose pending remote approvals and input requests to the initiating user.

Instance-hosted Codex tool approval

For instance-hosted Codex specialists, enabled Clapilot tools are authorized by the native runtime's tool dispatcher. The managed MCP bridge explicitly delegates tool approval to that dispatcher, so a specialist can use its permitted Video Studio, context, and planning tools while retaining the read-only filesystem sandbox. The “runtime permission bypass” setting is not required for these granted application tools; module permissions and action-specific approval checks continue to apply.

Remote Codex execution

Administrators can select Remote runner as the execution location, choose a connected Fleet machine, and enable computer use, browser control, and web search independently. Computer use has two desktop backends: the runner's native Codex Computer Use plugin, or an isolated Cua sandbox (see below). Select a Codex subscription model. Enable the required Clapilot business tools in the existing permissions editor; include documents_import_file to bring files back to Documents.

The instance still owns the specialist's instructions, personal conversation, memory, permissions, run history, and tool execution. The selected machine starts an isolated Codex app-server session. Its MCP bridge exposes only the tools permitted for that run. Native shell, computer-use, browser, and search tools execute through the runner's Codex installation. Model changes do not move the session to a different provider silently.

Follow-up messages resume the saved Codex thread for the same conversation and scope. Changing the runner or permitted tools starts a new thread. The workspace persists on the runner, while imported documents are shared workspace records. Files are transferred as bytes, never treated as paths on the instance. Imports support non-empty files up to 8 MiB from the specialist workspace, including browser downloads copied there, and shared destination folders.

Command/file/permission approvals and user questions appear in Clapilot and can only be answered by the initiating user. A file request includes the available change details; permission grants last for the turn. Arbitrary MCP elicitation forms and URL flows can currently only be declined through Clapilot. Native approval support does not make every browser click require confirmation: outbound-action review remains part of the specialist instructions and the existing Clapilot tool policy.

Runs remain pinned to their configured machine. Missing capabilities or a busy/offline machine fail after two minutes. Losing the runner lease, cancelling the chat, or disabling shell tools stops the remote attempt. A runtime restart fails the active attempt; UI actions are never automatically replayed. Review the last action before retrying. Public embeds and OpenAI-compatible specialist endpoints cannot use this execution mode.

Machine setup

Use remote runner 0.3.0 or later and Fleet connector 1.4.6 or later. Both standalone Fleet connectors and Fleet-enabled Mac runner apps start the specialist companion; the Mac app continues to own its existing coding jobs. Fleet's remote-agent switch must be enabled.

Enable a Codex subscription model in Clapilot, sign in to Codex on the runner, and install/enable its native Computer Use plugin. Desktop and browser actions use the accounts signed in on that machine; Clapilot business tools retain the initiating user's permissions. The runner discovers the installed unified-computer-use/.mcp.json and prefers the matching desktop-bundled Codex executable, including its code-mode helper. The ordinary standalone CLI remains suitable for sessions without desktop capabilities when its required helpers are installed. The process uses a separate Codex home and workspace for each conversation; instance and Fleet signing secrets are not inherited by its shell.

Optional operator overrides are CLAPILOT_REMOTE_RUNNER_CUA_MCP_CONFIG (absolute native plugin configuration path), CLAPILOT_REMOTE_RUNNER_CODEX_COMMAND (complete compatible Codex executable), and CLAPILOT_REMOTE_RUNNER_CUA_COMMAND (Cua CLI for the Cua sandbox backend; runner 0.3.3 or later). CLAPILOT_REMOTE_RUNNER_SPECIALIST_ONLY=1 disables ordinary coding-job polling for a companion managed alongside a native runner.

An installed plugin advertises available tool surfaces, not a guaranteed live browser connection. Chrome requires its supported native connection on the runner; a Chrome process alone is insufficient. OS desktop permissions and browser availability must be configured on that machine. Missing connections surface as native tool errors. There is no fallback to the instance's desktop.

Apply migration 301_remote_specialist_sessions.sql before starting the updated app and runtime. It adds execution configuration and durable session-to-runner/thread bindings; active claims and pending native prompts remain runtime-local.

For Team Chat remote specialists, business tools retain the room service-principal identity. Native approvals belong to the authenticated human who initiated the current turn (actorUserId); they do not inherit the previous sender. A different initiating user starts a new remote thread scope. Runs with no human user or actor remain unsupported.

Cua sandbox computer use

Cua provides isolated desktop sandboxes (Linux containers/VMs, Windows, macOS via Lume, or Cua Cloud Fleets). Clapilot uses the Cua CLI's MCP server (cua serve-mcp) as a second desktop backend, so computer use no longer depends on a Codex subscription or on a physical Mac's signed-in accounts.

Execution settings gain two fields: computerProvider (native or cua) and cuaSandbox (the sandbox name shown by cua sb ls). The sandbox must already exist; Clapilot never creates, deletes, or resizes sandboxes and the model cannot switch sandboxes because the name is pinned by the run configuration. Only computer:* permissions are granted; sandbox-management and skills tools are never exposed. computer:shell runs inside the sandbox only and is withheld when CLAPILOT_SHELL_TOOLS_ENABLED=false.

Instance-side (any model). With execution location Clapilot instance, enable Computer use in a Cua sandbox and enter the sandbox name. The native runtime starts one cua serve-mcp process per run, adds the discovered computer_* tools (screenshot, click, type, key, hotkey, scroll, drag, clipboard, window, file, shell) to the specialist's tool set next to its permitted business tools, and closes the process when the run ends. Screenshots return as tool-result images, so choose a vision-capable model. The agent image ships the Cua CLI (INSTALL_CUA_CLI build argument, CLAPILOT_CUA_COMMAND override); Cua Cloud credentials are passed through FLEETS_TOKEN or CUA_CLIENT_ID/CUA_CLIENT_SECRET. Public embeds and OpenAI-compatible endpoints never receive sandbox tools.

Remote runner. With the Cua sandbox backend, the runner spawns cua serve-mcp as the cua_sandbox MCP server of the Codex session instead of loading the native plugin. The runner advertises the cua capability only when the CLI resolves (cua on PATH or CLAPILOT_REMOTE_RUNNER_CUA_COMMAND); a specialist configured for Cua stays queued until a runner with that capability polls, and fails after the usual two minutes otherwise. The browser toggle still refers to the runner's native browser plugin. Local sandboxes (cua sb launch ubuntu:24.04 --local --name <name>) run on the runner's Docker/QEMU; cloud sandboxes require cua auth login on the runner or Fleet credentials in its environment. Instance and Fleet signing secrets are never passed to the sandbox process.

The sandbox's screen is treated as untrusted content: the specialist instructions forbid entering user passwords and keep outbound actions under the existing draft, review, and approval rules. Files inside the sandbox are not instance or runner paths; use computer_file_read and the permitted import tools to move content.

Inline remote-agent approvals

Remote specialist approvals and questions appear as cards in the originating conversation on web Chat, Team Chat, the floating chat panel, and the iOS/macOS chat transcripts. They never open a global modal or sheet. Each card provides the native request text, collapsible technical details, and the supported Allow/Decline or answer controls. A successful decision remains visible in that open transcript; the runtime audit remains authoritative after navigation or reload.

GET /api/specialized-agents/remote-requests?sessionId=<chat-session-or-team-room-id> returns only pending requests owned by the authenticated initiating user and matching that conversation. Pending request objects include sessionKey; matching ignores the specialist handle but compares the complete session identity. Omitting sessionId preserves the owner-scoped listing for existing clients. POST request and approval semantics are unchanged. Unsupported native elicitation still requires interaction on the runner; moving the card does not grant macOS permissions or bypass approval.