Runtime Flows

Generated explanatory diagrams

Runtime flows explained visually

These raster diagrams are generated documentation assets and are paired with exact captions below so the page is readable even when a tiny in-image label is hard to inspect.

This page explains what happens after a user or external system asks Clapilot to do work. The common pattern is:

1. The app records the user action or imported event.
2. The relevant context is attached: user, route, document, Mandant, mailbox, channel, or automation target.
3. The native runtime receives a run, job, or channel event.
4. The runtime uses memory, tools, model routing, and audit logs to produce the visible result.

Use this page when you need to trace an end-to-end behavior rather than a single API handler.

Document upload

1. User uploads file and metadata.
2. API resolves target mandant folder.
3. File is written to /app/workspace/mandanten/<mandant>/dokumente/<typ>/.
4. dokumente row is inserted with normalized relative file_path.

Inbox automation

1. Incoming files are written to /app/workspace/mandanten/_inbox.
2. Placeholder rows are inserted in dokumente.
3. Trigger marker .process-now is written.
4. On native-runtime installs, the preinstalled Inbox Document Processing automation picks up the trigger and runs the classification/follow-up prompt through clapilot-agent.
5. After document AI extraction completes in the RAG indexer, the preinstalled Document Post-Extraction Review event automation receives the new_document event and lets the agent decide whether to create, update, or leave Mandant/customer data unchanged.

Event-triggered automations

1. /geplante-aufgaben can now persist automations with trigger_kind set to new_mail, new_document, new_calendar_entry, or webhook instead of a time schedule.
2. These rows stay in scheduled_tasks, but they do not create native cron jobs and therefore do not need fake next-run timestamps.
3. When a matching event is created or imported, Clapilot dispatches the automation by starting a native runtime session directly against the automation’s stable session key.
4. new_mail automations can optionally scope themselves to all, personal, or agent mailboxes; dispatch skips mismatched mailbox events.
5. New mail events are emitted after the mailbox flow has finished AI analysis, so the automation receives summary, match, and action context instead of raw message metadata.
6. New document events are emitted after document AI processing completes in the RAG indexer, not immediately at upload/import time.
7. Webhook automations store a unique token in scheduled_tasks.provider_snapshot.webhookToken and expose /api/automation-webhooks/{token}. Acceptance atomically reserves an agent_runs row and persists the redacted request envelope in scheduled_task_event_queue before returning its run_id. GitHub delivery IDs (or a canonical payload hash) deduplicate retries. The native webhook queue reclaims interrupted rows after restart, reuses the reserved run ID, and records termination status, error_code, timestamps, and a redacted cause. Final automation posts are deduplicated by run ID so recovery publishes a release update once.
8. The preinstalled Document Post-Extraction Review automation uses that event to review extracted document context and decide on Mandant/customer mutations via normal Clapilot tools instead of hardcoded indexer logic.
9. New calendar entry events are emitted by direct calendar creation, Google Calendar sync, and agent-created calendar actions.
10. Because event-triggered runs reuse stable native session keys, the run/event log shown in /geplante-aufgaben continues to work for both schedule-backed and event-backed automations.
11. The inbox flow no longer exposes a user-facing PDF mode toggle. Native uploads and the scheduler both use the same direct-document prompt contract.
12. The runtime is told to inspect and validate the original files from the shared workspace instead of depending on a pre-extracted fallback transcript.
13. The native runtime receives the structured prompt for classification and follow-up. /api/documents/inbox wakes clapilot-agent through /internal/runs.
14. Document-analysis specialists can be configured under Settings -> Agent -> Spezialisierte Agenten with the normal prompt, tool, auth, Core-Memory, and permission-bypass controls. The specialist editor supports both the normal form and a Capability Graph where admins connect the specialist to Skill, grouped Tool, Auth, and Core-Memory nodes that write the same persisted capability allowlists; grouped Tool nodes expose the exact read/write/get/create grants in the inspector. A new_document automation can then bind one of those specialists to a concrete document type or workflow prompt while keeping review/freigabe in the automation prompt where needed.

Chained automation agents and attached skills

1. When an automation's workflow_config_json contains multiple agent nodes, the job runner executes them sequentially as a linear chain (chain order follows agent -> agent edges).
2. Step 1 runs with the normal automation run message (trigger/event context plus the flat automation prompt) in the automation's session key. Each later step runs in a step-scoped session (<session_key>:step-<n>) and receives a [Automations-Kette Schritt n/m] message containing the previous agent's output plus that agent node's own config.prompt.
3. Each step resolves its own specialized agent (config.specialized_agent_id), model (specialist default_model_ref → node config.model → task model), and attached skills. config.skill_keys are resolved against the installed-skill catalog and injected as an Injected skills: block into that step's extra system prompt — additively, independent of whether the step runs as the default agent or a specialized agent. A disabled/deleted specialized agent on any chain step fails the run.
4. Intermediate steps get a chain-handoff system prompt (result is forwarded to the next agent; no channel delivery); only the final step gets the normal automation delivery-policy prompt.
5. Output nodes wired from a specific agent node deliver that step's result with the output's own notify_result_mode; outputs without an incoming agent edge (and legacy flat notify targets) deliver the final chain result. A failed step aborts the chain and follows the normal automation failure/notification path; the job's last_run_id points at the final executed step.

Scheduled tasks

UI entry point: /geplante-aufgaben, with a top segmented control for Automationen and Agenten.

1. Clapilot creates or syncs scheduled-task metadata rows in scheduled_tasks.
2. The /geplante-aufgaben UI opens create and detail/edit flows in a centered dialog instead of a permanently visible right-side detail panel, but it still reads and writes the same scheduled_tasks contract. Switching to the Agenten segment renders the specialist overview from /api/specialized-agents/overview; the old /modules/agents route redirects to that segment.
3. On native-runtime installs, Clapilot also ensures the default system automations exist: Agent Inbox Sync, User Inbox Sync, Inbox Document Processing, Document Post-Extraction Queue, Learning Curator, and Memory Dreaming. User Inbox Sync defaults to a 10 minute interval. The document queue is the durable, sequential execution path for the protected new_document post-extraction review and backs off globally on verified provider limits.
4. These preinstalled system automations can be paused or resumed from /geplante-aufgaben, but they cannot be deleted from the app UI or API.
5. Runtime execution is delegated through the native clapilot-agent runtime client.
6. New tasks created in Clapilot become native jobs immediately for schedule-based automations; direct event-triggered automations stay app-managed but use the same scheduled_tasks metadata row.
7. When scheduled_tasks.specialized_agent_id is set, both native schedule execution and app-managed event execution resolve that specialist at run time, switch to the specialist session key, and inherit the specialist's current prompt, injected skills, model, tool/auth scope, and history policy instead of the generic main-agent automation path. Queued document revisions use the same specialist envelope.
8. The native runtime stores jobs in agent_jobs, executes due runs internally, and keeps scheduled_tasks synchronized through the runtime adapter. Scheduled-task reads also overlay current agent_jobs state directly so protected system automation rows show the real next/last run even when the metadata mirror is stale.
9. Native system jobs that do not create a full agent_runs conversation still write compact agent_events entries (job.completed or job.failed) under their session key, so the /geplante-aufgaben run log can show operational history for mailbox sync, inbox processing, and Learning curation. The detail loader also checks the legacy native-job:<jobId> session key when a job has moved to a stable Clapilot automation key.
10. Every automation now carries one assigned target session/channel via notify_target_json, with user-owned rows defaulting to main_session, optional main_session.sessionId targeting a concrete web chat for self wake-ups, team_chat.roomId targeting a selected Teamchat channel/group, and system-owned rows defaulting to team_chat (#general).
11. Output nodes in workflow_config are the notification contract for both model-backed and native system automations. Adding an output node enables result delivery according to that node's notify_result_mode; removing all output nodes from a saved workflow makes the automation run-log-only. The default graph for User Inbox Sync intentionally has no output node.
12. Automation runs receive a system prompt that treats that assigned target as the single implicit destination for final user-facing replies, so prompts no longer need an extra "post the result to X" instruction. This is now enforced, not just instructed: when the agent itself posts to the assigned channel_approval target during the run via channel_send_message, the runtime detects that send (matched on the resolved approval id) and suppresses the duplicate final-reply auto-delivery to the same target, so the user receives the task work once instead of the work plus a separate confirmation/summary. The same dedupe applies to live channel runs against the originating thread.
13. main_session delivery resolves deterministically: it targets the user's stable main session (or an explicit pinned main_session.sessionId) and never promotes a most-recently-active session as a recency guess. A pinned sessionId that no longer exists fails loud into the run log instead of silently delivering into a different, guessed session.
14. Silent runs are suppressed through the automation-specific silent token path; failed runs still emit assistant-origin automation messages into configured output targets.
15. Clapilot keeps DB metadata as the stable app contract so the provider can be replaced without changing the page or live-agent tool surface.

Blueprints

/geplante-aufgaben offers a compact From blueprint gallery as an onboarding layer over the normal create flow. A user selects one of seven curated blueprints, reviews its prefilled typed slots and schedule summary, can rename the task, and creates a normal editable scheduled task. The localized prompt is stored on that task; blueprints add no new execution or persistence semantics.

The catalog contains:

• Morning briefing: weekday calendar, open/overdue tasks, important unread email, and optional configured news.
• Nightly inbox digest: today's new email grouped by importance with suggested next actions.
• Document inbox sweep: periodic review of unprocessed inbox documents.
• Weekly client report: open tasks, deadlines, and stalled items across Mandanten/customers.
• News clipping digest: configured news topics summarized daily.
• Email follow-up reminder: sent messages awaiting a reply beyond the configured number of days.
• Calendar conflict check: a 14-day look-ahead for overlaps and back-to-back events across locations.

The native agent has the same catalog through scheduled_tasks_blueprints_list and creates from it through scheduled_tasks_blueprint_instantiate. Both the UI and agent path reuse src/lib/automation-blueprints.ts and the existing scheduled-task creation service.

Chat streaming and attachments

Detailed request-routing and prompt-assembly diagrams:

• Clapilot-Agent Request Execution Flows

1. UI captures text input (typed or microphone voice-to-text dictation in supported browsers) and optional attachments.
2. Every user has one protected main personal session plus any number of custom sessions. The app defaults into the main session on open, and users can still create custom sessions and set a per-session model override (web and Apple clients). The shared /team-chat room keeps its own Angela model override separately from personal sessions.
3. API persists user message in chat_nachrichten with session_id.
4. If RAG_ENABLED=true, API attempts retrieval and injects document context as a system message, but only when an active Mandant context is present via mandantId or the current /mandanten/:id route.
5. With the native backend enabled, ClapilotAICore also bootstraps session memory from shared workspace prompt files and synchronizes memory/**/*.md into the native recall store before retrieval.
6. During an engine migration, admins can additionally import persisted OpenClaw transcript memory from OPENCLAW_STATE_DIR/agents/*/sessions/*.jsonl into the same native recall store without keeping OpenClaw as the active runtime.
7. Native memory retrieval uses OpenAI text embeddings for vector recall plus PostgreSQL full-text search for keyword recall; document RAG remains a separate document-centric retrieval layer.
8. Memory flush and history compaction never block the foreground request. The native runtime starts the user-visible run from the last completed summary plus a token-budgeted recent-turn tail, then schedules maintenance in the detached post-response pipeline.
9. A flush extracts durable facts from completed turns into native recall storage.
10. When the session is getting large, the background maintenance pipeline runs a native safeguard compaction pass: older completed turns are summarized into a persisted structured session summary with identifier retention and auditing, while only the recent preserved turn tail remains verbatim. Maintenance is single-flight per session and coalesces a busy session to its latest pending pass.
11. Future native runs inject the latest completed compaction summary back into the prompt and replay only turns after compactedThroughRunCount. If a background pass is still running, the current request remains safe because recent-turn replay is independently capped and fitted to the selected model's token budget.
12. For the current request, the session layer still trims the replayed recent tail to fit the remaining budget and may drop approved-learning or retrieved-memory context before dropping recent turns.
13. Native runtime now adds a fixed runtime-owned system prompt describing tooling, safety, workspace, memory, docs, current time, and runtime/session identity before the shared bootstrap files, retrieved memory, retrieved structured knowledge graph context, and approved learned context are appended.
14. Native execution no longer stops at a single completion call. clapilot-agent can now run an iterative agent loop with internal tools exec_command, package_install, context_search, context_get, memory_search, memory_get, memory_grep, memory_describe, memory_expand, knowledge_search, knowledge_get_entity, knowledge_neighbors, knowledge_explain_claim, learning_search, learning_get_object, and session_status across OpenAI, Anthropic, and OpenAI-compatible backends.
15. Stored codex_auth_json is synchronized into .clapilotaicore/.codex/auth.json. When standard native chat selects an OpenAI-Codex model, it uses the Codex bridge instead of direct OpenAI /chat/completions or /responses calls. That bridge now also registers a Clapilot MCP stdio server inside the private Codex CODEX_HOME, so Codex-backed GPT runs can call Clapilot-native app tools when they pass the current session_key. Interactive Codex app-server sessions resolve Codex service tier per model from Codex-auth provider metadata (modelServiceTiers), with Global falling back to CLAPILOT_AGENT_CODEX_SERVICE_TIER only when that environment variable is set. The Codex settings picker asks the installed Codex app-server for model/list and uses its additionalSpeedTiers to show per-model Fast/Flex choices only when the runtime advertises them. Without a configured tier, Clapilot leaves the app-server request tier unset. Anthropic-Claude subscription rows are provisioned through a Claude setup-token, persisted as a normal provider secret, and then executed through the local Claude CLI bridge with the token injected as runtime auth. The bridge scrubs higher-precedence Anthropic API-key, bearer-token, proxy, and cloud-provider env vars from that Claude CLI child process before launch and compacts copied whitespace inside setup-token values. Subscription usage does not depend on the setup-token: it reads the full Claude Code OAuth credential from the Claude CLI login home (written by the settings Claude Auth flow, auto-refreshed via the official OAuth refresh grant) and calls Anthropic's OAuth usage/profile endpoints; a setup-token alone is valid for model execution but is rejected for usage with 403 (user:profile scope). Their model picker uses the shipped Claude subscription catalog instead of live /v1/models probing. Embeddings still stay on API-backed providers.
16. Native chat runs execute as fully authorized owner sessions inside the Dockerized runtime so coding-agent behaviors such as CLI use and package installation are not blocked by an app-side approval policy.
17. Native memory flush and native compaction now remain authoritative for all supported providers, including Codex OAuth and Anthropic.
18. For Anthropic-Claude subscription runs, the bridge now consumes Claude CLI stream-json output directly. Text deltas stream through from the CLI, Claude built-in tool calls (Bash, Read, Grep, Glob, WebFetch, WebSearch, etc.) are surfaced as runtime tool lifecycle events, and Clapilot-native tools are exposed to the same Claude run through an MCP stdio bridge bound to /api/agent-runtime/tool-proxy.
19. Tool calls are executed inside the native runtime, logged into agent_events, and streamed on /internal/runs as tool lifecycle events before the final assistant answer is returned.
20. OpenAI/Anthropic tool batches opt into parallel execution only when every requested tool in that batch is on the read-only allowlist. Mutating tools remain serialized even when the upstream engine supports parallel_tool_calls.
21. Before transport selection, /api/chat still answers session identity questions about backend/provider/model deterministically from runtime session metadata instead of model self-reporting. Shell and bash snippets are no longer executed via server-side shortcuts here; they stay in the normal agent prompt so the runtime agent decides whether to call exec_command, package_install, or simply analyze the text.
22. If the user did not explicitly pin a model for the session/request, the native runtime resolves the chat model from the global native_model_routing.priority list; the first entry becomes the default and the remaining entries become cross-provider fallbacks.
23. API applies the selected session model to native session state before transport selection. In the web composer, model choices are grouped behind user-facing labels such as Schnell, Qualität, and Automation; technical provider/model identifiers remain optional detail instead of being the primary affordance. /team-chat uses the same grouped choices, but moves them from the footer into the clickable Angela row in the right sidebar via /api/chat/group/room-config.

• Native provider execution classifies HTTP 529/overload responses as transient. Before moving through the approved model/provider fallback order, it performs a bounded retry sequence with exponential backoff and jitter. Retries stop after any streamed output or tool execution so one user turn cannot deliver duplicate text or repeat a side effect. Provider-attempt events and run usage record the reason, action, retry delay, and final outcome; exhausted overloads are rendered as a localized retry status instead of exposing the raw provider error.

1. Before tool execution, native session state now self-heals stale rows by re-resolving missing user_id, chat_session_id, and a useful currentContext from the active request, chat_sessions, and recent user-linked session context.
2. Before transport selection, /api/chat no longer performs deterministic image slash-command shortcuts. Image-generation and image-editing requests stay in the normal agent prompt so ClapilotAICore decides whether to call images_generate or images_edit; when the current turn includes uploaded images, the API imports them as user-owned generated-image assets and passes their ids in runtime context so a later dynamic images_edit tool call can target the latest chat image without a transport-level shortcut.
3. For plain chat without attachments, API uses the native runtime stream client on /internal/runs.
4. API prefers /internal/responses for file/image input.
5. API falls back to /internal/chat/completions when needed.
6. SSE stream is passed through to UI. While an assistant turn is still in progress client-side, /chat can queue additional outgoing messages locally; they are shown above the composer, can be canceled before dispatch, and are sent in order only after the current turn reaches client-side end-of-turn ([DONE] / pending=false). Native clapilot.tool events also surface inside the active assistant bubble: personal /chat, the right-side personal chat sidebar, Team Chat, and the Apple chat clients now start with an animated ... bubble, type through short working states such as Verarbeitung..., Denke nach, and Bereite Antwort vor, and temporarily switch that same bubble text to the live tool label while the agent is executing a tool before the streamed answer replaces it. When the per-user tool-call visibility preference (chat_preferences.show_tool_calls) is enabled, personal /chat, the floating chat sidebar, Team Chat, and the iOS/macOS chat clients additionally render a persistent vertical tool-call timeline log inside the assistant message (one compact row per tool call with a tool-type icon, hairline connector lines, a Skript badge for shell commands, a running indicator, and an error badge for failed calls); the log updates live from clapilot.tool events while streaming and stays visible after completion because the entries are persisted in message_meta.assistantToolStatuses (each entry now also carrying the raw toolName). This works across all agent harnesses: native and embedded-Pi runs emit concrete tool names directly, Claude-CLI bridge runs report harness-prefixed MCP names (mcp__clapilot__...) that the clients normalize, and Codex/Claude subscription-bridge calls through the tool_execute MCP helper are unwrapped by the runtime to the dispatched concrete tool name before the event reaches chat clients and run logs. The web chat timelines now also render per-message times and insert day separator labels whenever the visible history crosses into another calendar day. Personal /chat, the floating chat, Team Chat, and the Apple chat clients support inline composer popovers for / commands, # document/image references, emoji insertion from the emoji button or : aliases such as :D, and where applicable @ mentions: selected # refs are shown as chips, the typed #... token is removed from the visible message body after selection, and the selected document context is injected into the turn.
7. The floating right-side chat derives the active page context from the current route plus window.__CLAPILOT_PAGE_CONTEXT__, shows a subtle context hint in the header area, and swaps the generic empty-state greeting for module-specific quick actions (for example Aufgaben, E-Mails, Kalender, Dashboard). Clicking one of these actions dispatches the mapped prompt as a real user turn through the same queue/send pipeline as typed chat input.
8. Final assistant output is persisted. Fresh custom sessions are auto-renamed from their first user turn unless the user already renamed them manually. Assistant-origin automation/system alerts use the same main personal chat timeline but are stored without a fake user prompt.
9. Detached post-response maintenance performs memory flush and history compaction, stores summaries/shared facts, and creates source-backed Memory v2 assertions plus canonical Learning projections from the same output. Flush and compaction are coalesced per session and cannot delay run.started or the user-visible provider response. An opt-out policy automatically activates safe direct-human facts and preferences; assistant-generated automation/job/heartbeat inferences, conflicts, corrections, procedures, weak evidence, and other exceptions receive non-prompt-eligible Learning review objects. Memory Dreaming excludes system/automation runtime sessions before its candidate limit so Team Chat and other human conversations are the primary consolidation source; a Dream assertion can auto-activate only when every cited input is an approved direct-human fact projection. The protected Learning Curator later uses its configured model to inspect only new or changed canonical facts, keeping by default and rejecting only high-confidence false, unsafe, duplicate, non-durable, or worthless content. Memory Dreaming v2 partitions active DB-backed inputs by audience, validates schema-constrained assertions and Wiki proposals with exact evidence, then immediately applies its targeted deterministic safety pass and processes approved projections before returning. Every normal Dream Wiki change remains a review-only draft; the idempotent one-shot migration-201 graph bootstrap disables Wiki output entirely and uses a version-gated scheduler type so an older rolling-deployment agent cannot consume it. Only approved assertions enter embeddings, durable recall, prompt-eligible Learning state, and the automatically maintained Knowledge Graph. Opt-out activations remain reversible, and manual decisions are authoritative.
10. After the reply is finalized client-side, the web app emits a toast preview and the native Apple client emits an in-app toast or local notification depending on foreground/background state.

External channel replies

Detailed request-routing and prompt-assembly diagrams:

• Clapilot-Agent Request Execution Flows

1. Telegram and Slack inbound events enter through /api/agent-runtime/channels/:channel/inbound and are forwarded to clapilot-agent.
2. Native WhatsApp now uses a backend-owned WhatsApp Web session in ClapilotAICore, not a token-based Cloud API path. Admins link it by generating a QR in ClapilotAICore settings, scanning it from the phone, and letting clapilot-agent persist the Baileys auth state under .clapilotaicore.
3. Native approvals and channel-thread binding are resolved before a conversation run starts.
4. Approved group thread bindings can now also persist the built-in global_team_service execution principal. The thread history still stays on channel:<channel>:<thread>, but execution is no longer forced to piggyback only on the linked human user id.
5. Slack and WhatsApp currently use the broader Clapilot channel-response bridge first and fall back to native execution when that bridge fails.
6. Telegram now runs directly through the native runtime agent path when streaming is enabled. Photo and document attachments are downloaded from Telegram, converted into native input_image / input_file message parts, and sent into the same multimodal runtime used by web chat attachments.
7. For linked Telegram groups, document-like text queries also preload likely Clapilot document matches into the native system prompt so the agent can answer against workspace documents even without a currently open UI document.
8. Telegram streaming uses one visible reply message and updates that same message in place while deltas arrive. Tool execution and long-running model steps keep the chat alive with Telegram typing actions.
9. During native WhatsApp Web runs, clapilot-agent owns the live socket, reconnect loop, inbound routing, and outbound delivery. The public webhook ingress is not used for the WhatsApp Web path.

Live voice chat (Realtime)

1. User starts Live Voice from the /chat composer via Unterhaltung starten; adjacent controls keep manual speech dictation and the abstracted model picker available in a quieter, left-to-right control strip.
2. In iOS/macOS app, Live Voice can be started from the floating action control across the native chat, documents, calendar, and tasks screens; chat also still exposes composer controls for mute/stop. The Apple Watch app starts the same realtime session contract from its Live Mode button, with watchOS microphone streaming and assistant audio playback.
3. API route /api/chat/live/session resolves the configured Realtime provider and current UI context (route/module/page context), then returns provider-specific connection metadata.
4. Browser starts the provider-specific transport: OpenAI API Realtime uses the GA WebRTC flow with a short-lived /realtime/client_secrets token and an SDP handshake against /realtime/calls; OpenAI-compatible/Azure providers may use their configured legacy-compatible WebRTC endpoint; Google Gemini Live uses the configured Live WebSocket endpoint with PCM audio streaming. Native Apple clients use the returned provider websocket_url for their URLSession WebSocket audio transport when the selected provider is OpenAI-family.
5. Session is configured with a tool catalog: direct simple tools (calendar_*, aufgaben_*, scheduled_tasks_*, excel_*, word_*, website_*, emails_*, documents_*, notizen_*, mandanten_*) plus clapilot_delegate.
6. Agent policy: use direct simple tools for straightforward reads/writes; if a direct lookup path does not resolve the request, prefer clapilot_delegate over replying with a dead-end "cannot".
7. /api/chat/live/tools executes simple tools in-process (DB/API/module calls), supports delegated gateway execution (/v1/responses, fallback /v1/chat/completions) with merged UI context and inferred mode, and can auto-fallback into clapilot_delegate for certain lookup-style direct-tool misses using the last spoken user utterance.
8. Tool responses may also include structured uiActions[] plus a canonical mutationEventId for visible mutations. Current examples: Excel cell writes emit excel.cells.updated, Word document writes emit word.document.updated, Notizen mutations emit notizen.folder.updated, notizen.note.updated, or notizen.page.updated, calendar mutations emit calendar.event.updated, Aufgaben mutations emit aufgaben.task.updated, Mandanten mutations emit mandanten.client.updated, and email draft mutations emit emails.draft.updated.
9. Mutation metadata is persisted in ui_mutation_events and exposed through /api/ui-mutation-events. The authenticated web shell consumes its SSE stream; native Apple task views use the topic-filtered JSON cursor and refetch only affected task IDs. Open pages therefore apply agent-driven actions from other surfaces without repeatedly reloading full collections.
10. The calendar page additionally listens directly to Postgres-backed /api/calendar/live notifications for calendar_entries, then refetches and diffs the visible range so moved or newly created events animate from DB state changes even without an explicit agent UI action.
11. The Aufgaben board also listens directly to Postgres-backed /api/aufgaben/live notifications for aufgaben, refetches the changed task row from DB state, and wraps Kanban card moves in a View Transition so status changes animate across columns instead of snapping.
12. Within each Aufgaben Kanban status column, cards are ordered by the displayed due date with the newest date first. Cards without a due date follow dated cards, and equal due dates are ordered by creation time with the newest task first. The Kanban request uses the matching frist_desc API order so the rule is applied before the result cap; the table view keeps the default API order, while explicit follow-up and revenue filters retain their specialized ordering.
13. Tool output is sent back into Realtime as function_call_output; the agent must follow with a user-facing answer in the same conversation.

Chat display links

1. Web chat, floating chat, CopilotKit-style inline cards, and Apple native chat surfaces detect contextual entity ids in assistant text, resolve human-readable labels when possible, and persist a compact reference card with clickable actions so raw ids do not become the primary user-facing label.
2. Supported contextual links include Aufgaben/tasks, E-Mails, drafts, calendar events, documents, and Mandanten/customers. Detection requires nearby entity wording plus an id-like token, or a known internal route such as /aufgaben/{id}, so ordinary labels such as Termin: Wiedervorlage and untyped internal UUIDs remain plain text.
3. Known task, email, draft, calendar, document, and Mandant references now try to resolve titles or subjects from stored app data before the reply is finalized, so the visible transcript prefers labels such as a task title or document name over a UUID. Hyphenated ids that wrap across a line break are normalized before lookup, which keeps generated draft ids recognizable even when the model inserted a newline. Assistant markdown also unwraps inline-code formatting only around contextual entity ids before detection, so agent responses and user bubbles follow the same normalization rules across web and Apple clients.

Apple Watch audio

1. The iOS app keeps its normal secure session, then publishes the active login to the paired watch over WatchConnectivity. The watch stores that payload in its own secure store and only shows its login form when no usable phone session is available.
2. Live Mode creates a personal chat session if needed, calls /api/chat/live/session with routePath=/watch and platform=watchos, then streams microphone PCM into the configured Realtime provider and plays assistant PCM output on the watch.
3. Walkie Talkie mode records a single AAC voice note, sends it to /api/chat as an audio attachment with the apple-watch page context, and streams the text reply while the backend transcribes the note.
4. Because personal chat treats audio attachments as audio-reply requests, the finalized assistant answer is synthesized to an assistantAudio attachment in chat_nachrichten.message_meta; the watch reloads recent history and downloads that asset through /api/chat/audio/:id for playback.

Team group chat

1. /team-chat exposes a shared room (clapilot-members) for all authenticated Clapilot users, separate from personal /chat sessions.
2. The right sidebar loads the global people directory only for direct-message targets, while channel participant lists, presence, typing, and human @mention suggestions come from active chat_room_members. Channel rows expose a settings button where room admins invite/remove people independently of agents, manage main/specialist invitation and reply modes, and opt into agent-to-agent conversation. New channels start with the creator only; general remains the compatibility default for every user.
3. The main-agent row in that sidebar appears only while the built-in assistant is invited to the selected room, loads the shared team-room model from /api/chat/group/room-config, and opens the same grouped model selector used by personal chat. Channel admins can invite/remove the main agent and set it to mention_only or all_messages; specialist rows open direct specialist rooms and their invitations use the same reply-mode vocabulary. Pending group replies render directly inside the transcript with the same animated working bubble and streamed typewriter output used by personal chat instead of a separate sidebar progress card. Personal /chat, the floating chat, and the Apple chat clients also smooth streamed assistant text client-side by buffering incoming deltas and typing them out letter by letter with the animated agent suffix until the final response is complete.
4. User messages are stored in chat_group_messages with sender metadata and optional attachments; direct messages reuse the same table with room-scoped room_id values (dm:<user-a>:<user-b>). Large inline agent avatars are replaced with stable entity profile-image URLs at the shared persistence boundary so repeated Base64 image payloads do not enter new Team Chat rows.
5. Group turns are sent through the selected runtime with explicit group-chat system context (not 1:1 assumptions).
6. Native Teamchat runs also bind the built-in global_team_service execution principal so the bot identity can stay service-scoped while the persisted room history remains tied to room_id.
7. The built-in main agent runs only while invited and according to its room policy: mention_only requires an explicit @ mention and all_messages runs on human channel messages that do not explicitly target a specialist. An explicit specialist mention is exclusive unless the same message also explicitly mentions the main agent. Invited specialists answer only when mentioned unless their room invitation is set to all_messages.
8. When no reply is needed, KI-Assistent returns NO_REPLY and no assistant message is persisted for that turn.
9. Typing @ in the shared room uses the returned member list plus the enabled specialized-agent catalog for client-side autocomplete, while typing # in either the shared room or a direct room resolves document/image references that are persisted in chat_group_messages.message_meta.documentReferences and surfaced back into the web and Apple transcripts as reference chips.
10. Each explicitly mentioned invited @agentHandle in a Team Chat turn runs as a detached specialist task with only the current request, the specialist's own isolated continuity, and a compact visible context bundle of up to the last 10 relevant room messages. Multiple explicit specialist targets are queued in parallel as one exclusive target set. The main agent runs alongside them only when it is explicitly mentioned in the same message; its all_messages mode does not override the explicit specialist target set. Otherwise no main-agent pending row or model run is created. Prior named specialist replies are included in the compact context. By default specialist replies do not trigger other agents. If a channel admin enables agent_to_agent_enabled, a completed visible agent message may trigger invited all_messages specialists, or one explicitly mentioned invited specialist. The reaction prompt requires material new information and exact NO_REPLY silence otherwise. A database-backed source marker, one-reaction-depth limit, and atomic shared cap of six reaction messages per root turn prevent recursive or unbounded loops even when a specialist ignores the prompt.
11. Team-chat messages can also quote earlier room turns through message_meta.replyReference; on web, right-clicking a room message opens an inline action popover with Antworten, Direktnachricht, and Text kopieren, while the Apple client exposes the same reply path via message tap plus context menu. For personal and team-chat agent runs, /api/chat also materializes the quoted row's stored image/file attachments and supported markdown image links (generated-image assets or workspace files), including forwarding existing generated-image asset ids to image/video tools.
12. Pending KI-Assistent group replies render a dedicated inline typing bubble in the room transcript; when tool events are available they are also persisted in chat_group_messages.message_meta.assistantToolStatuses so other room viewers can see the same live Angela progress state.
13. Automation posts delivered into a room (via /api/agent-runtime/assistant-message with a team_chat target) carry message_meta.automation_id, automation_title, and run_id. In the inlined group-chat history these rows are attributed as [Automations-Post "<titel>" von <agent> | Run <run_id>] (agent-initiated non-automation rows as [Agenten-Nachricht von <agent>]), so the replying agent knows what was posted, by which automation, and under which run.
14. When a group turn replies to an automation post (explicit replyReference on an agent message, or any turn shortly after an automation post landed in the room), /api/chat injects a system context block built by src/lib/automation-run-context.ts with the run status, the automation's stored prompt, and the full run output. For deeper follow-ups the agent can call scheduled_tasks_get_run_context with the run_id from the history entry; older automation posts outside that window still surface a short tool hint instead of the full block.

Document RAG indexing

1. dokumente insert/update enqueues a job in document_index_jobs.
2. document-rag-indexer.mjs claims pending jobs.
3. Worker extracts text from the target file and creates chunks.
4. Embeddings are generated and stored in document_chunks.
5. Job is marked processed or failed with last_error.

Optional PDF Inspector routing

scripts/document-extract.mjs can route PDFs through the MIT-licensed Firecrawl pdf-inspector CLI before the established pdftotext/Tesseract/Vision cascade. The integration is off by default (DOC_PDF_INSPECTOR_ENABLED=false) and uses CLI subprocesses only; it adds no npm package, N-API binding, postinstall hook, or Rust build.

Set DOC_PDF_INSPECTOR_BIN to a directory containing executable detect-pdf and pdf2md files, or to the detect-pdf executable (with pdf2md beside it). Without the setting, both command names are resolved from PATH. The published @firecrawl/[email protected] package does not provide Linux arm64 prebuilts; arm64 deployments therefore remain on the fallback until operators supply compatible CLI binaries themselves. Inspector class and confidence are returned by the extractor, copied into RAG metadata, and stored in nullable OCR-cache columns. The cache configuration hash includes an extractor version so old extraction results are not reused across this routing change.

Email delegation

1. User mailbox APIs resolve the local IMAP/SMTP credentials from user_profiles and, when enabled, the user's Google Workspace Gmail connection from google_user_integrations. Agent mailbox APIs prefer the dedicated account_type='agent' Gmail connection and fall back to app_settings.agent_email_* IMAP/SMTP credentials when no Agent Gmail account is enabled.
2. On native-runtime installs, User Inbox Sync polls configured personal inboxes and Agent Inbox Sync polls the agent inbox.
3. Mailbox sync claims messages in email_ingestion_jobs instead of relying only on IMAP unread flags.
4. Attachment files are copied into /app/workspace/mandanten/_inbox and placeholder rows are inserted into dokumente.
5. Agent mailbox runs can delegate the new message to the runtime for draft creation in email_drafts; when the sender matches agent_email_auto_reply_allowlist, the runtime is instructed to send that freshly created draft immediately as a safe-case auto reply.
6. Successful personal mailbox imports can trigger follow-up automation processing for that user without relying on a separate proactive scheduler.
7. Legacy script-based installs can still fall back to the older scripts/agent-email-poller.mjs path.
8. When personal mailbox auto-analysis is enabled in Settings -> Admin, successful User Inbox Sync imports immediately trigger the prepared-answer workflow in the background for that new message.
9. Before task creation runs, the personal mailbox batch classifies new mails as human, transactional, notification, marketing, or bounce based on sender, domain, and content signals.
10. Messages classified as notification, marketing, or bounce are auto-ignored, written back as no_reply_needed automation rows for inbox visibility, and their sender is appended to app_settings.email_auto_process_blocklist with an auto-classified:<class> reason.
11. The /emails inbox merges local IMAP, Gmail, Microsoft 365 Mail, and Apple iCloud Mail rows into one chronological personal inbox. Its Agent mailbox scope lists the dedicated Agent Gmail account when enabled. Provider message IDs include the Google account type where needed so detail, mutations, attachment downloads, automation, delegation, and draft sending stay tied to the correct source mailbox.
12. The inbox reads cached workflow records from email_thread_automations and matching email_drafts so the list itself can surface inline AI status, priority, preview, handled/deadline counts, and smart filter tabs with Wichtig as the default inbox view before the user opens a message.
13. The inbox UI also runs a lightweight catch-up trigger for visible personal messages that are still Analyse ausstehend, including IMAP, Gmail, Microsoft 365 Mail, and Apple iCloud Mail rows, so previously imported but not-yet-analysed rows are processed without forcing the user to open each message manually.
14. Auto-analysis only runs for INBOX messages: opening or listing mails in sent, drafts, trash, archive, or custom folders never starts the prepared-answer workflow, and the workflow card is not rendered there (isInboxEmailFolder guard in both the automation endpoints and the /emails UI).
15. Opening an email detail in /emails loads attachment metadata, imports each attachment idempotently into dokumente as source_type='email_attachment', and still runs the prepared-answer flow on demand when fresh automation data is needed or a manual retry is required. Personal mailbox attachments land in the signed-in user's private Persönlich folder; agent mailbox attachments remain shared. Imported attachments are queued for document indexing; scanned PDFs first try pdftotext, then local page OCR, then optional configured vision extraction. The web detail UI renders each attachment as a document preview tile that opens /dokumente?preview=:id, while the separate download button keeps the direct mailbox attachment download behavior. The Apple client consumes the same linked document metadata, adds the imported documents to its native document cache, opens the attachment tile in the native document detail screen, and uses the authenticated attachment endpoint for its download/share action.
16. Mail attachment normalization hides inline/signature images from the normal attachment list before document import and download-index assignment. The shared filter checks inline disposition, content ids referenced from cid: HTML images, provider inline flags, and small signature-style image names such as image001.png; explicit real file attachments such as PDF/DOCX/XLSX and non-inline image attachments remain visible.
17. The app resolves the likely Mandant from sender data and recent workspace context, extracts customer master data from sender/signature/body content, updates an existing mandanten record or creates a new one when needed, then loads open tasks / recent actions / recent documents and persists the workflow record in email_thread_automations.
18. Before reply generation, the prepared-answer workflow runs an attachments stage: it imports the mail attachments into dokumente itself (using the same idempotent source_type='email_attachment' import as the detail view, including provider-backed Gmail/Microsoft/Apple attachment reads), enqueues document indexing, and waits within a bounded time budget (EMAIL_AUTOMATION_ATTACHMENT_WAIT_SECONDS, default 60s) for text extraction of analyzable attachments (PDF, images, text-like files up to 25 MB). Extracted attachment content (Inhalt (Auszug)) is then included in the reply-generation prompt so drafts answer from the actual attachment content instead of only filename/type metadata. If extraction is still pending when the wait budget ends, the draft is deferred (no reply is generated from metadata alone): the automation stays in the attachments stage and the RAG indexer resumes it through /api/internal/emails/automation/personal once extraction lands. Only terminally failed/empty extractions proceed without content, explicitly marked so the prompt cannot invent attachment content. Language-toggle draft regeneration reuses the same stored excerpts. The automation metrics record attachments_considered, attachments_pending_count, attachments_total_count, and attachments_skipped_count; an attachment that cannot be analyzed is therefore recorded as skipped instead of leaving consideration unknown. The workflow card shows a dedicated attachment-processing step with whether attachment content made it into the draft plus a short content summary of the analyzed attachments.
19. If a reply is warranted, Clapilot detects the source message language across supported UI languages (de, en, it), creates or updates a draft in email_drafts with auto_generated=true, and defaults the generated reply to the sender message language. The email detail view can regenerate the draft between source_language and the user's current UI reply_language.
20. When an auto-generated reply draft is sent from Clapilot, the matching email_thread_automations row is marked sent and the exact linked email-origin task (auto_task_id plus source_type='email' / source_id) is automatically moved to erledigt. A system comment on the task records Automatisch erledigt, weil E-Mail beantwortet wurde. for the visible audit trail. External sent-mail detection remains conservative until provider thread/message reference storage is expanded; the current default automation only completes tasks for replies sent through Clapilot's draft send paths.
21. Personal generated replies and newly created personal drafts append the user's configured /profil/email outgoing signature after generation. Plaintext is always retained as fallback; when an HTML signature is configured, email_drafts.inhalt_html stores the HTML alternative and SMTP/Gmail/Apple Mail send paths deliver multipart/alternative. Uploaded signature logos are stored in draft HTML as data images for portability, then rewritten to cid: references with inline MIME image parts at send time so Apple Mail, Gmail, and Outlook recipients do not depend on public image URLs. When a signature exists, generation prompts ask the model to omit its own closing/signature so the saved draft has one consistent footer.
22. When the workflow detects a concrete task or document request such as a contract revision, Stellungnahme, or requested Unterlagen, personal mailbox processing creates tasks in the signed-in user's private Privat board and creates/updates markdown document drafts in the private Persönlich folder. A plain incoming attachment is not treated as a prepared document draft; it stays an imported attachment document until the mail text explicitly asks Clapilot to create a new document. Agent mailbox processing remains shared. Mandanten created or updated from personal email signatures still land in the shared client register.
23. The prepared-answer card is rendered as workflow output instead of a chat bubble: it shows Mandant context, a Basierend auf E-Mail von ... source jump back to the original message, and any resulting task/deadline inline with the suggested reply.
24. The new-message composer loads recipient suggestions from Mandanten, team users, and the current user's previous draft recipients; An and CC normalize comma/semicolon-separated addresses before sending.
25. The detail screen prioritizes one main decision: send the prepared answer. Secondary actions stay lighter, while detected tasks/deadlines can still be adopted into Aufgaben / Kalender from the same flow and a prepared document can be opened directly in Word Editor.
26. If the message contains a clear follow-up, the workflow can auto-create a task in aufgaben; appointment suggestions stay available as one-click calendar actions and can be persisted into calendar_entries.
27. Inbox quick actions such as mark-read, archive, move, delete, and one-click task/calendar creation now also emit UI mutation events so /emails, /mandanten, and open task views can react without a hard refresh.
28. Persistent personal-mailbox filters are evaluated case-insensitively against sender, subject, and preview/body text in both the native recurring IMAP sync and the web inbox sync. Matches are marked read and moved to the on-demand Nicht relevante mails IMAP folder or Gmail label before prepared-answer automation runs.
29. Workflow status is rendered as a user-facing flow (E-Mail verstanden, Kontext, Analysiert, Antwort erstellt oder Keine Antwort nötig, Aufgabe erkannt, Termin erkannt) rather than technical debug output.
30. The workflow model is configurable globally through email_analysis_model; the Admin page now exposes this as a dropdown of all globally available runtime models. Empty or default keeps ClapilotAICore's normal default model selection, while an explicit provider/model slug pins this flow to a lighter or faster option.
31. Reply drafting is intentionally framed as a reviewable user draft, not as an autoresponder. The prompt asks for a concrete substantive answer whenever the incoming mail contains an answerable question.
32. If answer preparation fails, the UI shows a retry state (Antwort fehlgeschlagen). The automation may retry structured generation or salvage malformed/plain-text model output, but it no longer persists a canned deterministic fallback draft as Antwort vorbereitet.