Background Workers

This page lists every background process that entrypoint.sh starts inside the clapilot web container, what each one does, and which of them are legacy fallbacks that native ClapilotAICore system automations replace. Read it when you need to know "what runs in the background and where".

There are two automation layers:

• Web-container workers: Node scripts started by entrypoint.sh next to the app server, plus singleton in-process workers registered by src/instrumentation.ts.
• Native system automations: preinstalled jobs owned by clapilot-agent and visible in /geplante-aufgaben (Agent Inbox Sync, User Inbox Sync, Inbox Document Processing, Document Post-Extraction Queue, Learning Curator, Memory Dreaming, Skill Curator).

When CLAPILOT_SYSTEM_AUTOMATIONS_ENABLED=true (the default), the two legacy workers — task scheduler and agent email poller — are skipped at startup and their responsibilities move to the native automations. All other workers always run alongside the native runtime.

Instance performance check (user-schedulable)

In addition to the preinstalled system automations above, users can create a Performance Check automation from /geplante-aufgaben (create dialog → Task type → Performance check). It is a normal, editable scheduled task, so the user picks when it runs (once / interval / weekdays) and the report channel (the automation result target — main session, a team-chat channel, or an approved external channel) with the standard controls.

Unlike an agent task it does not spend an LLM turn: it maps to the deterministic clapilotPerformanceCheck job payload handled in services/clapilot-agent/src/jobs/system-automations.mjs (runPerformanceCheckJob). On each run it queries agent_model_request_logs over a lookback window and computes:

• overall model-request failure rate (failed / total),
• interactive agent-turn latency p50/p90 for user-facing source_types (chat, group_chat, channel),
• the top recurring errors.

It fails (and always posts) when the failure rate exceeds maxFailureRatePct (default 10) or the interactive p90 exceeds maxInteractiveP90Ms (default 90000); the lookback defaults to windowHours: 24. All three thresholds are configurable per task. The rendered report is delivered through the existing automation result-target path (deliverSystemAutomationRunResult → /api/agent-runtime/assistant-message), honoring the task's notify_result_mode. Use Run now on the task for an on-demand report.

Worker inventory

Document post-extraction review events are drained by the native clapilot-system:document-postprocess queue job. The queue is durable per document event, collapses identical burst callbacks by document_id plus a canonical payload hash, and creates a new revision when an already-seen document's extracted content changes. A revision that arrives while the previous revision is running is retained and becomes pending after the older run finishes, so updates are not lost. The worker processes up to ten documents per pass but executes model runs sequentially. Each run uses a stable queue-row/revision idempotency key, preserves configured specialist prompts/skills and delivery targets, and counts only real execution failures against the retry budget.

Only provider-originated errors are eligible for global 429/provider-limit handling; arbitrary document or model output text cannot pause the queue. Structured reset metadata and short runtime cooldown messages such as Retry in 12s are honored before the one-hour fallback. A pause records one warning event with the enabled waiting-document count, leaves failure attempts unchanged, and reschedules the job to the provider reset time. Processed and terminal rows older than DOCUMENT_POSTPROCESS_QUEUE_RETENTION_DAYS (90 by default) are pruned in bounded batches.

Task scheduler (legacy)

Script: scripts/task-scheduler.mjs

Current responsibility:

1. Process inbox trigger files.
2. Index _inbox files into dokumente placeholder rows.

Scheduled task execution is no longer performed by this worker. Planned/recurring tasks run via native clapilot-agent jobs (agent_jobs), while Clapilot keeps synchronized DB metadata in scheduled_tasks.

When CLAPILOT_SYSTEM_AUTOMATIONS_ENABLED=true (default), this legacy worker is skipped and replaced by the preinstalled Inbox Document Processing automation.

Learning curator

Runtime owner: clapilot-agent

The preinstalled Learning Curator automation runs inside the native runtime with the model selected on /geplante-aufgaben. Safe evidence-backed facts use an opt-out policy and become active immediately. The scheduled curator checks only canonical facts that are new or whose content changed since its last decision; keep is the default, while high-confidence false, contradicted, duplicate, non-durable, runtime-noise, unsafe, or clearly worthless facts are rejected. Its payload supports model, batchSize, and minimumRejectConfidence (default 0.8). A blank model fails closed. Every result records model and decision provenance, and manual admin/user decisions are authoritative. Dreaming v2 retains a targeted deterministic safety pass for its own new assertion ids so eligible assertions can reach durable recall and the Knowledge Graph immediately, while the scheduled agentic curator supplies asynchronous quality control.

Legacy upgrades may also receive the hidden one-shot Memory v2 graph bootstrap native job from migration 201_memory_v2_graph_bootstrap.sql. It is inserted only when historical Dream v1/legacy graph data exists while no active v2 graph exists; a completed Dream v2 does not suppress the repair when it produced no graph. Its version-gated job type and payload are ignored by older agents during a rolling restart. The fixed-id job runs forced, graph-only Dreaming with Wiki proposals disabled and disables itself only after all partitions, targeted curation, and projection-outbox work succeed; skipped/partial/empty or failed work remains retryable. A migration rerun never re-enables an existing completed/disabled row.

The native Memory v2 projection worker starts with the agent and polls every 30 seconds. It claims due assertion-projection outbox rows with the ledger lease, coalesces graph work per Dream or instance, and never overlaps a still-running local batch. Provider and validation failures remain in the bounded outbox retry lifecycle instead of waiting for another Dreaming or retention run to wake them.

Skill Curator

Runtime owner: clapilot-agent

The preinstalled Skill Curator runs daily at 03:40 Europe/Berlin in the isolated clapilot-system:skill-curator session. It loads every workspace skill marked origin: agent, then selects up to 20 completed non-system runs from the previous 24 hours that contain successful tool activity. Only trimmed request/outcome summaries are sent to one schema-constrained maintenance-model completion; full transcripts are not included.

The model can propose at most six create-draft, update, merge, or archive actions. The runtime independently enforces the action cap, safe slug and body limits, agent-only ownership, pinned-skill immunity, and the invariant that no skill file is deleted. Proposed new skills remain drafts until an admin activates them in the Skill Store. Every execution, including clean skips when the maintenance model is unavailable, is audited in agent_skill_curator_runs.

Agent email poller (legacy)

Script: scripts/agent-email-poller.mjs

1. Load mailbox settings from app_settings (env overrides allowed).
2. Connect via IMAP and fetch unseen UIDs.
3. Claim UID jobs in agent_email_jobs.
4. Auto-reply allowlisted senders or delegate to the runtime.
5. Persist draft/result and update job state.

Requires runtime dependency imapflow.

When CLAPILOT_SYSTEM_AUTOMATIONS_ENABLED=true (default), this legacy worker is skipped and replaced by the preinstalled Agent Inbox Sync and User Inbox Sync automations. User Inbox Sync defaults to every 10 minutes and no output node, so it writes operational history to the run log unless an admin adds an output node. Native mailbox sync writes attachment files into _inbox, inserts placeholder dokumente rows, tracks processing in email_ingestion_jobs, forwards the configured agent_email_auto_reply_allowlist safe-case flag into the delegated agent prompt for immediate agent-mailbox auto-send decisions, and also triggers the personal /emails prepared-answer workflow automatically for newly imported user mailbox messages when the global email_auto_analysis_enabled setting is on.

Worker controls

TASK_SCHEDULER_ENABLED=true
TASK_SCHEDULER_POLL_SECONDS=30
TASK_SCHEDULER_BATCH=10
TASK_SCHEDULER_RETRY_MINUTES=5

AGENT_EMAIL_POLL_ENABLED=true
AGENT_EMAIL_POLL_SECONDS=60
AGENT_EMAIL_POLL_BATCH=10
AGENT_EMAIL_POLL_RETRY_MINUTES=10
AGENT_EMAIL_POLL_STALE_MINUTES=30

RAG_INDEXER_ENABLED=true
RAG_INDEXER_POLL_SECONDS=30
RAG_INDEXER_BATCH=4
RAG_INDEXER_RETRY_MINUTES=10

GOOGLE_SYNC_ENABLED=true
ICAL_SYNC_ENABLED=true
CLAPILOT_INSTANCE_CLEANUP_ENABLED=true
CLAPILOT_SOCIAL_MEDIA_PUBLISHER_ENABLED=true

CLAPILOT_SYSTEM_AUTOMATIONS_ENABLED=true