Clapilot Docs
Docs

Product Telemetry

Privacy-aware internal product event tracking and pilot usage reports.

This page explains what product usage data Clapilot records, what is deliberately excluded, and where admins can see the resulting reports. It is for operators running pilots or fleets who need usage insight without exposing user content, and for developers adding new tracked events.

Clapilot stores minimal internal product telemetry in product_events. The default implementation has no external analytics provider dependency and can be disabled with ANALYTICS_DISABLED=true or NEXT_PUBLIC_ANALYTICS_DISABLED=true.

Privacy Boundaries

Telemetry must not include chat prompts, chat answers, email bodies, document contents, tenant contents, phone numbers, API keys, tokens, secrets, or sensitive free-text notes.

Allowed event records are metadata only:

  • event_name
  • module
  • route
  • success
  • duration_ms
  • error_code
  • internal IDs such as user_id and session_id
  • coarse user_agent and platform
  • allowlisted structured properties such as flow, step, module_slug, attachment_count, and document_reference_count

The server validates event names, drops sensitive property keys, limits text and JSON sizes, and rejects unknown event names.

Events

Supported events:

  • user_signed_in — successful login (POST /api/auth/login)
  • user_signed_out — explicit logout (POST /api/auth/logout)
  • user_login_failed — rejected login attempt (error_code: invalid_credentials)
  • chat_message_sent — chat message submitted
  • module_opened / athlete_matching_opened — module route opened (automatic client tracking)
  • match_viewed, outreach_draft_created, followup_created — reserved for athlete-matching module instrumentation
  • microsoft_connect_started / microsoft_connect_succeeded / microsoft_connect_failed — Microsoft OAuth flow
  • document_uploaded — document uploaded via POST /api/documents/upload
  • email_sent — outgoing email sent (property source: gmail, apple, or smtp)
  • task_created / task_completed — Aufgaben created or moved into any configured done category status
  • calendar_event_created — calendar entry created
  • scheduled_task_created — scheduled task created (property scope: trigger kind)
  • issue_reported — issue reporter submission (properties source: destination, app: normalized repository basename)
  • api_request_failed — automatic client tracking when a same-origin /api/* fetch fails with a network error or 5xx status (throttled, telemetry endpoints excluded)
  • error_shown — automatic client tracking for unhandled window errors and promise rejections (throttled)

Client Tracking

Use trackEvent() from @/lib/telemetry for fire-and-forget client events. The helper is a no-op when analytics are disabled and sends only sanitized metadata to POST /api/telemetry/events.

trackEvent("module_opened", {
  module: "athlete-brand-matching",
  route: "/modules/athlete-brand-matching",
  properties: {
    module_slug: "athlete-brand-matching",
    flow: "athlete-brand-matching",
    step: "opened",
  },
});

Pilot Reports

Authenticated users can call GET /api/telemetry/report?from=2026-05-01&to=2026-05-03. Admins can add user=<uuid|email|name> to inspect one user; non-admin users are scoped to their own events. The legacy user_id=<uuid> parameter is still accepted.

The report returns active users, sessions, top modules, top events, top errors, failed event count, flow-level friction points where starts are not matched by completions or failures occurred, and a timeseries array of per-bucket activity (bucketInterval is hour for ranges up to 3 days, otherwise day).

Tracking Dashboards (Admin -> Hub)

The tracking dashboards live under Admin -> Hub (/admin/hub). The old /admin/product-health page redirects to /admin/hub/instances/local.

The hub admin page lists all instances with tracking data:

  • the local instance (always shown, marked "This instance")
  • in local hub mode, every monitored/connected instance with health status, last-seen, and telemetry summaries (events and active users in the last 7 days, last event timestamp)

Selecting an instance opens /admin/hub/instances/[id] with the full tracking dashboard:

  • activity time-series chart (events, failures, active users)
  • active users and sessions in the selected range (24h / 7d / 30d / custom)
  • top modules and top events as ranked bar lists
  • friction flows where starts are not matched by completions or failures occurred
  • top error codes and failed events
  • pilot health through recent user/session activity and latest event metadata
  • a user filter that accepts an email, display name, or user UUID

Token-Verbrauch pro Instanz

The same admin dashboard also shows model token usage for the selected range: total input/output tokens, top models, top providers, and an hourly or daily token time series. The Hub instance list includes the compact total-token count for the previous seven days. This is usage accounting only; Clapilot does not calculate currency costs.

For the local instance, agent_model_request_logs is the source of truth. In remote hub mode, the existing debounced telemetry run reads new rows with a keyset cursor from model_usage_hub_sync_state and sends privacy-reduced batches to POST /api/hub/telemetry/model-usage/ingest. The payload never includes usage_json, prompt-layer token details, metadata, error messages, session keys, or endpoints. The hub stores accepted rows in hub_instance_model_usage_logs, deduplicated by instance host and source log ID, and aggregates them for the selected instance and user filter.

Token tracking is part of the same admin-only operations dashboard as the other Product Health signals. Agent/chat tooling is intentionally not included in this feature.

The dashboard calls GET /api/hub/telemetry/instances/[id]/report (admin only). id=local reads local product_events; monitored instance IDs read the hub-synced hub_instance_product_events table. The response only includes metadata and aggregates plus user labels such as email/display name for admin identification. It does not return chat content, email bodies, document contents, tenant contents, secrets, tokens, or sensitive free text.

These dashboards are admin-only operations views and intentionally have no chat-agent tool wiring.

Hub Telemetry Sync (Instance -> Hub)

When hub_mode is remote, the instance automatically reports its product telemetry to the configured hub:

  • after events are recorded, a debounced background sync pushes new product_events rows (with user email/display-name labels) in batches to POST /api/hub/telemetry/ingest on the hub, signed with the shared secret (same HMAC scheme as issue reports)
  • the sync cursor is stored in product_telemetry_hub_sync_state, so each event is sent once; the hub deduplicates by (instance_host, source_event_id)
  • admins can trigger an immediate sync from Admin -> Hub ("Sync now") or via POST /api/telemetry/hub-sync
  • the hub stores incoming events in hub_instance_product_events and auto-registers the sender as a monitored instance (discovery source telemetry)

Instances in local or disabled hub mode do not send telemetry anywhere; their data stays in the local product_events table.