Clapilot Docs
Docs

Live Stream Studio

Admin workflow for operating a generated 24/7 YouTube live stream from Clapilot.

This page explains the data model, operating steps, agent tools, and safety rules for Live Stream Studio. It is for admins running a generated 24/7 YouTube stream from Clapilot and for developers extending the media pipeline.

Live Stream Studio is the Clapilot-owned workflow for a generated 24/7 YouTube stream. The web app owns planning, approval, queueing, and status. The separate clapilot-streamer process owns FFmpeg and pushes the configured RTMPS stream so media encoding never blocks Next.js or ClapilotAICore.

Data model

The authoritative state is stored in Postgres:

  • livestream_channels: YouTube RTMPS configuration, encrypted stream key, fallback settings, desired stream state, streamer heartbeat, current playback pointers, and the agent top-up schedule (agent_topup_*).
  • livestream_assets: generated or uploaded media briefs/assets, prompts, provider/model provenance, duration, media paths, moderation state, rights summary, and music license notes.
  • livestream_queue_items: ordered approved clips waiting to play, currently playing, or completed/skipped/failed. Loop-selected rows are durable queue selections, not one-shot playback history.
  • livestream_runs: FFmpeg run history and heartbeat metadata.
  • livestream_events: operational audit log for channel changes, queue updates, stream starts/stops, and FFmpeg failures.
  • livestream_chat_messages: deduplicated YouTube Live Chat messages read through the linked Google OAuth user.
  • livestream_audience_requests: classified viewer questions, music wishes, video wishes, and topic ideas derived from chat messages.
  • media_generation_provider_configs: provider-specific endpoint/model configuration and defaults for Gemini video/music, xAI Grok video, plus Kie.ai video/music providers; Kie.ai rows store the shared encrypted Kie.ai key while Gemini and xAI media use their existing Runtime provider credentials.

Operation

  1. Admin configures the YouTube RTMPS URL and stream key under /livestream in the collapsed encoder options.
  2. Admin gives the stream a natural-language direction in the Stream Briefing prompt, for example chill music, action visuals, target clip length, and moderation constraints.
  3. ClapilotAICore uses livestream_set_direction or the Studio prompt to enable top-up and decide whether to submit video-generation jobs, music-generation jobs, or both.
  4. For image-based/news/TTS clips that should not use text-to-video providers, ClapilotAICore can call livestream_render_html_video. That renders local SVG/HTML-style slides with the configured Gemini TTS route and FFmpeg, registers the real media_path, and can approve/enqueue the result in one tool call. If Gemini TTS is unavailable, the renderer falls back to espeak-ng and stores that fallback in asset metadata.
  5. Provider-generated or locally rendered media is registered with a real media_path, reviewed, approved, and added to the queue.
  6. The Studio also scans the shared livestream media directory for video files and shows them as a draggable file library with inline 16:9 previews. The library is paginated (12 files per page, newest first) so large asset folders can be browsed page by page; the first page ships with the studio summary and later pages load on demand from GET /api/livestream/studio?media_only=1&media_page=N. Dropping a file into the Live Queue registers or reuses an approved uploaded asset and appends a queue item. Admins can delete files from this library when they are not pending or playing; linked livestream assets are archived and their media_path is cleared.
    • Each library file also has an Auf YouTube (Upload to YouTube) action. It calls POST /api/livestream/youtube-upload, which generates a title and description via the social-media module's AI draft endpoint, stages the clip in the social-media media directory, and creates + publishes a public YouTube video through a connected YouTube account. A connected YouTube account in the social-media module is required; the first active account is used. The agent tool livestream_upload_clip_to_youtube performs the same flow.
  7. Pending Live Queue rows can be reordered by dragging them within the queue. Each queue row also has a loop toggle and per-cycle count: 0 disables looping, while 1 or more keeps the row in the loop and inserts it that many times into each playlist cycle. Loop-selected rows remain in Postgres until an admin skips/removes them; after an ungraceful streamer restart, stale playing loop rows are recovered to pending. For example, loop counts 1, 2, 1 play the three videos as 1, 2, 2, 3, then repeat the same cycle.
  8. Admin can connect Google OAuth with YouTube readonly scope in the YouTube Live Chat panel, then enable chat ingest. clapilot-streamer calls the internal chat poll endpoint, which discovers the active broadcast, reads chat messages, stores them, and turns obvious questions/music/video/topic wishes into audience requests. Once a broadcast ID has been discovered, the Studio shows a muted embedded YouTube preview player above the queue so operators can watch the actual public stream from the same screen.
  9. ClapilotAICore can read those requests through livestream_audience_requests, then answer with a short stream segment, queue a requested music/video clip, or mark unsafe/irrelevant requests ignored. Chat text is treated as untrusted audience input, not as agent instructions.
  10. clapilot-streamer polls Postgres. When stream_desired_state=running, it streams the first pending approved media item.
  11. When agent_topup_enabled=true and queued playable media is below the configured target, clapilot-streamer calls the internal top-up endpoint. That wakes ClapilotAICore with current queue context so the agent can start media-generation jobs or queue ready assets.
  12. If no playable queue item exists, the streamer sends the configured fallback video or a generated Clapilot slate with silent AAC audio.

The MVP uses a manually configured YouTube stream key. YouTube OAuth / Live Streaming API broadcast creation can be added later without changing the queue or streamer contract.

Agent tools

Native ClapilotAICore sessions can use:

  • livestream_status
  • livestream_audience_requests
  • livestream_resolve_audience_request
  • livestream_create_brief
  • livestream_register_asset
  • livestream_approve_asset
  • livestream_enqueue_clip
  • livestream_upload_clip_to_youtube
  • livestream_set_stream_state
  • livestream_configure_agent_topup
  • livestream_set_direction
  • livestream_render_html_video
  • livestream_generate_video
  • livestream_generate_music

These tools mutate the same DB-backed state used by the UI and return refreshTopic=workflow for visible refreshes.

Media generation providers

Media-generation provider settings live under Settings -> ClapilotAICore -> AI Media. The settings UI exposes:

  • Video generation provider/model selection between Gemini/Veo and Kie.ai video models.
  • Music generation provider/model selection between Gemini/Lyria and Kie.ai music models.
  • One shared Kie.ai API-key field used by Kie.ai video and music.
  • No separate Gemini media key field; Gemini media requests reuse the configured Google Gemini Runtime provider key.

Kie.ai video supports the existing Veo 3 presets plus API Market models kling-3.0/video, bytedance/seedance-2, bytedance/seedance-2-fast, and bytedance/seedance-2-5. Veo requests use the Kie.ai Veo endpoint, while Kling and Seedance requests use Kie.ai createTask with model-specific duration, aspect-ratio, audio, and resolution parameters from provider settings. Seedance 2.5 accepts any integer duration from 4 to 30 seconds and omits the legacy Seedance 2 web_search/nsfw_checker flags. The streamer wakes an internal polling endpoint every 30 seconds so Kie.ai tasks with a stored task ID are checked through record-info / recordInfo, downloaded into the shared livestream assets directory, and moved from generating to review_ready or approved.

The first generation tools submit provider jobs and persist provenance on livestream_assets. Async video/music providers may leave assets in generating until a callback/polling follow-up registers the rendered file path; only reviewed assets with a real media_path can enter the stream queue.

Safety and reliability

  • Generated media must be rendered ahead of playback; the streamer does not wait on live AI generation.
  • Queueing requires approved assets with a real media_path.
  • Media-library deletion is restricted to files inside the shared livestream media directory and is blocked while a file is pending or playing in the active queue.
  • Music provenance and rights notes are persisted on each asset.
  • Stream key material is encrypted at rest using CLAPILOT_LIVESTREAM_SECRET, falling back to CLAPILOT_AGENT_CONFIG_SECRET or AUTH_SECRET.
  • Media-generation API keys are encrypted at rest using CLAPILOT_MEDIA_GENERATION_SECRET, falling back to the normal runtime secret chain.
  • The fallback stream is mandatory so an empty queue does not stop the channel.