Image Playground

What it does

Image Playground opens with a responsive gallery of saved image versions, newest first, and a + New image tile. Select a thumbnail to reopen that image in the editor, or select New image to open an empty canvas. Use Back to gallery to return to the overview. Starting a new image or editing an older version preserves the other gallery entries, within the existing limit of 80 retained versions. Enter a prompt, optionally attach reference images, and generate the first image. The global prompt bar stays visible at the bottom for subsequent changes. Right-click the image to open a point editor at that coordinate, with a coordinate hint, optional feathered mask, and optional reference image. Each result replaces the canvas image. Version history supports undo, redo, and restoring an earlier thumbnail.

The module API persists the current user's state through GET/PUT/DELETE /api/modules/image-playground/api/state, using per-user JSON files under the persistent workspace at <workspace>/.clapilotaicore/module-state/image-playground/ (the workspace root comes from CLAPILOT_WORKSPACE_DIR/OPENCLAW_WORKSPACE_DIR, default /app/workspace). State is never written inside the module folder, because the bundled-modules tree is part of the Docker image and is replaced on every update. Reopening the module displays the gallery from that state; a corrupt state file is treated as empty and replaced by the next save. If the state request itself fails (for example during a container restart), the module locks every editing control, shows a reload action, and never saves until a later load succeeds, so a transient outage cannot overwrite the stored versions. The last debounced save is flushed with a keepalive request when the module is closed. Image assets remain in generated_images; GET /api/modules/image-playground/api/health checks module health.

Model selection

Like Whiteboard, the model picker reads /api/generated-images/models with cache: "no-store". Selecting a model sends its exact model and optional provider_slug; Default sends neither.

How point edits work

The design uses option a1: the selected coordinates are encoded in the provider prompt. In addition, the client generates a PNG mask with a transparent, feathered circle and imports it as a generated-image asset, then sends its ID as mask_image_id. Transparent pixels identify the editable region. OpenAI API-key routing sends this as a real inpainting mask in the /images/edits mask field; Codex/Gemini routing receives it as a guidance image, and xAI keeps the mask ahead of extra references when its three-image input limit is reached. The mask is rendered at the exact pixel size of the source image, because the OpenAI mask must match the image dimensions. Sources beyond the browser canvas budget (longest edge over 4096 px or more than 16.7 megapixels) skip the mask with a notice and rely on the coordinate hint alone. The mask can be switched off for each edit. The coordinate hint remains part of the prompt.

Mask and reference uploads are imported with import_source set to image-playground-mask, image-playground-reference, or image-playground-upload in the asset metadata, so throwaway masks can be told apart from user drops and pruned later. Each attached reference is imported once and its asset ID is reused across retries; a point mask is likewise reused while the base image and point stay the same, so a failing provider call does not multiply stored assets. Cleanup itself is not part of this version.

Reference images

Multiple visual references use reference_image_ids on the existing generated-image endpoints. For edits, the first source image is always the image being edited, followed by the extra references. Both endpoints echo only the references the provider actually forwarded, so the client can tell when a provider cap dropped one; the version history records that forwarded count, not the requested one. Each version also stores the asset MIME type so downloads carry a matching file extension. Generation with references uses the edit pipeline. Uploads are persisted through POST /api/generated-images/import. Edits use size: "auto" to preserve the source ratio; first generation can use an explicitly selected size.

Data visibility

Playground editor state remains personal. Generated module work products are now registered in the workspace-global media library, using authenticated asset references without public tokens. The library indexes existing generated versions from saved Playground histories and automatically registers new generation/edit results. Temporary masks, reference uploads, and personal chat-only generations remain outside the library.

Agent access

The native agent uses the existing images_generate and images_edit tools for image work. Playground-specific state tools are intentionally not shipped in this version, and native tool arguments are unchanged. Agent image work does not manage the playground's canvas state or version history.

Touch fallback

Long-press the image for 500 ms to open the point editor. The bottom global prompt bar stays available on touch devices as well.

Limits

• Up to 4 reference images for generation and 3 extra references for edits.
• Up to 20 MB per upload.
• Up to 80 versions retained in playground state.

Testing

Open Modules -> Image Playground. Generate an image with and without references, apply a global change, and try right-click and touch long-press point edits with the mask enabled and disabled. Check undo, redo, thumbnail restore, and persistence after reopening the module. Verify another user's state and image assets remain inaccessible.

Native Apple editor

The iPhone, iPad, and macOS apps include a native SwiftUI Image Playground entry when the module is installed. All Apple clients start with the same thumbnail gallery and New image tile; opening a tile shows the full editor with a Back to gallery action. The editor uses the same personal state endpoint and generated-image assets as the web module: model/provider selection, generation, global edits, reference imports, version selection, undo/redo, and image export work across clients. The native section publishes /modules/image-playground as its page route.

Enable Edit region to paint an edit mask. On iPad, PencilKit captures Apple Pencil strokes; Draw with finger optionally enables finger input. On Mac, drag with the pointer. The brush width and clear-selection controls apply to the mask. Marks select the area to change; they are not drawn onto the output image. The native editor converts the marks into transparent regions of a source-sized PNG and submits mask_image_id through the existing edit endpoint. Provider-specific mask behavior and the 4096-pixel/16.7-megapixel limit described above still apply. Mask drawing is temporary and clears when switching images.

Native errors remain visible with a retry action. If saving history fails after generation, retry saves the returned asset without generating again. Do not navigate away before an unsaved-history error has been resolved. State is shared with the web editor using the existing last-write-wins contract; simultaneous edits in multiple clients are not merged.

Native chat/live page context includes view (gallery or editor) and the active image_id and prompt only while the editor is open. Returning to the gallery clears the active image context. Existing images_generate and images_edit tools remain available; agents do not operate the Pencil canvas or modify playground version history. This is the same intentional boundary as the web module.

Apple verification: build macOS and iOS Simulator targets, run ImagePlaygroundTests, then check generation/import, references, region edits, history restoration, and export against an authenticated instance. Physical Pencil pressure and palm rejection must be checked on an iPad with Apple Pencil.

Shared media library

The Media library action opens generated images and videos organized into folders and nested subfolders. Playground output defaults to Images/Image Playground; Social Media output defaults to Images/Social Media; completed Video Studio exports and scene clips appear under Videos/Video Studio. Folder names are library organization, not physical filesystem moves. Original source files stay in place, and moving an item never breaks existing project or history references.

The web editor, iOS, and macOS can select a library image and import it as an editable source. Generated results are automatically registered through library_source: "image-playground" on the generate/edit APIs. Imported originals and temporary masks are not registered as generated output. Existing saved history is indexed when the shared library is browsed; historical images no longer referenced by saved history are not automatically made shared.

Social Media can attach library images or videos to a draft. Livestream can import Video Studio output into its mounted media folder, then queue it through the existing explicit queue action. Neither selection nor folder organization publishes anything. File Explorer and Video Studio expose the same folder browser. The native Livestream and File Explorer surfaces use the existing module/web routes; there is no separate SwiftUI Livestream or File Explorer implementation.

Agents use media_library_list, media_library_update, and social_media_attach_media with source: "media_library" and asset_id. Personal chat generations can be explicitly shared using media_library_update with action: "share_image" and the generated image ID. List results are authenticated URLs, not public distribution links.