[Paper] | [Project Page] | [WSMs] | [World Snapshot Graphics] | [Models & Data] | [Documentation] | [Organization] | [Live Demo]
ControlWorld is the interactive control panel of the World Snapshot ecosystem. It turns a model's directed visual-task graph into an executable interface: select a trained route, provide its image or text conditions, run the recommended checkpoint, and inspect the decoded result and live inference log.
The host remains model-independent. Each model carries its own interface, backend, requirements, and task definitions as a plugin; ControlWorld supplies one consistent shell for discovery, execution, visualization, and plugin management.
- Visual task selection. Select an input and output from the task graph, or click a trained edge directly.
- Multi-condition execution. Plugins may request image, text, audio, reference, or other conditions within one route.
- Model-aware deployment. A plugin can expose recommended checkpoints, task warnings, categorical refiners, and modality-specific controls.
- Decoded visual evidence. Non-RGB results can use the official WSG Reader visualization and value legend supplied by the model plugin.
- Live inference logs. Long model loading, preprocessing, sampling, decoding, and output generation remain visible in the interface.
- Shared 3D visualization. Plugins that require interactive scenes can attach to the common Viser service.
- Non-destructive plugin management. Uninstalling a plugin disables its tab without deleting its local folder; it remains available to install again.
git clone --recursive git@github.com:World-Snapshot/ControlWorld.git
cd ControlWorld
conda create -n WSM python=3.11 -y
conda activate WSM
pip install -r env/requirements.txt
python start_modern.pyControlWorld opens the main interface at http://localhost:8080. Its shared Viser service runs at http://localhost:8081.
The WSM environment is shared with WSM v1; ControlWorld does not require a second model environment.
For a remote machine or a custom port:
python start_modern.py --frontend-port 8080 --viser-port 8081 --no-browserThe complete ControlWorld interface can run on Hugging Face GPU hardware. The
Docker deployment under
deploy/huggingface_space exports ControlWorld,
the WSM plugin, and WSG Reader as a standalone public Space repository and
downloads the released WSM Heads and shared FLUX.2 backbone when it starts.
python deploy/huggingface_space/export_space.py \
--output /tmp/wsm-controlworld-spaceThe default paper-demo profile loads the recommended Heads used by the paper
demonstrations. selected and all profiles provide smaller or broader
deployments without changing the application code.
ControlWorld scans both direct plugins and one-level plugin collections under plugins/. The official WSMs repository is included as a submodule:
plugins/
├── WSMs/ # repository submodule / plugin collection
│ ├── WSM_v1/
│ │ └── webui/ # automatically discovered plugin
│ └── WSM_v2/ # discovered automatically when available
└── MyModel/
└── webui_controlworld/ # or webui/
├── index.js # frontend entry
├── config.json # optional plugin metadata/configuration
├── viser_backend.py # optional Viser/message backend
├── api_handler.py # optional API handler
└── requirements.txt # optional plugin dependencies
For an existing clone, initialize or update the collection with:
git submodule update --init --recursivePlugin interface files may live directly in a plugin root, webui_controlworld/, or webui/. A repository such as WSMs may contain multiple version directories; ControlWorld discovers each version independently without recursively treating ordinary model folders as plugins. For backward compatibility, backend.py, plugin_backend.py, and requirement.txt are also recognized. Duplicate plugin IDs are rejected so that one visible task tab always resolves to one implementation.
The Plugin Manager can:
- discover local plugin folders and symbolic links;
- enable or disable their tabs without deleting files;
- preserve the selected state in the browser;
- refresh the local registry after a plugin is added or updated;
- install supported GitHub or Hugging Face repositories from a URL.
WSM v1 is the principal reference plugin. Its WebUI exposes the WSG Task Graph, recommended Channel Heads, multi-condition inputs, checkpoint selection, true inference, target-specific visualization, selectable output artifacts, and a live log panel.
A route can be chosen from either direction: selecting input and output controls highlights the corresponding graph edge, while clicking a trained graph edge updates the input, output, and checkpoint controls. Dense and sparse routes share the same interface.
Browser
├── plugin tabs and Plugin Manager
├── task-specific controls
└── decoded outputs and logs
│ HTTP / WebSocket
▼
ControlWorld
├── FastAPI application and static frontend
├── plugin discovery and lifecycle
├── backend message routing
└── shared Viser service
│
▼
Model plugins
├── frontend entry
├── inference/backend worker
├── checkpoints and task registry
└── model-specific readers and visualizations
Core entry points:
start_modern.pystarts the FastAPI frontend, WebSocket routing, plugin scanner, and shared Viser service.viser_backend.pydiscovers plugin backends and routes Viser controls and plugin messages.plugins/plugin-loader.jshandles browser-side discovery, loading, ordering, and enabled state.plugins/plugin-system.jsrenders tabs and the Plugin Manager.static/style/defines the shared ControlWorld visual language inherited by plugins.
| RGB to Relative Depth | DWPose + Text + RGB Reference to RGB |
|---|---|
![]() |
![]() |
| Normal to Edge | Relative Depth + RGB to Normal |
![]() |
![]() |
These desktop captures retain the complete task graph, inputs, inference controls, and decoded output. The full set of 32 task demonstrations is available in static/images/user_demos.
Model plugins, interface improvements, bug reports, and documentation updates are welcome. See Contributing.md for the plugin interface and contribution workflow.
ControlWorld was created by Sen Fang and Dimitris N. Metaxas as part of the
World Snapshot project. Citation metadata is provided in
CITATION.cff.
The original ControlWorld host software is released under the MIT License. This license does not apply to model plugins, Git submodules, model weights, datasets, or third-party assets that carry their own licenses. In particular, the WSMs plugin remains governed by the license provided in the WSMs repository.



