World Model Scenario Explorer

A gallery of 300 ten-second driving scenarios from NVIDIA’s Cosmos-Drive-Dreams, laid out on a synthetic grid and all playing simultaneously on one looping clock. Each cell is a complete scene — HD-map lanes and crosswalks, the ego trajectory, and every tracked vehicle and pedestrian as an oriented 3D box — streamed from four cross-scenario tile archives rather than 300 separate ones, which is what makes hundreds of worlds animating at once affordable in a browser.

The premise is that the vector scene is the authoritative artifact and the world model supplies one photoreal manifestation of it. Selecting a world in the gallery flies the camera into its cell and plays the Cosmos-generated video for that exact scenario locked to the same playhead driving the geometry — scrub the timeline and the video follows frame for frame. Because the generated corpus renders each scenario under a different condition, the grid reads as a mosaic of weather: filter to every rainy world, every snowy one, or the handful of scenarios that exist in more than one generated reality at once.

Built by cosmos_drive_dreams.py, which pulls the per-clip label tars from Hugging Face, rebases every clip onto a shared epoch and its own grid cell (STT tiles are geographic, so each scenario’s local metre frame is anchored to a cell on an equatorial lattice with the basemap hidden), and streams the generated videos straight out of the dataset’s split tar archive without ever storing the 40 GB part. Three "hero" scenarios also carry their full LiDAR sweep, streamed only when you fly into them.

python stt:scripts/data-generation/cosmos_drive_dreams.py   # ~300 scenarios + videos

Phased and resumable (index → videos → select → download → transform → build → sidecar → cleanup). The videos phase streams one part of the generated-video tar and keeps only the MP4s it wants; scenario selection then follows video coverage. Open the gallery at /worlds.

Full recipes for every dataset live in the data-generation guide.