Warning

This project is fully vibecoded and behind no line in this code is human soul.

pipeline_360

Convert equirectangular 360° videos into a COLMAP-ready cubemap camera-rig dataset — sharpest-frame extraction, movable-object masks, an equirect→cubemap projection, and a rig_config.json + nested file tree.

This is the Rust + WebGPU rewrite of the original Python pipeline (now in ../python, kept as legacy reference).

⚡ Status: fully vibecoded, and the rewrite is done. Everything described here is implemented and running on real 8K test footage — GPU sharpness, YOLO segmentation masks on the GPU, cubemap projection, and rig generation all work end-to-end. It should just work; if something doesn't, the Wiki and issue tracker are the place to look.

What it does

video(s) → 01_frames (sharpest, quality-filtered)
         → 03_cubemap/images (equirect → 6 cube faces, on GPU)
         → 03_cubemap/masks  (YOLO11x-seg, on GPU)
         → 04_colmap/{images,masks,rig_config.json} (nested tree for COLMAP)

Everything heavy is on the GPU via WebGPU (wgpu) — Vulkan / Metal / DX12, so it's vendor-neutral. YOLO masks use the system ONNX Runtime with the AMD MIGraphX provider for GPU inference (falls back to CPU). A native egui GUI ships in the same binary.

Build & run

cargo build                    # full pipeline (masks on by default)
cargo run -- rig --self-test   # quick sanity check

# single command, all four stages (cube faces → masks on the GPU):
cargo run -- all --project-dir out --video vid1.mp4 [vid2.mp4 …] \
    --mask-model yolo11x-seg.onnx --mask-device rocm

# native GUI (one binary, --features gui):
cargo build --features gui && ./target/debug/pipeline_360

Prerequisites: a Rust toolchain, ffmpeg/ffprobe on PATH, a GPU with a WebGPU backend, and an ONNX export of the segmentation model (yolo export model=yolo11x-seg.pt format=onnx imgsz=1280 dynamic=True).

Features

Extract — GPU sharpness/exposure/contrast scores + quality filtering, parallax (--min-motion) and bad-frame rejection, chunk or spread selection, multi-video, batched.
Masks — YOLO11x-seg on the cube-face images (no wasteful round-trip), COLMAP convention (black = excluded), GPU inference with auto cached compiled model (first run compiles, later runs are fast).
Cubemap — faithful port of py360convert.e2c, batched, skip/resume.
Rig — rig_config.json + nested {FACE}/{stem} tree, symlink/hardlink/copy.
GUI — native egui+wgpu, shells out to the CLI for the heavy work.

Project structure

Cargo.toml, main.rs      entry point, CLI, `all` orchestration
src/gpu/                 GpuContext + WGSL shaders
src/stages/              extract, masks, cubemap, rig
src/gui.rs               egui+wgpu GUI (feature `gui`)
wiki/                    the documentation (setup, usage, GPU, troubleshooting)

License

GPL-3.0-or-later — see LICENSE.

Wiki → wiki/Home.md — setup, CLI reference, pipeline internals, GPU/MIGraphX setup, troubleshooting.
Design notes → PLAN.md — the original rewrite plan.

3.2 KiB Raw Blame History