3sixty2cubemap/wiki/Architecture.md

Architecture

Source layout

Cargo.toml          binary + lib; features: default=["masks"], gui
src/
  main.rs           CLI entry, clap parsing, `all` orchestration, GUI launch
  lib.rs            crate root, re-exports
  project.rs        project layout (01_frames … 04_colmap) + file gathering
  logging.rs        emit / emit_progress ([progress] line protocol)
  error.rs          error + cancellation types
  math.rs           face rotations, quaternions, intrinsics (pure std)
  json.rs           minimal JSON writer (pure std)
  selftest.rs       rig self-test (6 check groups)
  gpu/
    mod.rs          GpuContext (device/queue), texture/readback helpers
    shaders/        equirect_to_cubemap.wgsl, sharpness.wgsl, c2e.wgsl
  stages/
    extract.rs      stage 0: GPU sharpness + selection + frame writes
    masks.rs        stage 1: YOLO segmentation (ONNX Runtime / MIGraphX)
    cubemap.rs      stage 2: GPU equirect → cubemap
    rig.rs          stage 3: rig config + nested tree
  video/
    mod.rs          ffmpeg/ffprobe CLI decode → raw RGBA frames
  gui.rs            egui+wgpu GUI (feature `gui`)
offline_check/      pure-std fallback build of the rig core (no cargo/network)

How a run flows

1. CLI parses the command; all builds a GpuContext once and runs each stage in order, sharing the device/queue.
2. Extract pipes decoded frames from ffmpeg to the GPU sharpness kernel, reads back tiny per-workgroup metrics, filters by quality, selects frames, and writes them out.
3. Cubemap uploads each equirect frame and dispatches the equirect_to_cubemap kernel once per face, reading back square faces.
4. Masks loads yolo11x-seg.onnx into ONNX Runtime (with the MIGraphX EP when requested), runs each cube face through the network, and synthesizes a binary mask (see GPU).
5. Rig scans images + masks, writes rig_config.json, and links the nested COLMAP tree (symlink/hardlink/copy).

Why Rust + WebGPU

• Parallelism: per-pixel/per-frame work maps naturally onto a compute shader; rayon covers CPU-side orchestration.
• Cross-vendor GPU: one WGSL codebase → Vulkan / Metal / DX12 / WebGPU, instead of per-vendor libraries.
• Safety: Rust's ownership model + wgpu's resource lifetimes avoid use-after-free and data races.
• Distribution: a single binary; no Python environment.

Video decode stays on the CPU through ffmpeg because codecs aren't something you hand-roll on a bindless GPU layer; everything after decode is GPU.

Feature flags

Feature	Effect
masks (default)	enable ONNX Runtime YOLO masks (ort + ndarray, load-dynamic, migraphx)
gui	build the native egui+wgpu GUI into the same binary

cargo build --no-default-features gives a light build with no ONNX Runtime.