SHA256
66 lines
2.8 KiB
Markdown
66 lines
2.8 KiB
Markdown
# Architecture
|
|
|
|
## Source layout
|
|
|
|
```
|
|
Cargo.toml binary + lib; features: default=["masks"], gui
|
|
src/
|
|
main.rs CLI entry, clap parsing, `all` orchestration, GUI launch
|
|
lib.rs crate root, re-exports
|
|
project.rs project layout (01_frames … 04_colmap) + file gathering
|
|
logging.rs emit / emit_progress ([progress] line protocol)
|
|
error.rs error + cancellation types
|
|
math.rs face rotations, quaternions, intrinsics (pure std)
|
|
json.rs minimal JSON writer (pure std)
|
|
selftest.rs rig self-test (6 check groups)
|
|
gpu/
|
|
mod.rs GpuContext (device/queue), texture/readback helpers
|
|
shaders/ equirect_to_cubemap.wgsl, sharpness.wgsl, c2e.wgsl
|
|
stages/
|
|
extract.rs stage 0: GPU sharpness + selection + frame writes
|
|
masks.rs stage 1: YOLO segmentation (ONNX Runtime / MIGraphX)
|
|
cubemap.rs stage 2: GPU equirect → cubemap
|
|
rig.rs stage 3: rig config + nested tree
|
|
video/
|
|
mod.rs ffmpeg/ffprobe CLI decode → raw RGBA frames
|
|
gui.rs egui+wgpu GUI (feature `gui`)
|
|
offline_check/ pure-std fallback build of the rig core (no cargo/network)
|
|
```
|
|
|
|
## How a run flows
|
|
|
|
1. **CLI** parses the command; `all` builds a `GpuContext` once and runs each
|
|
stage in order, sharing the device/queue.
|
|
2. **Extract** pipes decoded frames from ffmpeg to the GPU `sharpness` kernel,
|
|
reads back tiny per-workgroup metrics, filters by quality, selects frames,
|
|
and writes them out.
|
|
3. **Cubemap** uploads each equirect frame and dispatches the
|
|
`equirect_to_cubemap` kernel once per face, reading back square faces.
|
|
4. **Masks** loads `yolo11x-seg.onnx` into ONNX Runtime (with the MIGraphX EP
|
|
when requested), runs each cube face through the network, and synthesizes a
|
|
binary mask (see [GPU](GPU.md)).
|
|
5. **Rig** scans images + masks, writes `rig_config.json`, and links the nested
|
|
COLMAP tree (symlink/hardlink/copy).
|
|
|
|
## Why Rust + WebGPU
|
|
|
|
- **Parallelism**: per-pixel/per-frame work maps naturally onto a compute
|
|
shader; `rayon` covers CPU-side orchestration.
|
|
- **Cross-vendor GPU**: one WGSL codebase → Vulkan / Metal / DX12 / WebGPU,
|
|
instead of per-vendor libraries.
|
|
- **Safety**: Rust's ownership model + wgpu's resource lifetimes avoid
|
|
use-after-free and data races.
|
|
- **Distribution**: a single binary; no Python environment.
|
|
|
|
Video **decode** stays on the CPU through ffmpeg because codecs aren't
|
|
something you hand-roll on a bindless GPU layer; everything after decode is GPU.
|
|
|
|
## Feature flags
|
|
|
|
| Feature | Effect |
|
|
| --- | --- |
|
|
| `masks` (default) | enable ONNX Runtime YOLO masks (`ort` + `ndarray`, `load-dynamic`, `migraphx`) |
|
|
| `gui` | build the native egui+wgpu GUI into the same binary |
|
|
|
|
`cargo build --no-default-features` gives a light build with no ONNX Runtime.
|