SHA256
48 lines
2.1 KiB
Markdown
48 lines
2.1 KiB
Markdown
# Pipeline
|
|
|
|
The rewrite follows the original Python design but reorders the stages so
|
|
**cubemap runs before masks**: masks run on the cube-face images directly,
|
|
which avoids a wasteful equirect→cubemap→equirect round-trip.
|
|
|
|
## Output layout
|
|
|
|
```
|
|
out/
|
|
01_frames/ selected sharpest equirect frames (multi-video → <stem> prefix)
|
|
02_masks/ (legacy; no longer written by `all`)
|
|
03_cubemap/
|
|
images/ {stem}_{FACE}.{ext} (equirect → 6 cube faces)
|
|
masks/ {stem}_{FACE}.{ext}.png (per-face masks)
|
|
04_colmap/
|
|
images/ {FACE}/{stem}.{ext} (nested COLMAP tree)
|
|
masks/ {FACE}/{stem}.{ext}.png
|
|
rig_config.json rig cameras / intrinsics
|
|
```
|
|
|
|
`FACE` ∈ `{F, R, B, L, U, D}`. The nested `{FACE}/…` tree is produced via
|
|
symlink / hardlink / copy (`--rig-link-mode`).
|
|
|
|
## Stage details
|
|
|
|
1. **Extract** — decodes the video(s) with ffmpeg, computes per-frame GPU
|
|
sharpness (variance-of-Laplacian), exposure, and contrast, optionally
|
|
filters out blurry / badly exposed / flat frames, checks inter-frame motion
|
|
(parallax), then selects the sharpest frame per chunk (or spreads them out).
|
|
Writes `_sharpness_metadata_<stem>.csv` so the analysis is reused on
|
|
subsequent runs.
|
|
2. **Cubemap** — projects each equirect frame to six square cube faces on the
|
|
GPU (a faithful port of `py360convert.e2c`), batched for throughput.
|
|
3. **Masks** — runs YOLO11x-seg on each cube face, synthesizes a binary mask
|
|
(COLMAP convention: **black = excluded**), and writes it next to the face
|
|
image. See [GPU](GPU.md).
|
|
4. **Rig** — scans the cubemap images + masks, writes `rig_config.json`
|
|
(PINHOLE intrinsics + per-face `cam_from_rig_rotation`), and links the
|
|
nested COLMAP tree.
|
|
|
|
## Naming / mask convention
|
|
|
|
- Masks are named `<image_filename>.png` (hence `frame_00001_F.png.png`).
|
|
- In a mask, **black pixels are excluded** (movable objects); white pixels are
|
|
kept (the static scene used for reconstruction). Masks mirror the image tree
|
|
so COLMAP attaches them to the right cameras.
|