3sixty2cubemap/wiki/GPU.md

GPU

This project uses WebGPU via the wgpu crate, so the same code runs on Vulkan, Metal, DX12, and WebGPU/wasm.

GPU selection (all stages)

By default the pipeline asks wgpu for the discrete (high-performance) adapter and logs which one it picked:

[gpu] using AMD Radeon RX 9070 XT (RADV GFX1201) (DiscreteGpu) via Vulkan

If you see an integrated GPU or a software (llvmpipe) adapter, force the right backend/device. The request_adapter fallback enumerates adapters and prefers a discrete GPU. On dual-GPU laptops set WGPU_POWER_PREF / run under the discrete-GPU driver profile if wgpu still picks the iGPU.

YOLO masks on the GPU (ONNX Runtime → MIGraphX)

The masks stage runs YOLO through ONNX Runtime, loaded dynamically from the system libonnxruntime (ORT_DYLIB_PATH) rather than a downloaded CPU-only build. On AMD/ROCm systems that system lib is built with the MIGraphX execution provider, so:

# GPU inference on an AMD GPU
pipeline_360 masks --input-dir 03_cubemap/images --output-dir 03_cubemap/masks \
    --mask-model yolo11x-seg.onnx --mask-device rocm --mask-imgsz 1280

# CPU (no ROCm/MIGraphX runtime needed)
pipeline_360 masks --input-dir 03_cubemap/images --output-dir 03_cubemap/masks \
    --mask-model yolo11x-seg.onnx --mask-device cpu

• If the GPU EP can't be registered, the stage falls back to CPU and warns.
• The binary is built with --features masks by default.
• On your system the GPU-enabled libonnxruntime must be findable — the build uses load-dynamic, so it dlopens libonnxruntime.so (or whatever ORT_DYLIB_PATH points at) at runtime, and ROCm must be installed.

Compiled-model cache (important for first run)

MIGraphX compiles the ONNX graph to GPU kernels on the first run, which is slow (minutes for a large model like yolo11x-seg on a new GPU). It caches the compiled .mxr and loads it on later runs.

The pipeline auto-sets ORT_MIGRAPHX_CACHE_PATH to ~/.cache/pipeline_360/migraphx (override with --mask-ep-cache <dir>), so:

• first run with --mask-device rocm → compiles + saves (slow),
• every later run → loads the cached model in seconds.

Env vars (useful)

Variable	Meaning
ORT_DYLIB_PATH	path to the libonnxruntime to dlopen at runtime
ORT_MIGRAPHX_CACHE_PATH	MIGraphX compiled-model cache dir (auto-set)
ORT_MIGRAPHX_MODEL_CACHE_PATH	alias some MIGraphX builds read

Troubleshooting

• [gpu] using llvmpipe / iGPU — set WGPU_POWER_PREF=high or switch to the discrete GPU.
• "not a segmentation model" — you exported the detection model; export the seg one (yolo11x-seg.pt), not yolo11x.onnx.
• MIGraphX crash at inference with an empty .mxr path — the cache env var was unset. The pipeline sets it automatically; if you run via an external launcher that strips env vars, set ORT_MIGRAPHX_CACHE_PATH yourself.
• First run "very slow" — that's the one-time MIGraphX kernel compilation. Let it finish; later runs are fast.
• CPU-only ONNX Runtime — if you overrode ORT_DYLIB_PATH to a CPU build, the MIGraphX EP won't be available and it falls back to CPU.