SHA256
76 lines
3.2 KiB
Markdown
76 lines
3.2 KiB
Markdown
# GPU
|
|
|
|
This project uses **WebGPU** via the `wgpu` crate, so the same code runs on
|
|
Vulkan, Metal, DX12, and WebGPU/wasm.
|
|
|
|
## GPU selection (all stages)
|
|
|
|
By default the pipeline asks wgpu for the **discrete** (high-performance)
|
|
adapter and logs which one it picked:
|
|
|
|
```
|
|
[gpu] using AMD Radeon RX 9070 XT (RADV GFX1201) (DiscreteGpu) via Vulkan
|
|
```
|
|
|
|
If you see an integrated GPU or a software (llvmpipe) adapter, force the right
|
|
backend/device. The `request_adapter` fallback enumerates adapters and prefers
|
|
a discrete GPU. On dual-GPU laptops set `WGPU_POWER_PREF` / run under the
|
|
discrete-GPU driver profile if wgpu still picks the iGPU.
|
|
|
|
## YOLO masks on the GPU (ONNX Runtime → MIGraphX)
|
|
|
|
The masks stage runs YOLO through **ONNX Runtime**, loaded **dynamically from
|
|
the system libonnxruntime** (`ORT_DYLIB_PATH`) rather than a downloaded
|
|
CPU-only build. On AMD/ROCm systems that system lib is built with the
|
|
**MIGraphX** execution provider, so:
|
|
|
|
```bash
|
|
# GPU inference on an AMD GPU
|
|
pipeline_360 masks --input-dir 03_cubemap/images --output-dir 03_cubemap/masks \
|
|
--mask-model yolo11x-seg.onnx --mask-device rocm --mask-imgsz 1280
|
|
|
|
# CPU (no ROCm/MIGraphX runtime needed)
|
|
pipeline_360 masks --input-dir 03_cubemap/images --output-dir 03_cubemap/masks \
|
|
--mask-model yolo11x-seg.onnx --mask-device cpu
|
|
```
|
|
|
|
- If the GPU EP can't be registered, the stage **falls back to CPU** and warns.
|
|
- The binary is built with `--features masks` by default.
|
|
- On your system the GPU-enabled `libonnxruntime` must be findable — the build
|
|
uses `load-dynamic`, so it `dlopen`s `libonnxruntime.so` (or whatever
|
|
`ORT_DYLIB_PATH` points at) at runtime, and ROCm must be installed.
|
|
|
|
### Compiled-model cache (important for first run)
|
|
|
|
MIGraphX compiles the ONNX graph to GPU kernels on the **first run**, which is
|
|
slow (minutes for a large model like `yolo11x-seg` on a new GPU). It caches the
|
|
compiled `.mxr` and loads it on later runs.
|
|
|
|
The pipeline **auto-sets `ORT_MIGRAPHX_CACHE_PATH`** to
|
|
`~/.cache/pipeline_360/migraphx` (override with `--mask-ep-cache <dir>`), so:
|
|
|
|
- first run with `--mask-device rocm` → compiles + saves (slow),
|
|
- every later run → loads the cached model in seconds.
|
|
|
|
### Env vars (useful)
|
|
|
|
| Variable | Meaning |
|
|
| --- | --- |
|
|
| `ORT_DYLIB_PATH` | path to the libonnxruntime to dlopen at runtime |
|
|
| `ORT_MIGRAPHX_CACHE_PATH` | MIGraphX compiled-model cache dir (auto-set) |
|
|
| `ORT_MIGRAPHX_MODEL_CACHE_PATH` | alias some MIGraphX builds read |
|
|
|
|
## Troubleshooting
|
|
|
|
- **`[gpu] using llvmpipe / iGPU`** — set `WGPU_POWER_PREF=high` or switch to
|
|
the discrete GPU.
|
|
- **"not a segmentation model"** — you exported the detection model; export the
|
|
**seg** one (`yolo11x-seg.pt`), not `yolo11x.onnx`.
|
|
- **MIGraphX crash at inference with an empty `.mxr` path** — the cache env var
|
|
was unset. The pipeline sets it automatically; if you run via an external
|
|
launcher that strips env vars, set `ORT_MIGRAPHX_CACHE_PATH` yourself.
|
|
- **First run "very slow"** — that's the one-time MIGraphX kernel compilation.
|
|
Let it finish; later runs are fast.
|
|
- **CPU-only ONNX Runtime** — if you overrode `ORT_DYLIB_PATH` to a CPU build,
|
|
the MIGraphX EP won't be available and it falls back to CPU.
|