ComfyUI-Rust/README.md

ComfyUI-like Framework - Rust & React Implementation with ROCm Support for RX 9070 XT

Project Overview

This project implements an AI image/video generation tool inspired by the node-based workflow editor of Stable Diffusion Web UI. Built as a modern web application using:

• Backend: Pure Rust (Actix-web framework)
• Frontend: Native Rust frontend using egui (replacing previous React/TypeScript frontend)
• New Frontend Alternative: Native Rust frontend using egui
• GPU Acceleration: ROCm integration for AMD RX 9070 XT GPU or Vulkan GPU support

Key Features:

1. ✅ REST API for model inference requests
2. 🔄 ROCm integration on AMD RX 9070 XT GPU
3. 🧼 Vulkan GPU backend support (optional)
4. ⚙️ Task queue system using Tokio async runtime (Rayon parallelism)
5. 💾 File upload/download support with session management

Architecture Design:

┌─────────────────────┐    WebSocket     ┌──────────────────┐
│  Rust Frontend      │◄═══► progress   │ Rust Backend API ├─▶ ROCm GPU (RX9070XT)
│ - Node Graph Editor |                    ├── Inference Queue|
│ - Workflow Builder  |                      ◄──── Models      |
╰─────────────────────╯    REST/JSON     └────────┬──■═══►
                                           │    
                                       Session/JWT Auth   

Setup Instructions

Prerequisites:

• Rust toolchain (cargo)
• AMD ROCm installed for RX 9070 XT GPU acceleration
• Vulkan drivers installed for Vulkan GPU support (optional)

# Install dependencies if needed on Linux/AMD systems:

sudo apt-get update && sudo apt install -y build-essential cmake ninja-build libopenblas-dev vulkan-tools

ROCm installation (run as user with appropriate permissions):
# For Ubuntu/Debian systems:
sudo apt install rocm-dev rocm-utils

# For other systems, follow AMD's official ROCm installation guide:
# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/

# Verify ROCm installation:
rocminfo    # Should show your GPU info including "gfx900" or similar architecture

# Verify Vulkan installation (if using Vulkan backend):
vulkaninfo  # Should show Vulkan-compatible GPUs

Backend Setup (Rust):

cd backend  
cargo build --release  # For production builds, use release mode for better performance on AMD GPUs  

# Run the backend server with ROCm (default):
./target/release/comfyui-backend-server [port]

# Run the backend server with Vulkan backend:
GPU_BACKEND=vulkan ./target/release/comfyui-backend-server [port]

Backend runs at: http://localhost:[PORT]
API endpoints available after starting.

New Rust Frontend Setup (egui):

cd rust-frontend  
cargo run  # Run the egui-based frontend

# Build for release:
cargo build --release

Project Structure Overview:

Rust Backend (backend/src):

- src/main.rs            # Entry point & server configuration  
  - Actix-web app setup, CORS middleware for frontend access
  
src/
├── api/mod.rs          # REST API endpoint handlers (inference requests)
│   POST /api/infer     # Start inference on ROCm GPU

 ├── models/           # ML model loading and management
 │    └── stable_diffusion_loader.cpp  

 ├─ queue_service       # Task Queue using Tokio + Rayon for parallel tasks  
  - Parallel task scheduling across available CPU cores & AMD threads
  
├── rocminfo.rs        # ROCm GPU detection on RX9070XT hardware 
│                      │
 └──── session_manager   JWT/Session authentication middleware  

Rust Frontend (rust-frontend/src):

src/
├── main.rs            # Main application entry point
├── node_editor.rs     # Node-based workflow editor implementation
├── node_panel.rs      # Panel for selecting and managing nodes  
├── preview_pane.rs    # Preview pane for image results
└── api_client.rs      # Backend API communication layer

ROCm Integration Notes:

Key Components:

• tokio async runtime: Handles concurrent inference tasks efficiently
  • Parallelism configured based on AMD GPU thread count (RX9070XT)

// Example configuration from backend/src/config.rs

pub struct Config {
    pub gpu_backend_config = RocmConfig { // ROCm detection for RX900 series GPUs
    

Development Workflow:

1. Model Preparation:

   • Download/prepare Stable Diffusion checkpoints (.safetensors)

2. Backend API Testing:

curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
--data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'

3. Frontend Usage:

   • Run the egui frontend with cargo run or cargo build --release

4. ROCm GPU Acceleration is automatically detected and used when available.

5. Vulkan Backend Usage (optional): To use Vulkan backend instead of ROCm, build with the feature flag:

   cargo build --features vulkan-backend
   

   Then run with:

   GPU_BACKEND=vulkan ./target/release/comfyui-backend-server
   

   The /system-info endpoint will now show the selected backend.

API Endpoints:

Backend REST API

• GET /health - Health check endpoint
• GET /system-info - Get system and GPU information
• POST /infer - Start a new inference task
• GET /models - List all available models
• GET /tasks/{task_id} - Get status of specific task
• GET /tasks - List all tasks

Example Request:

curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
  --data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'

Troubleshooting:

Common Issues with Ryzen/AMD Setup:

1. Permission denied accessing /dev/kfd → Add user to render, video groups

sudo usermod -aG render,audio $USER && sudo gpasswd --add $(whoami) audio

2. ROCm not detected: Check AMD driver version for RX9070XT:

rocminfo | grep gfx900 # Should show architecture detection

```bash
# Reboot after installing/updating drivers  
sudo reboot  


## License & Contributing:

This project follows open-source best practices with community contributions welcome.

---

**Built specifically to leverage AMD RX 9070 XT GPU capabilities through ROCm framework for accelerated AI inference.**