ComfyUI-Rust/README.md

# ComfyUI-like Framework - Rust & React Implementation with ROCm Support for RX 9070 XT

## Project Overview
This project implements an AI image/video generation tool inspired by the node-based workflow editor of Stable Diffusion Web UI. Built as a modern web application using:

- **Backend**: Pure Rust (Actix-web framework)
- **Frontend**: Native Rust frontend using egui (replacing previous React/TypeScript frontend)
- **New Frontend Alternative**: Native Rust frontend using egui
- **GPU Acceleration**: ROCm integration for AMD RX 9070 XT GPU or Vulkan GPU support

### Key Features:
1. ✅ REST API for model inference requests
2. 🔄 ROCm integration on AMD RX 9070 XT GPU  
3. 🧼 Vulkan GPU backend support (optional)
4. ⚙️ Task queue system using Tokio async runtime (Rayon parallelism)
5. 💾 File upload/download support with session management

## Architecture Design:

```
┌─────────────────────┐    WebSocket     ┌──────────────────┐
│  Rust Frontend      │◄═══► progress   │ Rust Backend API ├─▶ ROCm GPU (RX9070XT)
│ - Node Graph Editor |                    ├── Inference Queue|
│ - Workflow Builder  |                      ◄──── Models      |
╰─────────────────────╯    REST/JSON     └────────┬──■═══►
                                           │    
                                       Session/JWT Auth   
```

## Setup Instructions

### Prerequisites:
- Rust toolchain (cargo)
- AMD ROCm installed for RX 9070 XT GPU acceleration  
- Vulkan drivers installed for Vulkan GPU support (optional)

```bash
# Install dependencies if needed on Linux/AMD systems:

sudo apt-get update && sudo apt install -y build-essential cmake ninja-build libopenblas-dev vulkan-tools

ROCm installation (run as user with appropriate permissions):
# For Ubuntu/Debian systems:
sudo apt install rocm-dev rocm-utils

# For other systems, follow AMD's official ROCm installation guide:
# https://rocm.docs.amd.com/projects/install-on-linux/en/latest/

# Verify ROCm installation:
rocminfo    # Should show your GPU info including "gfx900" or similar architecture

# Verify Vulkan installation (if using Vulkan backend):
vulkaninfo  # Should show Vulkan-compatible GPUs

```

### Backend Setup (Rust):

```bash
cd backend  
cargo build --release  # For production builds, use release mode for better performance on AMD GPUs  

# Run the backend server with ROCm (default):
./target/release/comfyui-backend-server [port]

# Run the backend server with Vulkan backend:
GPU_BACKEND=vulkan ./target/release/comfyui-backend-server [port]

Backend runs at: http://localhost:[PORT]
API endpoints available after starting.
```

### New Rust Frontend Setup (egui):

```bash
cd rust-frontend  
cargo run  # Run the egui-based frontend

# Build for release:
cargo build --release
```

## Project Structure Overview:

### Rust Backend (`backend/src`):
```rust
- src/main.rs            # Entry point & server configuration  
  - Actix-web app setup, CORS middleware for frontend access
  
src/
├── api/mod.rs          # REST API endpoint handlers (inference requests)
│   POST /api/infer     # Start inference on ROCm GPU

 ├── models/           # ML model loading and management
 │    └── stable_diffusion_loader.cpp  

 ├─ queue_service       # Task Queue using Tokio + Rayon for parallel tasks  
  - Parallel task scheduling across available CPU cores & AMD threads
  
├── rocminfo.rs        # ROCm GPU detection on RX9070XT hardware 
│                      │
 └──── session_manager   JWT/Session authentication middleware  

```

### Rust Frontend (`rust-frontend/src`):
```rust
src/
├── main.rs            # Main application entry point
├── node_editor.rs     # Node-based workflow editor implementation
├── node_panel.rs      # Panel for selecting and managing nodes  
├── preview_pane.rs    # Preview pane for image results
└── api_client.rs      # Backend API communication layer

```

## ROCm Integration Notes:

### Key Components:
- `tokio` async runtime: Handles concurrent inference tasks efficiently  
  - Parallelism configured based on AMD GPU thread count (RX9070XT)
 
```rust
// Example configuration from backend/src/config.rs

pub struct Config {
    pub gpu_backend_config = RocmConfig { // ROCm detection for RX900 series GPUs
    
```

## Development Workflow:

1. **Model Preparation**:
   - Download/prepare Stable Diffusion checkpoints (.safetensors)
   
2. **Backend API Testing**:  
```bash
curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
--data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'
```

3. **Frontend Usage**:
   - Run the egui frontend with `cargo run` or `cargo build --release`
   
4. ROCm GPU Acceleration is automatically detected and used when available.
   
5. **Vulkan Backend Usage** (optional):
   To use Vulkan backend instead of ROCm, build with the feature flag:
   ```bash
   cargo build --features vulkan-backend
   ```
   Then run with:
   ```bash
   GPU_BACKEND=vulkan ./target/release/comfyui-backend-server
   ```
   
   The `/system-info` endpoint will now show the selected backend.

## API Endpoints:

### Backend REST API
- `GET /health` - Health check endpoint  
- `GET /system-info` - Get system and GPU information
- `POST /infer` - Start a new inference task
- `GET /models` - List all available models
- `GET /tasks/{task_id}` - Get status of specific task
- `GET /tasks` - List all tasks

### Example Request:
```bash
curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
  --data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'
```

## Troubleshooting:

### Common Issues with Ryzen/AMD Setup:
1. **Permission denied accessing `/dev/kfd`** → Add user to `render`, video groups  
```bash
sudo usermod -aG render,audio $USER && sudo gpasswd --add $(whoami) audio

```
   
2. ROCm not detected: Check AMD driver version for RX9070XT:
   ```sh 
rocminfo | grep gfx900  # Should show architecture detection  

 ```
```bash
# Reboot after installing/updating drivers  
sudo reboot  


## License & Contributing:

This project follows open-source best practices with community contributions welcome.

---

**Built specifically to leverage AMD RX 9070 XT GPU capabilities through ROCm framework for accelerated AI inference.**