ComfyUI-Rust/README.md

# ComfyUI-like Framework - Rust & React Implementation with ROCm Support for RX 9070 XT

## Project Overview
This project implements an AI image/video generation tool inspired by the node-based workflow editor of Stable Diffusion Web UI. Built as a modern web application using:

- **Backend**: Pure Rust (Actix-web framework)
- **Frontend**: React + TypeScript with Node graph visualization (currently having TypeScript version conflict) 
- **New Frontend Alternative**: Native Rust frontend using egui
- **GPU Acceleration**: ROCm integration for AMD RX 9070 XT GPU

### Key Features:
1. ✅ REST API for model inference requests
2. 🔄 ROCm integration on AMD RX 9070 XT GPU  
3. ⚙️ Task queue system using Tokio async runtime (Rayon parallelism)
4. 💾 File upload/download support with session management

## Architecture Design:

```
┌─────────────────────┐    WebSocket     ┌──────────────────┐
│  React Web Frontend │◄═══► progress   │ Rust Backend API ├─▶ ROCm GPU (RX9070XT)
│ - Node Graph Editor |                    ├── Inference Queue|
│ - Workflow Builder  |                      ◄──── Models      |
╰─────────────────────╯    REST/JSON     └────────┬──■═══►
                                           │    
                                       Session/JWT Auth   
```

## Setup Instructions

### Prerequisites:
- Rust toolchain (cargo)
- Node.js & npm/yarn/pnpm  
- AMD ROCm installed for RX 9070 XT GPU acceleration  

```bash
# Install dependencies if needed on Linux/AMD systems:

sudo apt-get update && sudo apt install -y build-essential cmake ninja-build libopenblas-dev

ROCm installation (run as user with appropriate permissions):
wget https://repo.radeon.com/amd-install/latest/install.sh  
chmod +x amd-install/install.sh
./amd-install/install.sh --no-dkms  # Skip DKMS if AMD driver is already installed system-wide for RX9070XT  

# Verify ROCm installation:
rocminfo    # Should show your GPU info including "gfx900" or similar architecture

```

### Backend Setup (Rust):

```bash
cd backend  
cargo build --release  # For production builds, use release mode for better performance on AMD GPUs  

# Run the backend server:
./target/release/comfyui-backend-server [port]

Backend runs at: http://localhost:[PORT]
API endpoints available after starting.
```

### Frontend Setup (React - currently has TypeScript conflict):

```bash
cd frontend  
npm install  # Install dependencies

Run dev mode with hot reload and AMD GPU preview support:
yarn start  

# Production build for deployment on ROCm systems: 
RUST_AMD_ROCM_PATH=/usr/local/AMDROCmlib yarn run prod-build && npm run serve
```

### New Rust Frontend Setup (egui):

```bash
cd rust-frontend  
cargo run  # Run the egui-based frontend

# Build for release:
cargo build --release
```

## Project Structure Overview:

### Rust Backend (`backend/src`):
```rust
- src/main.rs            # Entry point & server configuration  
  - Actix-web app setup, CORS middleware for frontend access
  
src/
├── api/mod.rs          # REST API endpoint handlers (inference requests)
│   POST /api/infer     # Start inference on ROCm GPU

 ├── models/           # ML model loading and management
 │    └── stable_diffusion_loader.cpp  

 ├─ queue_service       # Task Queue using Tokio + Rayon for parallel tasks  
  - Parallel task scheduling across available CPU cores & AMD threads
  
├── rocminfo.rs        # ROCm GPU detection on RX9070XT hardware 
│                      │
 └──── session_manager   JWT/Session authentication middleware  

```

### Original Frontend Web App (`frontend/src`):
```typescript/react  
src/
 ├─ components/node-editor.tsx    // Node-based workflow editor (graph canvas)
  - Drag & drop node positioning on AMD GPU-aware preview panel
  
├── store/graph-store.js         # Redux state management for nodes
│                              │

 └──── utils/api-client          API calls to Rust backend server  

```

### New Rust Frontend (`rust-frontend/src`):
```rust
src/
├── main.rs            # Main application entry point
├── node_editor.rs     # Node-based workflow editor implementation
├── node_panel.rs      # Panel for selecting and managing nodes  
├── preview_pane.rs    # Preview pane for image results
└── api_client.rs      # Backend API communication layer

```

## ROCm Integration Notes:

### Key Components:
- `tokio` async runtime: Handles concurrent inference tasks efficiently  
  - Parallelism configured based on AMD GPU thread count (RX9070XT)
 
```rust
// Example configuration from backend/src/config.rs

pub struct Config {
    pub gpu_backend_config = RocmConfig { // ROCm detection for RX900 series GPUs
    
```

## Development Workflow:

1. **Model Preparation**:
   - Download/prepare Stable Diffusion checkpoints (.safetensors)
   
2. **Backend API Testing**:  
```bash
curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
--data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'
```

3. **Web Frontend Usage**:
   - Open React app in browser (http://localhost:[frontend_port])
   
4. ROCm GPU Acceleration is automatically detected and used when available.

## API Endpoints:

### Backend REST API
- `GET /health` - Health check endpoint  
- `GET /system-info` - Get system and GPU information
- `POST /infer` - Start a new inference task
- `GET /models` - List all available models
- `GET /tasks/{task_id}` - Get status of specific task
- `GET /tasks` - List all tasks

### Example Request:
```bash
curl http://localhost:8080/api/infer \
  --header "Content-Type: application/json" \ 
  --data '{"prompt":"A futuristic cityscape","negative_prompt":"","guidance_scale":7,"steps":20}'
```

## Troubleshooting:

### Common Issues with Ryzen/AMD Setup:
1. **Permission denied accessing `/dev/kfd`** → Add user to `render`, video groups  
```bash
sudo usermod -aG render,audio $USER && sudo gpasswd --add $(whoami) audio

```
   
2. ROCm not detected: Check AMD driver version for RX9070XT:
   ```sh 
rocminfo | grep gfx900  # Should show architecture detection  

 ```
```bash
# Reboot after installing/updating drivers  
sudo reboot  


## License & Contributing:

This project follows open-source best practices with community contributions welcome.

---

**Built specifically to leverage AMD RX9070 XT GPU capabilities through ROCm framework for accelerated AI inference.**