74 lines
2.1 KiB
Markdown
74 lines
2.1 KiB
Markdown
# ComfyUI with Burn Integration
|
|
|
|
This project demonstrates the integration of stable-diffusion-burn with the comfyui-rs framework to enable image generation capabilities using Burn tensor operations with GPU acceleration.
|
|
|
|
## Features
|
|
|
|
- CLI interface for image generation using stable-diffusion-burn
|
|
- Integration with Burn tensor operations
|
|
- Support for different backends (CPU, GPU)
|
|
- Workflow execution with Burn tensor operations
|
|
|
|
## Requirements
|
|
|
|
- Rust 1.70+
|
|
- Cargo
|
|
|
|
## Building
|
|
|
|
### For CPU/GPU (default)
|
|
```bash
|
|
# Build debug version
|
|
cargo build
|
|
|
|
# Build release version
|
|
cargo build --release
|
|
```
|
|
|
|
### For Vulkan GPU Acceleration (requires Vulkan drivers)
|
|
```bash
|
|
# Enable wgpu-backend feature to use Vulkan
|
|
cargo build --features wgpu-backend
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Show model info
|
|
```bash
|
|
./target/debug/comfyui-cli info --model-path /path/to/SDv1-4.mpk
|
|
```
|
|
|
|
### Generate image
|
|
```bash
|
|
./target/debug/comfyui-cli generate \
|
|
--model-path /path/to/SDv1-4.mpk \
|
|
--prompt "a beautiful sunset" \
|
|
--output ./output.png \
|
|
--steps 20 \
|
|
--device cpu
|
|
```
|
|
|
|
## Integration Details
|
|
|
|
This implementation demonstrates how to integrate with stable-diffusion-burn:
|
|
|
|
1. **Model Loading**: Uses the stable-diffusion-burn framework to load model files
|
|
2. **Device Management**: Supports different backends (CPU, GPU, etc.) via Burn
|
|
3. **Tensor Operations**: Implements actual Burn tensor operations for image generation
|
|
4. **Workflow Execution**: Executes workflows using Burn tensor operations
|
|
|
|
## PyTorch Dependency Note
|
|
|
|
**Important**: When using the `wgpu-backend` feature for Vulkan GPU acceleration, PyTorch is NOT required. The integration uses:
|
|
- `burn-wgpu` for Vulkan acceleration (no PyTorch needed)
|
|
- `burn-tch` only when using CPU/CUDA/MPS backends (requires PyTorch)
|
|
|
|
The default build works without PyTorch dependencies, and the wgpu-backend feature enables Vulkan support without requiring PyTorch.
|
|
|
|
## Future Work
|
|
|
|
- Implement actual stable-diffusion-burn integration
|
|
- Add proper image saving functionality
|
|
- Add more comprehensive CLI commands and options
|
|
- Optimize memory usage for large models
|
|
- Add configuration options for different parameters |