DiffuGen Image Generator
STDIOYour AI art studio embedded in code, generating images using Flux and Stable Diffusion models.
Your AI art studio embedded in code, generating images using Flux and Stable Diffusion models.
Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.
⭐ New: Now includes OpenAPI server support and OpenWebUI OpenAPI Tools (OWUI Version 0.60.0 Required) integration for seamless image generation and display in chat interfaces! The OpenAPI is seperate from the MCP server and allowss for initigrations into your own projects!
DiffuGen is a powerful MCP-based image generation system that brings cutting-edge AI models directly into your development workflow. It seamlessly integrates both Flux models (Flux Schnell, Flux Dev) and Stable Diffusion variants (SDXL, SD3, SD1.5) into a unified interface, allowing you to leverage the unique strengths of each model family without switching tools. With comprehensive parameter control and multi-GPU support, DiffuGen scales from rapid concept sketches on modest hardware to production-quality visuals on high-performance systems.
Built on top of the highly optimized stable-diffusion.cpp implementation, DiffuGen offers exceptional performance even on modest hardware while maintaining high-quality output.
MCP (Model Context Protocol) is a protocol that enables LLMs (Large Language Models) to access custom tools and services. In simple terms, an MCP client (like Cursor, Windsurf, Roo Code, or Cline) can make requests to MCP servers to access tools that they provide.
DiffuGen functions as an MCP server that provides text-to-image generation capabilities. It implements the MCP protocol to allow compatible IDEs to send generation requests and receive generated images.
The server exposes two main tools:
generate_stable_diffusion_image: Generate with Stable Diffusion modelsgenerate_flux_image: Generate with Flux modelsDiffuGen consists of several key components:
The system works by:
stable-diffusion.cpp is a highly optimized C++ implementation of the Stable Diffusion algorithm. Compared to the Python reference implementation, it offers:
This allows DiffuGen to provide high-quality image generation with exceptional performance, even on modest hardware setups.
The easiest way to install DiffuGen is using the provided setup script:
git clone https://github.com/CLOUDWERX-DEV/diffugen.git cd DiffuGen chmod +x diffugen.sh chmod +x setup_diffugen.sh ./setup_diffugen.sh
Follow the interactive prompts to complete the installation.
The setup script will:
If you prefer to install manually, follow these steps:
git clone https://github.com/CLOUDWERX-DEV/diffugen.git cd DiffuGen git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp mkdir -p build && cd build
With CUDA:
cmake .. -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON make -j$(nproc) cd ../..
Without CUDA:
cmake .. -DCMAKE_BUILD_TYPE=Release make -j$(nproc) cd ../..
python3 -m venv diffugen_env source diffugen_env/bin/activate # On Windows: diffugen_env\Scripts\activate pip install -r requirements.txt
stable-diffusion.cpp/models/
├── ae.sft # VAE model
├── clip_l.safetensors # CLIP model
├── flux/
│ ├── flux1-schnell-q8_0.gguf # Flux Schnell model (default)
│ └── flux1-dev-q8_0.gguf # Flux Dev model
├── sd3-medium.safetensors # SD3 model
├── sdxl-1.0-base.safetensors # SDXL model
├── sdxl_vae-fp16-fix.safetensors # SDXL VAE
├── t5xxl_fp16.safetensors # T5 model
└── v1-5-pruned-emaonly.safetensors # SD1.5 model
You can download the models from the following sources:
# Create model directories mkdir -p stable-diffusion.cpp/models/flux # Flux models # Flux Schnell - Fast generation model (Q8 Quantized,requires t5xxl, clip-l, vae) curl -L https://huggingface.co/leejet/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-schnell-q8_0.gguf # Flux Dev - Development model with better quality (Q8 QUantized, requires t5xxl, clip-l, vae) curl -L https://huggingface.co/leejet/FLUX.1-dev-gguf/resolve/main/flux1-dev-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-dev-q8_0.gguf # Required models for Flux # T5XXL Text Encoder curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/t5xxl_fp16.safetensors -o stable-diffusion.cpp/models/t5xxl_fp16.safetensors # CLIP-L Text Encoder curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/clip_l.safetensors -o stable-diffusion.cpp/models/clip_l.safetensors # VAE for image decoding curl -L https://huggingface.co/pretentioushorsefly/flux-models/resolve/main/models/vae/ae.safetensors -o stable-diffusion.cpp/models/ae.sft # Stable Diffusion models # SDXL 1.0 Base Model (requires sdxl-vae) curl -L https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors -o stable-diffusion.cpp/models/sd_xl_base_1.0.safetensors # SDXL VAE (required for SDXL) curl -L https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae-fp16-fix.safetensors -o stable-diffusion.cpp/models/sdxl_vae-fp16-fix.safetensors # Stable Diffusion 1.5 (standalone) curl -L https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors -o stable-diffusion.cpp/models/v1-5-pruned-emaonly.safetensors # Stable Diffusion 3 Medium (standalone) curl -L https://huggingface.co/leo009/stable-diffusion-3-medium/resolve/main/sd3_medium_incl_clips_t5xxlfp16.safetensors -o stable-diffusion.cpp/models/sd3_medium_incl_clips_t5xxlfp16.safetensors
Note: Model download may take a long time depending on your internet connection. The SDXL model is approximately 6GB, SD3 is about 13GB, SD1.5 is around 4GB, and Flux models are 8-13GB each.
Set shell script as Executable
chmod +x diffugen.sh
Configuration Approach:
DiffuGen uses a single configuration file (diffugen.json) as the source of truth for all settings. The workflow is:
diffugen.json in the DiffuGen root directory with your desired settingssetup_diffugen.sh to automatically update paths in this filediffugen.json to your IDE's MCP configuration fileThe file contains all necessary settings:
{ "mcpServers": { "diffugen": { "command": "/home/cloudwerxlab/Desktop/Servers/MCP/Tools/DiffuGen/diffugen.sh", "args": [], "env": { "CUDA_VISIBLE_DEVICES": "0", "SD_CPP_PATH": "path/to/stable-diffusion.cpp", "default_model": "flux-schnell" }, "resources": { "models_dir": "path/to/stable-diffusion.cpp/models", "output_dir": "path/to/outputs", "vram_usage": "adaptive" }, "metadata": { "name": "DiffuGen", "version": "1.0", "description": "Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.", "author": "CLOUDWERX LAB", "homepage": "https://github.com/CLOUDWERX-DEV/diffugen", "usage": "Generate images using two primary methods:\n1. Standard generation: 'generate an image of [description]' with optional parameters:\n - model: Choose from flux-schnell (default), flux-dev, sdxl, sd3, sd15\n - dimensions: width and height (default: 512x512)\n - steps: Number of diffusion steps (default: 20, lower for faster generation)\n - cfg_scale: Guidance scale (default: 7.0, lower for more creative freedom)\n - seed: For reproducible results (-1 for random)\n - sampling_method: euler, euler_a (default), heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm\n - negative_prompt: Specify elements to avoid in the image\n2. Quick Flux generation: 'generate a flux image of [description]' for faster results with fewer steps (default: 4)" }, "cursorOptions": { "autoApprove": true, "category": "Image Generation", "icon": "🖼️", "displayName": "DiffuGen" }, "windsurfOptions": { "displayName": "DiffuGen", "icon": "🖼️", "category": "Creative Tools" }, "default_params": { "steps": { "flux-schnell": 8, "flux-dev": 20, "sdxl": 20, "sd3": 20, "sd15": 20 }, "cfg_scale": { "flux-schnell": 1.0, "flux-dev": 1.0, "sdxl": 7.0, "sd3": 7.0, "sd15": 7.0 }, "sampling_method": { "flux-schnell": "euler", "flux-dev": "euler", "sdxl": "euler", "sd3": "euler", "sd15": "euler" } } } } }
diffugen.json file and paste it into ~/.cursor/mcp.jsondiffugen.json file and paste into ~/.codeium/windsurf/mcp_config.jsondiffugen.json file into Roo Code's MCP configurationdiffugen.json file into Cline's MCP settingsClaude can use DiffuGen if you've set it up as an MCP server on your system. When asking Claude to generate images, be specific about using DiffuGen and provide the parameters you want to use.
To start the DiffuGen server manually:
cd /path/to/diffugen ./diffugen.sh
Or using Python directly:
cd /path/to/diffugen python -m diffugen
You should see: DiffuGen ready when the server is successfully started.
The OpenAPI server provides a REST API interface for direct HTTP access to DiffuGen's image generation capabilities. This is in addition to the MCP integration and can be useful for:
For detailed setup instructions and advanced configuration options, see the OpenAPI Integration Guide.
To start the OpenAPI server:
python diffugen_openapi.py
The server can be configured to use a different host or port if needed. By default, it runs on:
The server will be available at http://0.0.0.0:8080 with interactive documentation at http://0.0.0.0:8080/docs.
Generated images are saved to the /output directory by default. If this directory is not accessible, the server will automatically create an output directory in the current working directory. Images are served through the /images endpoint.
Once added, DiffuGen will appear in the available tools list when clicking the tools icon in the chat interface. The following endpoints will be available:
generate_stable_image_generate_stable_post: Generate with Stable Diffusiongenerate_flux_image_endpoint_generate_flux_post: Generate with Flux Modelslist_models_models_get: List Available ModelsExample using curl:
curl -X POST "http://0.0.0.0:5199/generate/flux" \ -H "Content-Type: application/json" \ -d '{"prompt": "A beautiful sunset", "model": "flux-schnell"}'
Example using Python requests:
import requests response = requests.post( "http://0.0.0.0:5199/generate/flux", json={ "prompt": "A beautiful sunset", "model": "flux-schnell" } ) result = response.json()
Each model has specific default parameters optimized for best results:
| Model | Default Steps | Default CFG Scale | Best For |
|---|---|---|---|
| flux-schnell | 8 | 1.0 | Fast drafts, conceptual images |
| flux-dev | 20 | 1.0 | Better quality flux generations |
| sdxl | 20 | 7.0 | High-quality detailed images |
| sd3 | 20 | 7.0 | Latest generation with good quality |
| sd15 | 20 | 7.0 | Classic baseline model |
These default parameters can be customized by adding a default_params section to your IDE's MCP configuration file:
"default_params": { "steps": { "flux-schnell": 12, // Customize steps for better quality "sdxl": 30 // Increase steps for more detailed SDXL images }, "cfg_scale": { "sd15": 9.0 // Higher cfg_scale for stronger prompt adherence } }
You only need to specify the parameters you want to override - any unspecified values will use the built-in defaults.
Note: For model-specific command line examples and recommendations, see Model-Specific Parameter Recommendations section.
Here are examples of how to ask an AI assistant to generate images with DiffuGen:
Generate an image of a cat playing with yarn
Create a picture of a futuristic cityscape with flying cars
Generate an image of a medieval castle using the sdxl model
Create a flux image of a sunset over mountains
Generate an image of a cyberpunk street scene, model=flux-dev, width=768, height=512, steps=25, cfg_scale=1.0, seed=42
Create an illustration of a fantasy character with model=sd15, width=512, height=768, steps=30, cfg_scale=7.5, sampling_method=dpm++2m, negative_prompt=blurry, low quality, distorted
DiffuGen can be used from the command line with the following basic syntax:
./diffugen.sh "Your prompt here" [options]
Example:
./diffugen.sh "A futuristic cityscape with flying cars"
This command generates an image using default parameters (flux-schnell model, 512x512 resolution, etc.) and saves it to the configured output directory.
Below are the parameters that can be used with DiffuGen (applicable to both MCP interface and command line):
| Parameter | Description | Default | Valid Values | Command Line Flag |
|---|---|---|---|---|
| model | The model to use for generation | flux-schnell/sd15 | flux-schnell, flux-dev, sdxl, sd3, sd15 | --model |
| width | Image width in pixels | 512 | 256-2048 | --width |
| height | Image height in pixels | 512 | 256-2048 | --height |
| steps | Number of diffusion steps | model-specific | 1-100 | --steps |
| cfg_scale | Classifier-free guidance scale | model-specific | 0.1-30.0 | --cfg-scale |
| seed | Random seed for reproducibility | -1 (random) | -1 or any integer | --seed |
| sampling_method | Diffusion sampling method | euler | euler, euler_a, heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm | --sampling-method |
| negative_prompt | Elements to avoid in the image | "" (empty) | Any text string | --negative-prompt |
| output_dir | Directory to save images | Config-defined | Valid path | --output-dir |
These parameters can be specified when asking an AI assistant to generate images or when using the command line interface. Parameters are passed in different formats depending on the interface:
parameter=value (e.g., model=sdxl, width=768, height=512)--parameter value (e.g., --model sdxl --width 768 --height 512)The default values are chosen to provide good results out-of-the-box with minimal waiting time. For higher quality images, consider increasing steps or switching to models like sdxl.
Note: These recommendations build on the Default Parameters by Model section and provide practical examples.
For best results when using specific models via command line:
# Flux-Schnell (fastest) ./diffugen.sh "Vibrant colorful abstract painting" \ --model flux-schnell \ --cfg-scale 1.0 \ --sampling-method euler \ --steps 8 # Flux-Dev (better quality) ./diffugen.sh "Detailed fantasy landscape with mountains and castles" \ --model flux-dev \ --cfg-scale 1.0 \ --sampling-method euler \ --steps 20
# SDXL (highest quality) ./diffugen.sh "Hyperrealistic portrait of a Celtic warrior" \ --model sdxl \ --cfg-scale 7.0 \ --sampling-method dpm++2m \ --steps 30 # SD15 (classic model) ./diffugen.sh "Photorealistic landscape at sunset" \ --model sd15 \ --cfg-scale 7.0 \ --sampling-method euler_a \ --steps 20
The command-line interface of DiffuGen uses the following defaults if not otherwise specified in configuration:
euler (best for Flux models)1.0 for Flux models, 7.0 for standard SD models8 for flux-schnell, 20 for other modelsWhen using the command line, you don't need to specify these parameters unless you want to override the defaults. If you frequently use specific parameters, consider adding them to your configuration file rather than specifying them on each command line.
./diffugen.sh --helpDiffuGen uses a single configuration approach centered around the diffugen.json file:
diffugen.json in the DiffuGen root directory is the single source of truth for all settingsdiffugen.json to your IDE's MCP configuration fileFor advanced usage, you can override settings using environment variables:
SD_CPP_PATH: Override the path to stable-diffusion.cppDIFFUGEN_OUTPUT_DIR: Override the output directoryDIFFUGEN_DEFAULT_MODEL: Override the default modelDIFFUGEN_VRAM_USAGE: Override VRAM usage settingsCUDA_VISIBLE_DEVICES: Control which GPUs are used for generationDiffuGen allows you to have different configurations for different IDEs by using environment variables in each IDE's MCP configuration. This lets you maintain a single base diffugen.json while customizing parameters per IDE.
The configuration priority works as follows:
diffugen.json file (base configuration)Example: Different Output Directories for Different IDEs
For Cursor (in ~/.cursor/mcp.json):
"env": { "CUDA_VISIBLE_DEVICES": "0", "SD_CPP_PATH": "/path/to/stable-diffusion.cpp", "DIFFUGEN_OUTPUT_DIR": "/cursor/specific/output/directory", "default_model": "flux-schnell" }
For Windsurf (in ~/.codeium/windsurf/mcp_config.json):
"env": { "CUDA_VISIBLE_DEVICES": "0", "SD_CPP_PATH": "/path/to/stable-diffusion.cpp", "DIFFUGEN_OUTPUT_DIR": "/windsurf/specific/output/directory", "default_model": "sdxl" }
Example: Different Default Models and VRAM Settings
"env": { "CUDA_VISIBLE_DEVICES": "0", "SD_CPP_PATH": "/path/to/stable-diffusion.cpp", "default_model": "flux-dev", "DIFFUGEN_VRAM_USAGE": "maximum" }
This approach lets you customize DiffuGen's behavior per IDE while still using the same underlying installation.
diffugen.sh script (must be absolute path)CUDA_VISIBLE_DEVICES: Controls which GPUs are used for generation
"0": Use only the first GPU"1": Use only the second GPU"0,1": Use both first and second GPUs"-1": Disable CUDA and use CPU onlySD_CPP_PATH: Path to the stable-diffusion.cpp installation directory
default_model: The default model to use when none is specified
models_dir: Directory containing the model files
models directory inside your stable-diffusion.cpp installationoutput_dir: Directory where generated images will be saved
vram_usage: Controls VRAM usage strategy
"adaptive": Automatically adjust memory usage based on available VRAM"minimal": Use minimal VRAM at the cost of speed"balanced": Balance memory usage and speed (default)"maximum": Use maximum available VRAM for best performanceEach IDE has specific options you can customize in the diffugen.json file:
"cursorOptions": { "autoApprove": true, "category": "Image Generation", "icon": "🖼️", "displayName": "DiffuGen" }
"windsurfOptions": { "displayName": "DiffuGen", "icon": "🖼️", "category": "Creative Tools" }
You can customize default parameters for each model in the default_params section:
"default_params": { "steps": { "flux-schnell": 12, "sdxl": 30 }, "cfg_scale": { "sd15": 9.0 }, "sampling_method": { "flux-schnell": "euler", "sdxl": "dpm++2m" } }
When using the automatic setup script, a properly configured diffugen.json file is created with the correct paths for your system when you run option 5. To integrate DiffuGen with your IDE:
setup_diffugen.sh to update paths in diffugen.jsondiffugen.json file~/.cursor/mcp.json)The key advantage of this approach is a single source of truth for configuration, making it easier to maintain and update your DiffuGen setup.
The DiffuGen Python module can be imported and used programmatically in your own Python scripts:
from diffugen import generate_image # Generate an image programmatically result = generate_image( prompt="A starry night over a quiet village", model="sdxl", width=1024, height=768, steps=30, cfg_scale=7.0, seed=42, sampling_method="dpm++2m", negative_prompt="blurry, low quality" ) print(f"Image saved to: {result['file_path']}")
You can also use the OpenAPI server programmatically in your applications:
import requests def generate_image_via_api(prompt, model="flux-schnell", width=512, height=512): response = requests.post( "http://0.0.0.0:5199/generate/flux", json={ "prompt": prompt, "model": model, "width": width, "height": height } ) return response.json() # Example usage result = generate_image_via_api( prompt="A magical forest at night", model="flux-schnell", width=768, height=512 ) print(f"Generated image: {result['file_path']}")
Missing models or incorrect paths
CUDA/GPU issues
CUDA_VISIBLE_DEVICES to target a specific GPUImage quality issues
File permission errors
chmod +x diffugen.sh)If you encounter issues not covered here, you can:
DEBUG=1 ./diffugen.sh "your prompt"Contributions to DiffuGen are welcome! To contribute:
Please ensure your code follows the project's coding standards and includes appropriate tests.
This project is licensed under the Apache License - see the LICENSE file for details.
______ __ ___ ___ _______
| _ \ |__|.' _|.' _|.--.--.| _ |.-----.-----.
|. | \| || _|| _|| | ||. |___|| -__| |
|. | \__||__| |__| |_____||. | ||_____|__|__|
|: 1 / |: 1 |
|::.. . / |::.. . |
`------' `-------'
Made with ❤️ by CLOUDWERX LAB