Questions

Docs · Questions

StreamDiffusionTDUsage

Which models are compatible with StreamDiffusionTD and how do I know if a model will work?

GPU and suggested starting settings - use this guide for current configuration advice.

StreamDiffusionTD Model Compatibility Guide

Quick Reference

Model Type Works? TensorRT? Notes
sd-turbo Yes Yes Default, best supported
sdxl-turbo Yes (v0.3.0+) Yes (v0.3.0+) Higher memory use; see the GPU guide
SD 1.5 + LCM-LoRA Yes Yes Requires compatible LCM-LoRA
SD 2.1 models Partial Yes sd-turbo is 2.1-based
SDXL models Limited Yes (v0.3.0) v0.3.0 improved support
Flux No No Not realtime-capable
Video models No No Use ComfyTD instead

Built-in Model Options

These are pre-configured in the Modelid dropdown:

  • stabilityai/sd-turbo (default)
  • stabilityai/sdxl-turbo
  • prompthero/openjourney-v4

How to Evaluate Any Model

Checklist

  1. Architecture Check

    • Must be SD 1.5, SD 2.1, or SDXL architecture
    • NOT video models, NOT Flux, NOT custom architectures
  2. Turbo/Distilled?

    • Turbo models (sd-turbo, sdxl-turbo) work best - optimized for few-step inference
    • Non-turbo models need LCM-LoRA for speed
  3. File Format

    • .safetensors single file → Use full path
    • Diffusers folder format → Use folder path
    • HuggingFace ID → Use org/model-name format
  4. VRAM Budget

    • SD 1.5/2.1: ~5-8GB minimum
    • SDXL: ~16-24GB minimum
    • Add more for ControlNet (+2-4GB) and IP Adapter (+2-4GB)

LoRA Compatibility - CRITICAL

sd-turbo is SD 2.1 based, so SD 1.5 LoRAs will NOT work with it!

Model LoRA Version Needed
sd-turbo SD 2.1 LoRAs (rare)
SD 1.5 models SD 1.5 LoRAs
SDXL models SDXL LoRAs

LCM-LoRA Auto-Selection

The Customlcm parameter handles this:

  • auto - Automatically selects based on model
  • latent-consistency/lcm-lora-sdv1-5 - For SD 1.5
  • latent-consistency/lcm-lora-sdxl - For SDXL

Models That Won’t Work

Model Type Reason
Flux Too slow, not distilled for realtime
Video models Different architecture - use ComfyTD
SDXL Lightning Untested, may have issues

Per dotsimulate: “For using a model such as [video model], you will need to use ComfyTD. StreamDiffusion is a very specific pipeline for realtime image models.”

Loading Custom Models

Via HuggingFace ID

  1. Find model on huggingface.co
  2. Copy the path (e.g., stabilityai/sd-turbo)
  3. Paste into “Model Id” parameter
  4. Start stream - downloads automatically on first use

Via Local File

  1. Download .safetensors file
  2. Enter full path in “Model Id”: C:/path/to/model.safetensors
  3. Start stream

Working Models List

After successful streaming (200+ frames), models are auto-saved to: StreamDiffusion/streamdiffusionTD/working_models.json

These appear in the “My Models” dropdown.

Known Issues

SD 1.5 + LCM-LoRA Blur

Multiple users report blurry output. Potential causes:

  • Wrong LCM-LoRA version (using SD 1.5 LoRA with wrong model)
  • Step count too low
  • Parameter misconfiguration

Model Not Recognized

The “My Models” dropdown only populates after successful streaming. Use the full path directly in “Model Id” parameter.


Related FAQs: #19 (model downloads), #23 (TensorRT per model)