Usage Examples

Image-to-Image with Input

  1. Connect a TOP to the first input of StreamDiffusionTD (this is the input image)
  2. SD Mode is set to img2img by default
  3. Use Step Sliders to control denoising:
    • Higher values = closer to input image
    • Lower values = more AI transformation
  4. Create custom effects and change the prompt and adjust settings to refine the look of the generated image stream

ControlNet (Local Mode)

Local ControlNet uses the second input for preprocessed control images:

  1. Connect preprocessed image (edges, depth, pose, etc.) to the second input
  2. Enable ControlNet in the ControlNet parameter page before starting stream
  3. ControlNet must be enabled when starting stream, but can be toggled during streaming
  4. Limited to single ControlNet for now, but supports many types
  5. Automatic download available for 12 different ControlNet types depending on your model
  6. You can use any ControlNet model as long as it matches your base model type

ControlNet (Daydream Mode)

Daydream ControlNet works differently - it uses the same input as the main image:

  1. Connect input image to the first input (same as regular img2img)
  2. Daydream has its own built-in preprocessors - no preprocessing required
  3. Multiple ControlNets can be combined for complex control
  4. Each ControlNet has individual weight controls
  5. Five ControlNet types available: OpenPose, HED, Canny, Depth, and Color

V2V Temporal Consistency

  1. Enable V2V in V2V parameter page
  2. Adjust temporal settings for smooth frame transitions
  3. Not compatible with TensorRT acceleration
  4. Ideal for video sequences and smooth animations

Performance Optimization

High FPS Setup

  • Use TensorRT acceleration (local mode)
  • Use default resolution (512x512)
  • Lower step count (1-2 steps for highest FPS)
  • Use optimized models (SD-Turbo, SDXS)

Quality Setup

  • Increase step count (2-4 steps)
  • Try higher resolution like 576x1024 for good 16:9 ratio
  • Use SDXL-Turbo model for highest quality (will be slower)
  • Enable V2V for temporal consistency