Questions

Docs · Questions

StreamDiffusionTDUsage

How does TensorRT engine building work? When do I need to rebuild the engine in different StreamDiffusionTD versions?

TensorRT Engine Building - Pre-v0.3.0 vs v0.3.0+

TensorRT acceleration requires building an “engine” optimized for your specific settings. The rules changed significantly in v0.3.0.

Pre-v0.3.0 Behavior (v0.1.x - v0.2.99)

Applies to: All versions before November 6, 2025

Engine is Dependent On:

  1. Resolution (width x height)
  2. Step count (number of diffusion steps)
  3. Model
  4. ControlNet enabled/disabled
  5. IP Adapter enabled/disabled (v0.2.6+)

When You MUST Rebuild:

  • Changing resolution (e.g., 512x512 → 768x768)
  • Changing step count (e.g., 1 step → 4 steps)
  • Switching models
  • Enabling/disabling ControlNet
  • Enabling/disabling IP Adapter

Build Time:

  • First build: 20-30 minutes (sometimes longer)
  • Do NOT interrupt - let it complete
  • Console will show “Building TensorRT engine…” progress

Non-512x512 Resolution Support:

  • v0.1.0-v0.1.10: Only 512x512 supported with TensorRT
  • v0.1.11 (May 16, 2024)+: Non-512x512 resolutions supported

v0.3.0+ Behavior (November 2025+)

Major Improvement: TensorRT is now step-independent.

Engine is Dependent On:

  1. Model
  2. ControlNet enabled/disabled
  3. IP Adapter enabled/disabled

Engine is NO LONGER Dependent On:

  1. ✅ Resolution - can change within range without rebuild
  2. ✅ Step count - can switch between 1-4 steps freely

Flexibility in v0.3.0:

  • Resolution: 384 to 1024 pixels (auto-rounds to multiple of 64)
    • Example: 512x512, 768x512, 640x896 all work without rebuild
  • Steps: 1-4 steps available, switch freely after initial build
  • Only rebuild when: Changing model OR toggling IP Adapter/ControlNet

Build Time:

  • Still 20-30 minutes for initial build
  • But you only build once per model+configuration combo

Practical Examples

Pre-v0.3.0 Workflow:

1. Start stream: 512x512, 1 step → builds engine (30 min)
2. Want 768x768 → MUST rebuild (30 min)
3. Want 4 steps → MUST rebuild again (30 min)

v0.3.0 Workflow:

1. Start stream: 512x512, 1 step → builds engine (30 min)
2. Want 768x768 → just change parameter (instant)
3. Want 4 steps → just change parameter (instant)
4. Want IP Adapter → rebuild required (30 min)

Tips to Avoid Unnecessary Rebuilds

For Pre-v0.3.0:

  • Decide your settings first before starting the build
  • Test without TensorRT initially to find good parameters
  • Don’t toggle IP Adapter/ControlNet mid-session
  • Save TensorRT engines using “Model Preset Config” system (v0.1.11+)

For v0.3.0+:

  • Still decide on IP Adapter/ControlNet before building
  • Resolution and steps are now flexible for standard inference
  • IP Adapter is “sticky” - decide at build time (can’t toggle after)
  • StreamV2V (v0.3.1): Cached attention engines are locked to their build resolution. If you built at 512x512, StreamV2V only works at 512x512. Other resolutions require a new engine build

Common Issues

Engine Build Fails or Times Out

  • Usually due to insufficient VRAM
  • Try smaller resolution first
  • On SDXL, disable IP Adapter or ControlNet if VRAM limited (<24GB)

Engine “Corrupted”

Error: Exception: Acceleration has failed

Solution: Delete engines folder and rebuild

  • Location: StreamDiffusion/engines/td/
  • Often caused by interrupted builds or toggling settings mid-build

Version Timeline:

  • v0.1.0-v0.1.10: 512x512 only, resolution+step dependent
  • v0.1.11 (May 2024): Non-512x512 support added
  • v0.2.0-v0.2.99: Resolution+step+IP/CN dependent
  • v0.3.0 (Nov 2025): Step-independent, flexible resolution (384-1024)
  • v0.3.1 (Mar 2026): Same flexibility, but StreamV2V engines are resolution-locked