Questions

Docs · Questions

StreamDiffusionTDError

Why do I get "Input type (torch.cuda.HalfTensor) and weight type (torch.HalfTensor) should be the same" error?

Issue: StreamDiffusionTD error: “Input type (torch.cuda.HalfTensor) and weight type (torch.HalfTensor) should be the same” when running inference.

Status: Known upstream issue in v0.3.0 and v0.3.1. TensorRT acceleration is required for ControlNet, IPAdapter, and StreamV2V. The non-TRT path (acceleration=“none”) has device placement bugs that remain unfixed upstream.

When it Occurs:

  • v0.3.0 and v0.3.1
  • Happens when TensorRT acceleration is turned OFF (acceleration=“none”)
  • Affects multiple GPU types (RTX 3050, 4090, 5080, 5090)
  • Error repeats every frame during streaming
  • Floods TouchDesigner textport with error messages

Root Cause: PyTorch device mismatch. Model weights end up on CPU instead of CUDA despite both being HalfTensor (FP16) type. This is a known issue when using “no acceleration mode” for local inference.

Fix: Enable TensorRT acceleration.

  1. Open StreamDiffusionTD Settings tab
  2. Set “Acceleration” to “TensorRT” (NOT “none”)
  3. Click “Start Stream”
  4. Allow TensorRT engine to build (first run takes 5-30 minutes depending on GPU)
  5. Stream should work without error

Important Configuration Check: Some users report TensorRT appears enabled in UI but the generated config file shows acceleration: "none". If the error persists:

  1. After clicking “Start Stream”, check the yaml file at: ...\StreamDiffusion\StreamDiffusionTD\td_config.yaml
  2. Verify it shows acceleration: "tensorrt" (not “none”)
  3. If it shows “none”, restart stream and verify settings

Why This Works: TensorRT keeps all model weights properly on GPU and handles device placement correctly, bypassing the device mismatch bug.

Status:

  • Known upstream issue, not specific to StreamDiffusionTD
  • TensorRT is required for all advanced features (ControlNet, IPAdapter, StreamV2V)
  • Use TensorRT acceleration for reliable operation

Applies to: v0.3.0 and v0.3.1