Docs · Questions
LOPsUsage
How do I use local AI models (like `llama.cpp` or Ollama) with LOPs?
Problem
Users frequently encounter errors or limitations when trying to use LOPs with their own local AI models, such as those served with llama.cpp or Ollama.
1. Connecting to a llama.cpp Server
- Symptom: You receive errors when trying to connect LOPs to a local
llama.cppserver, even if it exposes an OpenAI-compatible endpoint (e.g., viallama-swap). - Solution: This is a known issue that will be addressed in a future update. In the meantime, you can try the following:
- Verify Endpoint URL: Double-check that the URL and port are correct in the ChatTD API configuration.
- Check Firewall Settings: Ensure that your firewall is not blocking the connection between TouchDesigner and the local server.
- Consult
litellmDocs: For advanced configurations, refer to thelitellmdocumentation, as it is the underlying library that LOPs uses to connect to models.
2. Running Multiple Local Models in Parallel
- Symptom: You experience errors or instability when trying to use more than one local model at the same time (e.g., a vision model and a text model).
- Solution: This is a known limitation that will be fixed in a future update. As a temporary workaround, you can:
- Consolidate Models: If possible, serve all your models from a single server endpoint.
- Use Separate Machines: Run each local model on a separate computer to avoid port conflicts and resource competition.
3. RAGIndex Fails with Local Ollama Models
- Symptom: When using the
RAGIndexLOP with a local Ollama model, you get a “Could not load OpenAI embedding model” error. - Solution: This is a missing dependency issue. You can fix it by running the following command in the LOPs Python Manager:
pip install llama-index-embeddings-ollama