In case of LLM's you should look at AirLLM. I suppose there is no conviniet integrations to local chat tools, but issue at Ollama already started.
Sylovik
joined 3 weeks ago
In case of LLM's you should look at AirLLM. I suppose there is no conviniet integrations to local chat tools, but issue at Ollama already started.
You may try Harbor. The description claims to provide an OpenAI-compatible API.