Directory › Inference runtimes › Ollama
Ollama
Ollama is a self-hostable tool that allows users to run open LLMs locally via a simple CLI and OpenAI-compatible REST API. It pulls and serves GGUF models. The tool operates on local data.
Who it's for: Users who want to utilize open LLMs locally may benefit from using Ollama.
Strengths
- Supports run of open LLMs locally
- Provides OpenAI-compatible REST API
- Serves GGUF models
Trade-offs
- Limited functionality compared to cloud-based services
- Dependent on local data storage
Data locality
Runs fully offline — no data leaves your machine.