Directory › Inference runtimes › llama.cpp

llama.cpp is a C/C++ LLM inference engine for GGUF models that can run on CPU/GPU across various platforms. It is open-sourced under the MIT license. The tool is designed to be self-hostable.

Who it's for: Developers and researchers who need a flexible and customizable LLM inference engine for their projects should consider using llama.cpp.