Directory › Inference runtimes › Text Generation Inference (TGI)
Text Generation Inference (TGI)
Text Generation Inference (TGI) is a production toolkit for deploying and serving Large Language Models (LLMs). It provides a flexible framework for inference tasks. The tool is designed to be self-hostable.
Who it's for: Developers and organizations looking to deploy and serve LLMs should consider using Text Generation Inference.
Strengths
- Provides a production-ready toolkit for deploying and serving LLMs
- Supports local-only data storage
- Open-sourced under the Apache-2.0 license
Trade-offs
- May require expertise in deployment and serving LLMs
- Data storage limitations apply to local-only mode
Data locality
Runs fully offline — no data leaves your machine.