Text Generation Inference
HuggingFace's Rust-based serving stack with tensor parallelism and streaming built in.
TGI is a mature inference server that handles sharding across multiple GPUs, continuous batching and token streaming. It integrates tightly with the HuggingFace ecosystem, which matters when your model of choice only exists as a HF checkpoint.
Best for: Multi-GPU serving inside the HuggingFace ecosystem
Deploy: Self-hostable