NVIDIA Triton Inference Server
An open-source inference serving platform that enables deployment of AI models from multiple frameworks with optimized performance for real-time, batched, and streaming inference across cloud, edge, and embedded devices.