MLServer

MLServer is an open-source serving runtime for machine learning models, designed to deploy and serve models from various frameworks with high performance and scalability.

Available Pages

MLServer - Serve ML Models Without Writing Another Flask Wrapper

Discover MLServer, the Python inference server designed for production ML model deployment. Learn why it's essential, how to get started, and troubleshoot common issues for seamless serving.

Related Technologies

Competition

tensorflow serving

Direct competitors

torchserve

Direct competitors

bentoml

Direct competitors

triton inference server

Direct competitors

ray serve

Can replace or substitute

Integration

Compatible With

mlflow

Works well together

Integrates With

prometheus

Official integration support

Compatible With

docker

Works well together

Compatible With

onnx runtime

Works well together

Dependencies

kserve

Enables other tools

seldon core

Enables other tools

fastapi

Requires for operation

python

Foundation technology

uvicorn

Requires for operation

kubernetes

Foundation technology

grpc

Requires for operation

starlette

Requires for operation

Similar

clipper

Similar functionality