Dynamo
A distributed inference serving framework for teams that deploy inference workloads at datacenter scale. A Datacenter Scale Distributed Inference Serving Framework
Pricing
Open source (free)
Update history
Last updated 2026-11-20;Listed 2026-11-18
Author
ai-dynamo
Trust info
Listed on 2026-11-18; Last updated 2026-11-20; Data synced on 2026-11-20
Trial feedback
Like it? Vote to help it rank higher.
Deploy command
# Pull a prebuilt container
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0
# Inside the container — start frontend and worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --discovery-backend file &
# Once the worker is ready, send a request
curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "Qwen/Qwen3-0.6B",
"messages": [{"role": "user", "content": "Hello!"}],
"max_tokens": 100
}' | jqTaken from the project README. You need a server with Docker; you may still need to add environment variables.
From the README
Dynamo can expose traffic through two Kubernetes request routing topologies. Both expose an OpenAI-compatible API and support the same backends, disaggregated serving, and KV-aware routing.
- Stars
- ★ 8.5k
- Category rank
- #498
- Language
- Rust
- License
- Not specified
- Last updated
- 2026-11-20
- Category
- AI assistants & agents
- Section
- General







