Weekly digestOne email a week: top 5 new tools + trending stars. Unsubscribe anytime.

Dynamo

ai-dynamo/dynamo
Open source (free)
Needs setup You need your own server plus database, environment variables, domain or mail configuration. Read the official deployment docs first.

A distributed inference serving framework for teams that deploy inference workloads at datacenter scale. A Datacenter Scale Distributed Inference Serving Framework

diffusiondisaggregated-servingkubernetesllm-inference
Dynamo website screenshot

Pricing

Open source (free)

Update history

Last updated 2026-11-20;Listed 2026-11-18

Author

ai-dynamo

Trust info

Listed on 2026-11-18; Last updated 2026-11-20; Data synced on 2026-11-20

Trial feedback

Like it? Vote to help it rank higher.

Deploy command

# Pull a prebuilt container
docker run --gpus all --network host --rm -it nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.6.0

# Inside the container — start frontend and worker
python3 -m dynamo.frontend --http-port 8000 --discovery-backend file > /dev/null 2>&1 &
python3 -m dynamo.vllm --model Qwen/Qwen3-0.6B --discovery-backend file &

# Once the worker is ready, send a request
curl -s localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Qwen/Qwen3-0.6B",
  "messages": [{"role": "user", "content": "Hello!"}],
  "max_tokens": 100
}' | jq

Taken from the project README. You need a server with Docker; you may still need to add environment variables.

From the README

Dynamo can expose traffic through two Kubernetes request routing topologies. Both expose an OpenAI-compatible API and support the same backends, disaggregated serving, and KV-aware routing.

Stars
★ 8.5k
Category rank
#498
Language
Rust
License
Not specified
Last updated
2026-11-20
Category
AI assistants & agents
Section
General