Skip to content

Latest commit

 

History

History

guides

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
dynamo_deploy.md
dynamo_deploy.md
 
 
dynamo_run.md
dynamo_run.md
 
 
dynamo_serve.md
dynamo_serve.md
 
 

README.md

Guide to Dynamo CLI

After installing Dynamo with the following command, Dynamo can be used primarily through its CLI.

apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -yq python3-dev python3-pip python3-venv libucx0
python3 -m venv venv
source venv/bin/activate

pip install ai-dynamo[all]

Dynamo workflow

Dynamo CLI has the following 4 sub-commands.

  • 🏃 dynamo run: quickly spin up a server to experiment with a specified model, input and output target.
  • 🫴 dynamo serve: compose a graph of workers locally and serve.
  • 🔨 (Experiemental) dynamo build: containerize either the entire graph or parts of graph to multiple containers
  • 🚀 (Experiemental) dynamo deploy: deploy to K8 with helm charts or custom operators

For more detailed examples on serving LLMs with disaggregated serving, KV aware routing, etc, please refer to LLM deployment examples