fix: NATS queue initialization failure caused by customized served_model_name in PD disagg

[dashanji]

Overview:

If we use a customized served_mode_name, the vllm will parse it as a list but not a string.
https://github.com/ai-dynamo/dynamo/blob/main/examples/llm/utils/vllm.py#L49

Thus, the NATS queue initialization will crash there as the stream_name (assigned by served_model_name) is a list.
https://github.com/ai-dynamo/dynamo/blob/main/examples/llm/utils/nats_queue.py#L42

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: server hang for kv cache router and disaggregated #288

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi dashanji! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[a4zhangfei]

@dashanji When is it expected to merge into the main branch?

[rainj-me]

Please not directly choose the first element. How about we use sha256 hash or base64 encoding as stream name ? In this way we don't have to replace special characters later on?

[rainj-me]

Please not directly choose the first element. How about we use sha256 hash or base64 encoding as stream name ? In this way we don't have to replace special characters later on?

Refer commit bytedance-iaas@bd3f004

[dashanji]

@rainj-me I think it's not a good idea to use a random one. Once we use the PD disagg in a distributed environment, how to make the P worker and D worker use the same nats stream? In this case, a customized served_model_name by users makes sense.

[rainj-me]

@rainj-me I think it's not a good idea to use a random one. Once we use the PD disagg in a distributed environment, how to make the P worker and D worker use the same nats stream? In this case, a customized served_model_name by users makes sense.

sha256 is not random value