Replies: 5 comments 16 replies
-
|
Hi @Columboom - we will have a look at the issue - I kind of doubt we have tested this with Alibaba model - have you tried the same with openai? |
Beta Was this translation helpful? Give feedback.
-
|
The problem is greatly alleviated when using gpt-4o. Are the innner prompts in autogen specifically designed and optimized for gpt？ |
Beta Was this translation helpful? Give feedback.
-
It's because the model couldn't generate a tool call for handing off to the user. As mentioned in the guide, Swarm relies on tool calls to make the transition. No magic here. To mitigate this issue with your model, besides promoting, try different termination conditions as described in the Terminations section of the tutorial. You can try TextMessageTermination
Could you explain. What is query?
Please upgrade to the latest version it has been added. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your sharing. I managed to get the llm results in the loop, the content field was indeed empty. Now I've written custom Termination classes to end the loop temporarily. Since the issue is essentially caused by insufficient model capabilities, is there any plan to add more boundary conditions into autochat to improve its robustness? I wonder if the innner prompts in autogen are specifically designed and optimized for openai model. Have you tested autochat using non-openai models? Besides, "query" refers to the string content output of agent before it handoffs to user. Sorry for not making that clear. |
Beta Was this translation helpful? Give feedback.
-
|
Converting this to discussion as it isn't a bug. Consider improving documentation with help of OP. |
Beta Was this translation helpful? Give feedback.
-
|
@ekzhu Thank you for you reply, I also met this issue while using AzureOpenAI gpt-4o and using Alibaba qwen-max which are the most common models so I would appreciate if boundary conditions can be optimized |
Beta Was this translation helpful? Give feedback.
-
|
Also, I have tried HandoffTermination(target="user") and TextMessageTermination. It doesn't work well. |
Beta Was this translation helpful? Give feedback.
-
I believe the model can generate a tool call. It is just that the way the prompt is written might have prevented it from doing so.
Based on the log output, it looks like |
Beta Was this translation helpful? Give feedback.
-
|
This is my attempt to reproduce this using import asyncio
from autogen_agentchat.messages import HandoffMessage
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_agentchat.teams import Swarm
from autogen_agentchat.conditions import HandoffTermination
from autogen_ext.models.openai import OpenAIChatCompletionClient
from prompt_toolkit.styles import Style
style = Style.from_dict({
'': '#00ffff bg:#444400 bold',
})
def get_storage_account_creation_default_settings():
"""Use this tool to get default settings for creating a storage account."""
return "default Kind is StorageV2, default SKU is Standard_LRS, default location is China East 2"
def create_storage_account(subscription_id: str, resource_group_name: str, storage_account_name: str,
location: str = "China East 2", kind: str = "StorageV2", sku: str = "Standard_LRS", ) -> str:
"""Use this tool to create a storage account.
only 3 arguments: storage_account_name, resource_group_name, subscription_id are required from user.Optional arguments are location, kind, sku.
Do not add any other arguments."""
return "Storage account operation is in progress. Operation succeeded."
def check_storage_account_exists(subscription_id: str, resource_group_name: str,storage_account_name: str) -> str:
return f"Storage account '{storage_account_name}' does not exist in resource group '{resource_group_name}'.Pass the check."
model_client = OpenAIChatCompletionClient(
model="gpt-4o",
)
manager_agent = AssistantAgent(
name="manager_agent",
model_client=model_client,
description="Manager agent",
system_message="""You are an Azure Platform Service Manager to assist developers with inquiries related to the Azure platform, including [Azure resources, AKS, StackStorm].
Your role is to:
1. Send your role to user first
2. Determine if they're asking about:
- handoff to querier_agent for queries about azure
Always handoff to a single agent at a time.
If you get empty response, please ask user to provide more information again.
""",
handoffs=[
"querier_agent",
],
)
query_agent = AssistantAgent(
name="querier_agent",
description="Azure resources querier agent",
model_client=model_client,
tools=[check_storage_account_exists,get_storage_account_creation_default_settings],
system_message="""You are an Azure resources querier agent in Azure platform.
If user is going to have operations on Azure resources:
1. asked necessary information(name/resource group/subscription id) for searching resources in azure
2. handoff back to user to get the inputs
3. if you get empty response, please ask user to provide more information again.
3. Once you got the information from, search resources in azure:
If resource already exists, ask user to change the name.
If resource is not found, handoff to executor agent
Before create the resource, use tool to get default settings and ask user for approval,
Always handoff to a single agent at a time. """,
handoffs=["user", "executor_agent"],
)
executor_agent = AssistantAgent(
name="executor_agent",
description="Azure resources operation agent",
model_client=model_client,
tools=[create_storage_account],
system_message="""You are an Azure resources operation agent in Azure platform.
Please assist in creating and modifying Azure resources.
if the operation is successful, delegate back to query agent.
if the operation is failed or canceled by user, ask user to correct the information.
If you get empty response, please ask user to provide more information again.
Always handoff to a single agent at a time.""",
handoffs=[
"user", "querier_agent"
],
)
termination = HandoffTermination(target="user")
team = Swarm(participants=[manager_agent, query_agent, executor_agent], termination_condition=termination)
async def run_team_stream() -> None:
print("Hello, I am the Cloud AI Assistant🤖. How can I help you today?")
task = input("User: ")
task_result = await Console(team.run_stream(task=task))
last_message = task_result.messages[-1]
while True:
if isinstance(last_message, HandoffMessage) and last_message.target == "user":
user_message = input(f"Respond to {last_message.source}: ")
task_result = await Console(
team.run_stream(task=HandoffMessage(source="user", target=last_message.source, content=user_message))
)
else:
task_result = await Console(team.run_stream(task=task))
last_message = task_result.messages[-1]
asyncio.run(run_team_stream())And I get perfectly fine output: |
Beta Was this translation helpful? Give feedback.
-
|
Could you try to run a basic example with the Qwen model? For example, the quick start example for getting weather for a city? Perhaps there is some incompatibility with the model regarding how tool is used. |
Beta Was this translation helpful? Give feedback.
-
|
@ekzhu Hi, when you're using OpenAI, most of the time it works pretty well, but sometimes it might also run into issues like getting repeated messages or stuck in loops. Can you try a few more times? |
Beta Was this translation helpful? Give feedback.
-
|
Yes it does happens sometimes though in those cases I think a good set of termination conditions or perhaps custom termination condition can help. Have you tried using TextMessageTermination, or creating your own termination condition? Not sure how can I help with Qwen Max. I feel it should be prompting problem. Have you tried a simple example with Qwen and see how it behaves with regarding to tool calls? |
Beta Was this translation helpful? Give feedback.
-
What happened?
Describe the bug
I used Swarm in autogen_autochat involves human-computer interaction. It obtains user input through handoffs. When the agent failed to understand the prompt well —resulting in no handoff target—the call fell into an infinite loop. In which both the context messages and content were empty. I suspect the root cause lies in either the swarm’s handoff management logic or a multithreading issue.
To Reproduce
I ran the demo Swarm Quickstart with LLM qwen-max.
The agents didn't follow the prompt to output query sentence firstly before handoff to user. The query sentence and handoff function call were returned in the same message.
Therefore, I modified the prompt with "If you need information from the user, you must first only send your message, then you can handoff to the user."
Now the query sentence is output as expected, but the infinite loop occurs.
Expected behavior
Issures
I would appreciate it if you could answer:
Which packages was the bug in?
Python AgentChat (autogen-agentchat>=0.4.0)
AutoGen library version.
Python 0.4.5
Other library version.
No response
Model used
qwen-max
Model provider
Other (please specify below)
Other model provider
alibaba
Python version
None
.NET version
None
Operating system
None
Beta Was this translation helpful? Give feedback.
All reactions