Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSPY 2.5 + Avatar agent: max_iters ignored #1588

Open
tikoehle opened this issue Oct 5, 2024 · 2 comments
Open

DSPY 2.5 + Avatar agent: max_iters ignored #1588

tikoehle opened this issue Oct 5, 2024 · 2 comments

Comments

@tikoehle
Copy link

tikoehle commented Oct 5, 2024

In the avatar example notebook there is a comment about max_iters "is used to control the number of iterations in multi step execution". With my agent instance (max_iters=10)

self.avatar = Avatar(
            tools=self.tools,
            signature=TestSignature,
            verbose=verbose,
            max_iters=10,
        )

the process was terminated at action_73 with querying on a new tool_name (not exist in my list of Tools!), which consequently resulted in a validation error because tool_output was None. Creating the FINISH action somehow adaptive to the problem resolution could be an interesting feature because I observed my result_72 was very detailed and perfectly solved the problem.

Action 73: VERIFICATION_TOOL (...)
1 validation error for ActionOutput
tool_output
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
    For further information visit https://errors.pydantic.dev/2.9/v/string_type

In dspy/predict/avatar/avatar.py debugging shows the max_iters is None and the while loop never terminates.

        max_iters = None if "max_iters" not in kwargs else kwargs["max_iters"]
        while tool_name != "Finish" and (max_iters > 0 if max_iters else True):

This simple fix solved the max_iters problem:

        # max_iters = None if "max_iters" not in kwargs else kwargs["max_iters"]                                                                                            
        max_iters = self.max_iters
@okhat
Copy link
Collaborator

okhat commented Oct 6, 2024

@krypticmouse

@tikoehle
Copy link
Author

tikoehle commented Oct 6, 2024

Attaching a diff for avatar.py which solved the following problems for me. Feel free to use or object.

  • max_iters as reported above,
  • With a high number of max_iters, if the LM invents a tool_name (VERIFICATION_TOOL example above), then make the action a Finish action,
  • Finish action name can get uppercase 'Finish' vs. 'FINISH', causing string compare to fail.
    Action 6: Finish (All actions ... )
    Action 5: FINISH (Conclude the ... )
  • Fix tool_output is None
    avatar.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants