Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the Finalizer and add integration tests #240

Open
10 tasks
grischperl opened this issue Oct 29, 2024 · 0 comments
Open
10 tasks

Improve the Finalizer and add integration tests #240

grischperl opened this issue Oct 29, 2024 · 0 comments

Comments

@grischperl
Copy link
Collaborator

grischperl commented Oct 29, 2024

Description

Add more complex integration test cases to ensure the functionality of the Finalizer. Making the tests green will most likely involve improving the prompt of the Finalizer.

Background

The Finalizer was moved into the Supervisor subgraph in the issue #170. The basic functionality of the Finalizer was covered in integration tests. However, some of the functionality suggested in the prompt could not be seen in more complex tests cases. To rectify this:

Considerations/Recommendations for Integration tests:

  • Add a syntax scoring evaluation function. It should give a bad score if:
    • Any agent names are in the final response
    • Extra headers/sub-headers are added to the final response

Test case(s):

  • Two agents, one responding with invalid answers, and the other responding with valid answers.
    • The valid answers should be included in the final response.
    • The invalid answers should be removed from the final response.
    • Apology for the invalid answers should be included in the final response.

Acceptance Criteria(s):

  • Confirm with the PO/Team the desired behaviour
    • Current behaviour: If the answer is wrong then the Finalizer apologies. But if the answer is partially correct then in the final answer it only includes the partially correct part and skips the incorrect part.
  • Come up with the necessary integration test cases
  • Make the tests green by improving the Finalizer prompt
@grischperl grischperl changed the title draft Improve the Finalizer Oct 29, 2024
@mfaizanse mfaizanse changed the title Improve the Finalizer Improve the Finalizer and add integration tests Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant