Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agents-api,memory-store): Tasks queries #978

Merged
merged 6 commits into from
Dec 21, 2024

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented Dec 19, 2024

User description

  • feat(agents-api): Remove auto_blob_store in favor of interceptor based system
  • fix(agents-api): Minor fixes
  • wip(agents-api,memory-store): Tasks queries

PR Type

Enhancement


Description

  • Added comprehensive task management system with version control:
    • CRUD operations for tasks with proper validation
    • Workflow management with step tracking
    • Version control for task updates
  • Enhanced data validation and constraints:
    • Added canonical_name field with regex validation
    • Added length constraints for name fields
    • Added JSON schema validation for metadata fields
  • Improved database schema:
    • Added cascading deletes for related tables
    • Added proper indexing for performance
    • Added constraints for data integrity

Changes walkthrough 📝

Relevant files
Enhancement
Tasks.py
Add canonical name and field validation to Task models     

agents-api/agents_api/autogen/Tasks.py

  • Added canonical_name field with validation pattern for task-related
    models
  • Enhanced field documentation with docstrings
  • Added length constraints for name and canonical_name fields
  • +56/-2   
    Add task management queries and version control                   

    agents-api/agents_api/queries/tasks/

  • Implemented CRUD operations for tasks (create, read, update, delete)
  • Added version control for task updates
  • Added workflow management functionality
  • Implemented task listing with filtering and pagination
  • 000010_tasks.up.sql
    Add tasks and workflows database schema                                   

    memory-store/migrations/000010_tasks.up.sql

  • Added tasks and workflows tables with constraints
  • Implemented version control for tasks
  • Added foreign key relationships and cascading deletes
  • Added validation checks for JSON fields
  • +16/-17 

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    Copy link
    Contributor

    qodo-merge-pro-for-open-source bot commented Dec 19, 2024

    CI Failure Feedback 🧐

    (Checks updated until commit b8cf8ce)

    Action: Typecheck

    Failed stage: Generate openapi code [❌]

    Failed test name: ""

    Failure summary:

    The action failed because the tsp command was not found in the system. The script tried to execute
    tsp compile . in the typespec/ directory, but the TypeSpec compiler (tsp) was not installed or not
    available in the system PATH.

    Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    188:  Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.8/x64
    189:  Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.8/x64
    190:  LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.8/x64/lib
    191:  ##[endgroup]
    192:  + set -e
    193:  + cd typespec/
    194:  + tsp compile .
    195:  scripts/generate_openapi_code.sh: line 10: tsp: command not found
    196:  ##[error]Process completed with exit code 127.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

    Configuration options

    • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
    • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
    • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
    • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
    • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

    See more information about the checks tool in the docs.

    @creatorrr creatorrr changed the title wip(agents-api,memory-store): Tasks queries feat(agents-api,memory-store): Tasks queries Dec 21, 2024
    @whiterabbit1983 whiterabbit1983 marked this pull request as ready for review December 21, 2024 17:28
    @whiterabbit1983 whiterabbit1983 merged commit ba86224 into f/switch-to-pg Dec 21, 2024
    7 of 14 checks passed
    @whiterabbit1983 whiterabbit1983 deleted the f/task-queries branch December 21, 2024 17:28
    Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Debug Code

    There is a debug print statement "UPDATING TIIIIIME" that should be removed before merging to production.

        print("UPDATING TIIIIIME")
    Debug Code

    There are multiple debug print statements that should be removed before merging to production.

            print("data", data)
            print("-" * 10)
            print("objs", objs)
            print("-" * 100)

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    ❌ Changes requested. Reviewed everything up to b8cf8ce in 1 minute and 18 seconds

    More details
    • Looked at 3028 lines of code in 42 files
    • Skipped 0 files when reviewing.
    • Skipped posting 0 drafted comments based on config settings.

    Workflow ID: wflow_4EwwntLjJnlK5R6A


    Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    @@ -238,6 +245,10 @@ def _return_data(rec: list[Record]):
    return obj

    objs: list[ModelT] = [cls(**item) for item in map(transform, data)]
    print("data", data)
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Avoid using print statements for debugging in production code. Consider using a logging framework instead.

    @@ -238,6 +245,10 @@
    return obj

    objs: list[ModelT] = [cls(**item) for item in map(transform, data)]
    print("data", data)
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Remove this print statement to keep the code clean.

    Copy link
    Contributor

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Remove trailing comma in SQL INSERT statement to prevent syntax errors

    Fix SQL syntax error - remove trailing comma after 'input_schema' in INSERT
    statement which can cause issues in some SQL implementations.

    agents-api/agents_api/queries/tasks/patch_task.py [55-56]

    -input_schema,            -- $9
    +input_schema            -- $9
     )
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: Trailing commas in SQL INSERT statements can cause syntax errors in some SQL implementations. This is a critical fix that could prevent runtime failures.

    9
    Improve error handling logic for empty database query results to prevent invalid data from being processed

    Move the results check before appending to all_results to prevent appending empty or
    invalid results. Also, simplify the condition for checking empty results.

    agents-api/agents_api/queries/utils.py [172-184]

     results: list[Record] = await method(
         query, *args, timeout=timeout
     )
     if method_name == "fetchrow":
    -    results = (
    -        [results]
    -        if results is not None
    -        and results.get("bool", False) is not None
    -        and results.get("exists", True) is not False
    -        else []
    -    )
    -
    -if method_name == "fetchrow" and len(results) == 0:
    -    raise asyncpg.NoDataFoundError("No data found")
    +    if results is None or not results:
    +        raise asyncpg.NoDataFoundError("No data found")
    +    results = [results]
     
     all_results.append(results)
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion improves error handling by checking for empty results before appending them, preventing potential issues with invalid data propagation. The simplified logic also makes the code more maintainable.

    8
    Ensure consistent cascade deletion behavior across foreign key constraints

    Add ON DELETE CASCADE to the foreign key constraint fk_user_files_user to ensure
    referential integrity when a user is deleted. This maintains consistency with other
    similar constraints in the schema.

    memory-store/migrations/000005_files.up.sql [65]

    -CONSTRAINT fk_user_files_user FOREIGN KEY (developer_id, user_id) REFERENCES users (developer_id, user_id),
    +CONSTRAINT fk_user_files_user FOREIGN KEY (developer_id, user_id) REFERENCES users (developer_id, user_id) ON DELETE CASCADE,
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: The suggestion maintains schema consistency by adding ON DELETE CASCADE to match other similar constraints, preventing orphaned records and ensuring proper cleanup when users are deleted.

    8
    General
    Remove debug print statement from production code

    Remove the debug print statement 'UPDATING TIIIIIME' as it's not appropriate for
    production code and could clutter logs.

    agents-api/agents_api/queries/tasks/update_task.py [145]

    -print("UPDATING TIIIIIME")
    +# Remove this line entirely
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: Debug print statements should not be present in production code as they can clutter logs and impact performance. This is a valid cleanup suggestion.

    7
    Enforce data integrity by validating JSON object structure in database

    Add a CHECK constraint to validate that the 'output' JSONB field contains valid JSON
    object data, consistent with other JSONB validations in the schema.

    memory-store/migrations/000012_transitions.up.sql [49-51]

     output JSONB,
     task_token TEXT DEFAULT NULL,
     metadata JSONB DEFAULT '{}'::JSONB,
    +CONSTRAINT ct_output_is_object CHECK (output IS NULL OR jsonb_typeof(output) = 'object'),
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: Adding JSON object validation for the output field aligns with the schema's pattern of JSONB field validation and prevents invalid data, though the suggestion correctly handles NULL values which shows good attention to detail.

    7
    Strengthen data integrity by ensuring array columns cannot be set to null even after creation

    Add a NOT NULL constraint to the tool_calls array to ensure data integrity, since
    it's using a default value and has a check constraint.

    memory-store/migrations/000015_entries.up.sql [40]

    -tool_calls JSONB[] NOT NULL DEFAULT '{}'::JSONB[],
    +tool_calls JSONB[] NOT NULL DEFAULT '{}'::JSONB[] CHECK (tool_calls IS NOT NULL),
    • Apply this suggestion
    Suggestion importance[1-10]: 3

    Why: The suggestion is redundant since the column is already declared as NOT NULL. Adding an additional CHECK constraint would not provide meaningful additional protection.

    3

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    3 participants