Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add actions db model and caching V0 #980

Merged
merged 2 commits into from
Oct 15, 2024
Merged

add actions db model and caching V0 #980

merged 2 commits into from
Oct 15, 2024

Conversation

wintonzheng
Copy link
Contributor

@wintonzheng wintonzheng commented Oct 15, 2024

Important

Add ActionModel and caching for actions with database migration, client methods, and integration updates.

  • Database:
    • Add actions table in 2024_10_15_0004-f342c424c132_actions_table.py with columns for action_id, action_type, source_action_id, organization_id, workflow_run_id, task_id, step_id, step_order, action_order, executed, reasoning, intention, response, element_id, skyvern_element_hash, skyvern_element_data, action_json, created_at, and modified_at.
    • Add foreign key constraints to organizations, actions, steps, tasks, and workflow_runs.
    • Create indices for organization_id, task_id, step_id, action_id, and source_action_id.
  • Models:
    • Add ActionModel to models.py with fields matching the actions table.
    • Add generate_action_id() to id.py for generating unique action IDs.
  • Database Client:
    • Add create_action(), retrieve_action_plan(), and get_previous_actions_for_task() methods to client.py for handling actions.
  • Caching:
    • Implement action caching in caching.py with retrieve_action_plan() and personalize_actions().
    • Introduce CachedActionPlanError in exceptions.py.
  • Misc:
    • Add DISABLE_MOUSE_MOVEMENT setting to config.py and handle it in agent_functions.py.
    • Update imports and function calls in agent.py to support new action handling.

This description was created by Ellipsis for 136c297. It will automatically update as commits are pushed.

<!-- ELLIPSIS_HIDDEN -->

> [!IMPORTANT]
> Add `ActionModel` and caching for actions with database migration, client methods, and integration updates.
>
>   - **Database Migration**:
>     - Adds `actions` table in `2024_10_15_0004-f342c424c132_actions_table.py` with columns for `action_id`, `action_type`, `source_action_id`, `organization_id`, `workflow_run_id`, `task_id`, `step_id`, `step_order`, `action_order`, `executed`, `reasoning`, `intention`, `response`, `element_id`, `skyvern_element_hash`, `skyvern_element_data`, `action_json`, `created_at`, and `modified_at`.
>     - Adds foreign key constraints to `organizations`, `actions`, `steps`, `tasks`, and `workflow_runs`.
>     - Creates indices for `organization_id`, `task_id`, `step_id`, `action_id`, and `source_action_id`.
>   - **Models**:
>     - Adds `ActionModel` to `models.py` with fields matching the `actions` table.
>     - Adds `generate_action_id()` to `id.py` for generating unique action IDs.
>   - **Database Client**:
>     - Adds `create_action()`, `retrieve_action_plan()`, and `get_previous_actions_for_task()` methods to `client.py` for handling actions.
>   - **Caching**:
>     - Implements action caching in `caching.py` with `retrieve_action_plan()` and `personalize_actions()`.
>     - Introduces `CachedActionPlanError` in `exceptions.py`.
>   - **Misc**:
>     - Adds `DISABLE_MOUSE_MOVEMENT` setting to `config.py` and handles it in `agent_functions.py`.
>     - Updates imports and function calls in `agent.py` to support new action handling.
>
> <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=Skyvern-AI%2Fskyvern-cloud&utm_source=github&utm_medium=referral)<sup> for 590e24ca94207380edd0f154e7aea3c1255bcf12. It will automatically update as commits are pushed.</sup>

<!-- ELLIPSIS_HIDDEN -->
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 7bb8f62 in 47 seconds

More details
  • Looked at 1277 lines of code in 18 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. skyvern/forge/sdk/api/crypto.py:1
  • Draft comment:
    The calculate_sha256 function is defined twice in different files with similar functionality. Consider consolidating it into a single utility function to avoid redundancy.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The calculate_sha256 function is defined twice in different files with similar functionality. It would be better to have a single utility function for calculating SHA256 to avoid redundancy.
2. skyvern/forge/sdk/api/files.py:113
  • Draft comment:
    The calculate_sha256 function is defined twice in different files with similar functionality. Consider consolidating it into a single utility function to avoid redundancy.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The calculate_sha256 function is defined twice in different files with similar functionality. It would be better to have a single utility function for calculating SHA256 to avoid redundancy.
3. skyvern/webeye/scraper/scraper.py:14
  • Draft comment:
    The calculate_sha256 function is defined twice in different files with similar functionality. Consider consolidating it into a single utility function to avoid redundancy.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The calculate_sha256 function is defined twice in different files with similar functionality. It would be better to have a single utility function for calculating SHA256 to avoid redundancy.

Workflow ID: wflow_s0c2C3zkZBu2bXbG


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 7bb8f62 in 1 minute and 15 seconds

More details
  • Looked at 1277 lines of code in 18 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. skyvern/webeye/actions/caching.py:39
  • Draft comment:
    Ensure that previous_actions is not longer than cached_actions to prevent IndexError. Consider adding a check or handling this case.
  • Reason this comment was not posted:
    Comment did not seem useful.
2. skyvern/webeye/actions/actions.py:261
  • Draft comment:
    Use action.get('action_type', '').upper() to avoid KeyError if action_type is missing.
  • Reason this comment was not posted:
    Comment was on unchanged code.
3. skyvern/webeye/actions/handler.py:268
  • Draft comment:
    Consider raising an exception for unsupported action types to ensure proper error handling.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The current implementation uses ActionFailure to handle unsupported action types, which is consistent with the rest of the error handling in the function. Raising an exception would change the flow of the program and might not be compatible with the expected return type of the function. The comment does not provide strong evidence that the current approach is incorrect.
    I might be missing the broader context of how errors are generally handled in this codebase. If exceptions are preferred over ActionFailure in other parts of the code, the comment might be valid.
    The use of ActionFailure seems intentional and consistent with the rest of the function's error handling. Without evidence that exceptions are preferred, the comment seems speculative.
    The comment should be deleted as it suggests a change without strong evidence that the current implementation is incorrect.
4. skyvern/webeye/utils/dom.py:490
  • Draft comment:
    Consider raising an exception if the hash value is not found instead of defaulting to an empty string, if the hash is critical for further operations.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is speculative, as it suggests a change based on the assumption that the hash value is critical. The current implementation defaults to an empty string, which might be intentional. Without strong evidence that the hash value is critical, the comment does not meet the criteria for a necessary code change.
    I might be missing the context of how critical the hash value is for the operations. However, the comment is speculative and does not provide strong evidence for a required change.
    The comment does not provide evidence that the hash value is critical, and the current implementation might be intentional. Without clear evidence, the comment should be removed.
    Remove the comment as it is speculative and does not provide strong evidence for a required code change.

Workflow ID: wflow_KmZMFzbirgYJbEWQ


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.


# if there are no elements, fail the scraping
if not elements:
raise Exception("No elements found on the page")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider raising a custom exception with a descriptive message when no elements are found on the page.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 136c297 in 38 seconds

More details
  • Looked at 87 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. alembic/versions/2024_10_15_1903-137eee1d3b3e_actions_table.py:35
  • Draft comment:
    Consider specifying a length for the status column to improve storage efficiency and performance.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is suggesting a potential improvement in terms of database performance and storage efficiency, which is a valid consideration. However, the necessity of this change depends on the specific database being used and the expected data size. Without more context, it's hard to determine if this is a critical issue.
    The comment might be considered speculative since it assumes that specifying a length will improve performance without knowing the specific database or use case. It might not be a necessary change for all scenarios.
    While the comment could be seen as speculative, it does point out a potential area for optimization that is generally applicable across many database systems. It is a common best practice to specify lengths for string columns when possible.
    The comment highlights a potential improvement that is generally considered a best practice. It is actionable and clear, so it should be kept.

Workflow ID: wflow_cgYTkG0DR2VeH97g


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@wintonzheng wintonzheng merged commit 9048cdf into main Oct 15, 2024
2 checks passed
@wintonzheng wintonzheng deleted the shu/actions_in_db branch October 15, 2024 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant