Fix mypy #14
Merged
Fix mypy #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add mypy library to `pyproject.toml` - Fix some configuration of mypy inside `pyproject.toml` that were not exist
- not necessary
|
Error left: 63 errors in 19 files. |
…tions. Clean up test cases for MongoDB, MSSQL, MySQL, and PostgreSQL performance measurements. Fix minor issues in test scripts and ensure consistency across test data files.
- Updated mypy version in pyproject.toml from 1.8.0 to 1.14.0. - Enhanced README.md by improving headings and adding a badge for Trunk version. - Refactored code in various files to improve type hinting and ensure better handling of optional parameters. - Adjusted the logic in RdbmsClient and DataSourceUtil to handle pagination and data retrieval more robustly. - Cleaned up the demo condition info.toml file by removing extra spaces in the title. - Updated environment setup script for better readability. These changes aim to improve the overall maintainability and clarity of the project.
…ing and optional parameter management (mypy fixes) - Updated DataSourceUtil to check for statement type when accessing memstore. - Modified CSVExporter to handle empty fieldnames and added a check for buffer file existence before writing. - Enhanced UnifiedBufferedExporter with better buffer file initialization and metadata update logic, including retry mechanisms. - Improved XMLExporter to ensure buffer file checks are in place before writing data. These changes enhance the robustness and maintainability of the data export process.
…ement start times and task exporters - Added a dictionary to track statement start times in the Context class. - Introduced a dictionary for task exporters in the SetupContext class, along with corresponding getter and setter methods. - Updated references in generate_task.py to utilize the new task exporters property instead of the private attribute. - Cleaned up commented-out code in generate_task.py and task_util.py for better readability. These changes improve the structure and maintainability of the context management system.
- Updated the _get_entity method to accept a more general Context type instead of SetupContext, enhancing flexibility. - Adjusted the execute method to utilize the new Context type, allowing for broader context compatibility. - Refined logic for determining variable prefixes and suffixes by ensuring proper context type checks, improving robustness. These changes enhance the maintainability and adaptability of the VariableTask class within the context management system.
…pe consistency - Updated the execute methods in KeyTask and NestedKeyTask to accept a more general Context type instead of the specific GenIterContext, enhancing flexibility. - Added type checks to ensure that certain operations are only performed when the context is of type GenIterContext, improving robustness. - Refined logic for adding current product fields based on context type, ensuring better compatibility with different context scenarios. These changes enhance the maintainability and adaptability of the task classes within the context management system.
…and optional parameters - Updated the `get_cyclic_data` method in `DatabaseClient` to allow `pagination` to be optional. - Enhanced `SetupContext` to provide default values for `default_separator` and `default_locale` if not specified. - Refined type hints in `KeyVariableTask` for `_generator` and improved handling of optional parameters in various methods. - Adjusted `VariableTask` to ensure `statement.cyclic` defaults to `False` when not provided. These changes enhance type safety and flexibility across the database client and task management system.
…improved features and compatibility
…and SQLAlchemy 2.x compatibility - Updated SQLAlchemy queries to align with version 2.x standards, enhancing compatibility and performance. - Refined the `consume` method in `DatabaseExporter` to improve type hints and handle optional metadata. - Enhanced pagination handling in `RdbmsClient` methods to ensure robust data retrieval. - Improved error handling in sequence number management, ensuring better transaction safety and clarity. These changes enhance the maintainability and adaptability of the database client and exporter components.
Co-authored-by: Dang Ly <[email protected]>
- Cleaned up mypy-related comments and issues in various task classes, enhancing type consistency and clarity. - Updated type hints in `datamimic_ce/__init__.py` and `datamimic_ce/data_sources/data_source_util.py` for better type safety. - Removed unnecessary `continue-on-error` lines in GitHub Actions workflow. - Improved handling of required environment variables and pagination in data processing functions. These changes improve code maintainability and type safety throughout the project.
…ation - Added command to install mypy types in the GitHub Actions workflow for improved type checking. - Refined import statements in `generate_task.py` by removing unnecessary imports and ensuring proper usage of `OrderedDict`. These changes improve type safety and streamline the development workflow.
- Upgraded psutil dependency to version 6.1.0 for improved performance and compatibility. - Added pandas-stubs, types-psutil, and types-xmltodict to dev-dependencies for enhanced type checking. - Removed the mypy types installation command from the GitHub Actions workflow to streamline the process. - Introduced new pytest configuration options for better async test handling. These changes enhance type safety, improve dependency management, and optimize the development workflow.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Begin with: 407 errors in 88 files (checked 230 source files)