notes: Add test-coverage

Author: tony

notes/test-coverage.md

@@ -0,0 +1,237 @@
+# VCSPull Test Coverage Checklist
+
+This document provides a comprehensive checklist of test coverage for the VCSPull codebase, identifying common use cases, uncommon scenarios, and edge cases that should be tested to ensure robust functionality.
+
+## Core Modules and Their Testing Priorities
+
+### 1. Configuration Management (config.py, _internal/config_reader.py)
+
+#### Common Cases:
+- [ ] **Config File Loading:** Loading valid YAML/JSON files from common locations
+  - [ ] Home directory (~/.vcspull.yaml, ~/.vcspull.json)
+  - [ ] XDG config directory
+  - [ ] Project-specific config files
+- [ ] **Directory Expansion:** Resolving paths with tilde (~) and environment variables
+- [ ] **Basic Configuration Format:** Standard repository declarations with required fields
+- [ ] **Multiple Repositories:** Configurations with multiple repositories in different paths
+- [ ] **Filtering Repositories:** Basic pattern matching for repository names
+- [ ] **Repository Extraction:** Converting raw configs to normalized formats
+
+#### Uncommon Cases:
+- [ ] **Deeply Nested Configurations:** Multiple levels of directory nesting in config
+- [ ] **Configuration Merging:** Combining multiple configuration files
+- [ ] **Duplicate Detection:** Identifying and handling duplicate repositories
+- [ ] **Conflicting Configurations:** When the same repository is defined differently in multiple files
+- [ ] **Relative Paths:** Config files using relative paths that need resolution
+- [ ] **Custom Config Locations:** Loading from non-standard locations
+
+#### Edge Cases:
+- [ ] **Empty Configuration Files:** Files with empty content or only comments
+- [ ] **Malformed YAML/JSON:** Syntax errors in configuration files
+- [ ] **Circular Path References:** Directory structures with circular references
+- [ ] **Very Large Configurations:** Performance with hundreds of repositories
+- [ ] **Case Sensitivity Issues:** Path case differences between config and filesystem
+- [ ] **Unicode and Special Characters:** In repository names, paths, and URLs
+- [ ] **Inaccessible Paths:** Referenced paths that exist but are not accessible
+- [ ] **Path Traversal Attempts:** Paths attempting to use "../" to escape sandboxed areas
+- [ ] **Missing Config Files:** Behavior when specified config files don't exist
+- [ ] **Mixed VCS Types:** Configurations mixing git, hg, and svn repositories
+- [ ] **Invalid URLs:** URL schemes that don't match the specified VCS type
+
+### 2. Validation (validator.py, schemas.py)
+
+#### Common Cases:
+- [ ] **Basic Schema Validation:** Checking required fields in configurations
+- [ ] **VCS Type Validation:** Validating supported VCS types (git, hg, svn)
+- [ ] **URL Validation:** Basic validation of repository URLs
+- [ ] **Path Validation:** Checking that paths are valid
+- [ ] **Git Remote Validation:** Validating git remote configurations
+
+#### Uncommon Cases:
+- [ ] **Nested Validation Errors:** Multiple validation issues in nested structures
+- [ ] **URL Scheme Mismatches:** When URL scheme doesn't match the VCS type
+- [ ] **Advanced URL Validation:** SSH URLs, usernames in URLs, port specifications
+- [ ] **Custom Fields Validation:** Handling of non-standard fields in configs
+- [ ] **Shell Command Validation:** Validating shell commands in configs
+
+#### Edge Cases:
+- [ ] **Pydantic Model Conversion:** Converting between raw and validated models
+- [ ] **Partial Configuration Validation:** Validating incomplete configurations
+- [ ] **Deeply Nested Errors:** Validation errors in deeply nested structures
+- [ ] **Custom Protocol Handling:** git+ssh://, git+https://, etc.
+- [ ] **Invalid Characters:** Non-printable or control characters in fields
+- [ ] **Very Long Field Values:** Fields with extremely long values
+- [ ] **Mixed Case VCS Types:** "Git" vs "git" vs "GIT"
+- [ ] **Conflicting Validation Rules:** When multiple validation rules conflict
+- [ ] **Empty vs. Missing Fields:** Distinction between empty and missing fields
+- [ ] **Type Coercion Issues:** When field values are of unexpected types
+- [ ] **Invalid URL Formats by VCS Type:** URLs that are valid in general but invalid for specific VCS
+
+### 3. CLI Interface (cli/__init__.py, cli/sync.py)
+
+#### Common Cases:
+- [ ] **Basic CLI Invocation:** Running commands with minimal arguments
+- [ ] **Repository Filtering:** Using patterns to select repositories
+- [ ] **Config File Specification:** Using custom config files
+- [ ] **Default Behaviors:** Running with default options
+- [ ] **Help Command:** Displaying help information
+- [ ] **Version Display:** Showing version information
+
+#### Uncommon Cases:
+- [ ] **Multiple Filters:** Using multiple inclusion/exclusion patterns
+- [ ] **Interactive Mode:** CLI behavior in interactive mode
+- [ ] **Multiple Config Files:** Specifying multiple config files
+- [ ] **Special Output Formats:** JSON, detailed, etc.
+- [ ] **Custom Working Directory:** Running from non-standard working directories
+- [ ] **Verbosity Levels:** Different verbosity settings
+
+#### Edge Cases:
+- [ ] **Invalid Arguments:** Handling of invalid command-line arguments
+- [ ] **Output Redirection:** Behavior when stdout/stderr are redirected
+- [ ] **Terminal vs. Non-Terminal:** Behavior in different terminal environments
+- [ ] **Signal Handling:** Response to interrupts and other signals
+- [ ] **Unknown Commands:** Behavior with non-existing commands
+- [ ] **Very Long Arguments:** Command line arguments with extreme length
+- [ ] **Unicode in CLI Arguments:** International characters in arguments
+- [ ] **Permission Issues:** Running with insufficient permissions
+- [ ] **Environment Variable Overrides:** CLI behavior with environment variables
+- [ ] **Parallel Execution:** Running multiple commands in parallel
+
+### 4. Repository Operations (libvcs interaction)
+
+#### Common Cases:
+- [ ] **Repository Cloning:** Basic cloning of repositories
+- [ ] **Repository Update:** Updating existing repositories
+- [ ] **Remote Management:** Adding/updating remotes for Git
+- [ ] **Status Checking:** Checking repository status
+- [ ] **Success and Error Handling:** Managing operation outcomes
+
+#### Uncommon Cases:
+- [ ] **Repository Authentication:** Cloning/updating repos requiring auth
+- [ ] **Custom Remote Configurations:** Non-standard remote setups
+- [ ] **Repository Hooks:** Pre/post operation hooks
+- [ ] **Shell Commands:** Executing shell commands after operations
+- [ ] **Repository Recovery:** Recovering from failed operations
+
+#### Edge Cases:
+- [ ] **Network Failures:** Behavior during network interruptions
+- [ ] **Interrupted Operations:** Handling of operations interrupted mid-way
+- [ ] **Repository Corruption:** Dealing with corrupted repositories
+- [ ] **Large Repositories:** Performance with very large repositories
+- [ ] **Repository Lock Files:** Handling existing lock files
+- [ ] **Concurrent Operations:** Multiple operations on the same repository
+- [ ] **Shallow Clones:** Behavior with shallow clone operations
+- [ ] **Submodule Handling:** Repositories with submodules
+- [ ] **Unknown VCS Versions:** Operating with uncommon VCS versions
+- [ ] **Custom Protocol Handlers:** git+ssh://, svn+https://, etc.
+- [ ] **Path Collisions:** When different configurations target the same path
+
+### 5. Utilities and Helpers (util.py, log.py)
+
+#### Common Cases:
+- [ ] **Path Manipulation:** Basic path operations
+- [ ] **Dictionary Updates:** Merging and updating configuration dictionaries
+- [ ] **Logging Configuration:** Basic logging setup and usage
+- [ ] **Process Execution:** Running external commands
+
+#### Uncommon Cases:
+- [ ] **Complex Path Resolution:** Resolving complex path references
+- [ ] **Advanced Logging:** Logging with different levels and formats
+- [ ] **Process Timeouts:** Handling command execution timeouts
+- [ ] **Environment Variable Expansion:** In various contexts
+
+#### Edge Cases:
+- [ ] **Path Edge Cases:** Unicode, very long paths, special characters
+- [ ] **Dictionary Merging Conflicts:** When merge keys conflict
+- [ ] **Logging Under Load:** Behavior with high-volume logging
+- [ ] **Process Execution Failures:** When commands fail or return errors
+- [ ] **Environment with Special Characters:** Environment variables with unusual values
+- [ ] **Shell Command Injection Prevention:** Security of process execution
+- [ ] **Resource Limitations:** Behavior under resource constraints
+
+## Pydantic Model Testing
+
+As part of the transition to Pydantic models, these specific areas need thorough testing:
+
+### Common Cases:
+- [ ] **Model Creation:** Creating models from valid data
+- [ ] **Model Validation:** Basic validation of required fields
+- [ ] **Model Serialization:** Converting models to dictionaries
+- [ ] **Field Type Coercion:** Automatic type conversion for compatible types
+
+### Uncommon Cases:
+- [ ] **Model Inheritance:** Behavior of model inheritance
+- [ ] **Custom Validators:** Advanced field validators
+- [ ] **Model Composition:** Models containing other models
+- [ ] **Validation Error Handling:** Managing and reporting validation errors
+
+### Edge Cases:
+- [ ] **Conversion Between Raw and Validated Models:** Edge cases in model conversion
+- [ ] **Circular References:** Handling models with circular references
+- [ ] **Optional vs. Required Fields:** Behavior with different field requirements
+- [ ] **Default Values:** Complex default value scenarios
+- [ ] **Union Types:** Fields accepting multiple types
+- [ ] **Field Constraints:** Min/max length, regex patterns, etc.
+- [ ] **Custom Error Messages:** Override of validation error messages
+- [ ] **JSON Schema Generation:** Accuracy of generated schemas
+- [ ] **Recursive Models:** Self-referential model structures
+- [ ] **Discriminated Unions:** Type discrimination in unions
+
+## Data-Driven and Property-Based Testing Opportunities
+
+### Property-Based Testing:
+- [ ] **Configuration Structure Invariants:** Properties that should hold for all valid configs
+- [ ] **Model Conversion Roundtrips:** Converting between models and back preserves data
+- [ ] **Path Normalization:** Properties of normalized paths
+- [ ] **URL Parsing:** Properties of parsed and validated URLs
+- [ ] **Repository Configuration Consistency:** Internal consistency of repository configs
+
+### Data Generation Strategies:
+- [ ] **Random Valid Configurations:** Generating syntactically valid configurations
+- [ ] **Random Invalid Configurations:** Generating configurations with specific issues
+- [ ] **Repository URL Generation:** Creating varied repository URLs
+- [ ] **Path Generation:** Creating diverse filesystem paths
+- [ ] **VCS Type Combinations:** Various combinations of VCS types and configurations
+
+## Test Infrastructure Improvements
+
+### Fixtures:
+- [ ] **Repository Fixtures:** Pre-configured repositories of different types
+- [ ] **Configuration Fixtures:** Sample configurations of varying complexity
+- [ ] **File System Fixtures:** Mock file systems with different characteristics
+- [ ] **Network Fixtures:** Mock network responses for repository operations
+- [ ] **VCS Command Fixtures:** Mock VCS command execution
+
+### Mocking:
+- [ ] **File System Mocking:** Simulating file system operations
+- [ ] **Network Mocking:** Simulating network operations
+- [ ] **Process Execution Mocking:** Simulating command execution
+- [ ] **Time Mocking:** Controlling time-dependent operations
+
+### Test Categories:
+- [ ] **Unit Tests:** Testing individual functions and methods
+- [ ] **Integration Tests:** Testing interactions between components
+- [ ] **End-to-End Tests:** Testing full workflows
+- [ ] **Property Tests:** Testing invariant properties
+- [ ] **Performance Tests:** Testing operation speed and resource usage
+- [ ] **Security Tests:** Testing security properties
+
+## Test Coverage Goals
+
+### Overall Coverage Targets:
+- [ ] **High-Risk Modules:** 95%+ coverage (config.py, validator.py)
+- [ ] **Medium-Risk Modules:** 90%+ coverage (CLI modules, schema modules)
+- [ ] **Low-Risk Modules:** 80%+ coverage (utility modules)
+
+### Coverage Types:
+- [ ] **Statement Coverage:** Executing all statements in the code
+- [ ] **Branch Coverage:** Executing all branches in the code
+- [ ] **Condition Coverage:** Testing all boolean sub-expressions
+- [ ] **Path Coverage:** Testing all possible paths through the code
+
+### Functional Coverage:
+- [ ] **Configuration Loading:** 100% of configuration loading code paths
+- [ ] **Validation:** 100% of validation code paths
+- [ ] **Repository Operations:** 95% of operation code paths
+- [ ] **CLI Interface:** 90% of CLI code paths
+- [ ] **Error Handling:** 95% of error handling code paths