Introduce test data optimization ADR #700
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
18 tasks
|
#690 |
7 tasks
aj-stein-gsa
approved these changes
Sep 20, 2024
| ``` | ||
| We have one constraint, `information-type-has-categorization`, that checks the existence of the `categorization` element. In the fail case, we need to remove this `categorization` block from the invalid test data file entirely in order to make the fail case fail successfully (categorization element doesn't exist). We have another constraint, `information-type-system`, that checks if the `system` attribute is a valid value. In the fail case, the `information-type-system` constraint cannot check the `system` attribute because the entire `categorization` element is removed to satisfy the `information-type-has-categorization` constraint fail case. This conflict makes it impossible for both of these constraints to share the same invalid test data file when validating the constraints. | ||
|
|
||
| ## Possible Solutions |
There was a problem hiding this comment.
Non-blocking: one minor suggestion, show the alternative:
Suggested change
| ## Possible Solutions | |
| ## Possible Solutions | |
| 1. Do nothing, continue to maintain test architecture as-is. |
Comment on lines
+23
to
+24
| Solution 1: | ||
| - Longer test run time. |
There was a problem hiding this comment.
Non-blocking: one minor comment, for comparison.
Suggested change
| Solution 1: | |
| - Longer test run time. | |
| Solution 1: | |
| - Current average test execution time is maintained. | |
| - Test suite cannot be realistically enhanced whenever there is a conflicting requirement, which will be often. | |
| - Development and maintenance of individual tests and overall test suite will slow and eventually halt. | |
| Solution 2: | |
| - Longer test run time. | |
| - Developers can continue current velocity of test development or increase, but not stop without this workaround. | |
| - The test architecture achieves meaningful flexibility to permit development and testing of conflicting test requirements simultaneously. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Committer Notes
This ADR pertains to the refinement of the method used for employing the test data files 'ssp-all-VALID.xml' and 'ssp-all-INVALID.xml' in the constraint validation tool.
{Please provide a description of what this PR accomplishes. Be sure to reference any issues addressed. If the PR is a work-in-progress submitted for early review, please submit the PR as a draft PR using the "Draft pull request" dropdown.}
All Submissions:
By submitting a pull request, you are agreeing to provide this contribution under the CC0 1.0 Universal public domain dedication.