Refactor test data for constraints for conflicting test requirements

[aj-stein-gsa]

This is a ...

improvement - something could be better

This relates to ...

• the FedRAMP OSCAL Registry
• the FedRAMP OSCAL baselines
• the Guide to OSCAL-based FedRAMP Content
• the Guide to OSCAL-based FedRAMP System Security Plans (SSP)
• the Guide to OSCAL-based FedRAMP Security Assessment Plans (SAP)
• the Guide to OSCAL-based FedRAMP Security Assessment Results (SAR)
• the Guide to OSCAL-based FedRAMP Plan of Action and Milestones (POA&M)
• the FedRAMP SSP OSCAL Template (JSON or XML Format)
• the FedRAMP SAP OSCAL Template (JSON or XML Format)
• the FedRAMP SAR OSCAL Template (JSON or XML Format)
• the FedRAMP POA&M OSCAL Template (JSON or XML Format)
• the FedRAMP OSCAL Validations

User Story

As a FedRAMP developer, to better add or change tests for the constraints, I would like a tests to be able to effectively run with the right amount of test data, especially when two or more tests have conflicting test requirements that cannot exist (or not exist) in a singular test data file.

Goals

• Make tests and test data more manageable
• Make tests and their state operate independently
• Limit cognitive load and minimize conflicts (cognitive, tooling issues a la git merge) to reduce friction on development and maintenance of rules

Dependencies

N/A

Acceptance Criteria

• An architectural decision record (ADR) is made to explain what will stay the same or change given dev team discussions on the topic
• Tooling and CI/CD change issues will be added to the list below once the ADR has been review and team plans completion steps.
  • Adjust dev test tool to scaffold independent content files, not just modify ALL-VALID.xml and ALL-INVALID.xml in Refactor test data for constraints for conflicting test requirements #691
  • TBD example

Once the second sublist is complete, this architecture decision will be complete and read to merge.

Other information

Today, the FR Automation Team devs had a meeting and posed an interesting challenge with the tests as they exist today in the feature branch, soon to be develop branch tests. When performing negative tests when an assembly or field should not exist, and an improper value must be tested, having all the tests target a single exhaustive list of invalid test vectors in one large SSP file exhibits certain challenges. We have decided to review this approach, consider alternative options, and act accordingly.

[aj-stein-gsa]

@Gabeblis thanks for volunteering to pick this one up. I hope the goals and background are clear for the ADR portion of this.

[Gabeblis]

@Gabeblis thanks for volunteering to pick this one up. I hope the goals and background are clear for the ADR portion of this.

Thank you!