Fix functional testing and cached output errors

[schrolla]

🗣 Description

This request contains updates to fix failures when running tests that modify configuration setting export results prior to testing. Specifically this update:

• Makes sure that Invoke-SCuBACached outputs CSV results for only the configured products
• Modified provider settings file is output to the correct filename
• The filepath of the export settings file is correctly specified when writing to disk
• Modifies two MS.SHAREPOINT.2.1v1 test pre-conditions to ensure strict ordering

💭 Motivation and context

Some testing requires manual changes to represent uncommon or undesirable test results. Doing so means the functional test harness may intercept and change those results before running a test. The bug was preventing the modified results from being used in the tests and causing tests to fail when they should pass. We want tests to accurately report their results.

Closes #1328
Closes #1329
Closes #1333
Closes #1337

🧪 Testing

To test this PR, either run the nightly functional test pipeline to validate that all tests using cached results (UpdateProviderExport in preconditions) pass as expected, particularly MS.AAD.3.2v1 and MS.AAD.3.3v1 tests. Or run the functional tests using the ScubaGearCheck.ps1 script as follows (note: this is for internal dev use only and requires additional configuration):

ScubaGearCheck.ps1 -Baseline aad -Tenant 2 -UserAuth:$false -Filter "MS.AAD.3.[23]v1*"

All tests should pass.

Also test via:
Create output with merged json using the default invoke options, then run the following:
Invoke-ScubaCached -OutPath .\path-to-output-folder-from-original-run\ -ExportProvider $false

Also, when running functional tests verify that this error is not present in Defender tests:

WARNING: Error involving the creation of CSV version of output. See the exception message for more details: Cannot find
path 'D:\a\ScubaGear\ScubaGear\repo\M365BaselineConformance_2024_09_24_11_43_30\IndividualReports\AADReport.json' 
because it does not exist.

✅ Pre-approval checklist

• This PR has an informative and human-readable title.
• PR targets the correct parent branch (e.g., main or release-name) for merge.
• Changes are limited to a single goal - eschew scope creep!
• Changes are sized such that they do not touch excessive number of files.
• All future TODOs are captured in issues, which are referenced in code comments.
• These code changes follow the ScubaGear content style guide.
• Related issues these changes resolve are linked preferably via closing keywords.
• All relevant type-of-change labels added.
• All relevant project fields are set.
• All relevant repo and/or project documentation updated to reflect these changes.
• Unit tests added/updated to cover PowerShell and Rego changes.
• Functional tests added/updated to cover PowerShell and Rego changes.
• All relevant functional tests passed.
• All automated checks (e.g., linting, static analysis, unit/smoke tests) passed.

✅ Pre-merge checklist

• PR passed smoke test check.

• Feature branch has been rebased against changes from parent branch, as needed

  Use Rebase branch button below or use this reference to rebase from the command line.

• Resolved all merge conflicts on branch

• Notified merge coordinator that PR is ready for merge via comment mention

✅ Post-merge checklist

• Feature branch deleted after merge to clean up repository.
• Verified that all checks pass on parent branch (e.g., main or release-name) after merge.

[buidav]

There still seems to be errors with the functional tests in Defender, Entra, and SharePoint that all seem unconnected to each other and any other pending PRs.

[schrolla]

The errors we expect to see at this point are 9 test failures for Entra in MS.AAD.7.x related to Get-PrivilegedUser. There is another PR in the works that will resolve those.
The SharePoint error in this run is caused by a race condition where the Selenium server doesn't start in time for the first test, causing it to fail. It is a separate issue to address noted in #1338.

As for Defender, some of the ones in this run are due to issues with the webdriver timing out. A couple look like actual failures. I will look into those a bit more.

There still seems to be errors with the functional tests in Defender, Entra, and SharePoint that all seem unconnected to each other and any other pending PRs.

[buidav]

The errors we expect to see at this point are 9 test failures for Entra in MS.AAD.7.x related to Get-PrivilegedUser. There is another PR in the works that will resolve those. The SharePoint error in this run is caused by a race condition where the Selenium server doesn't start in time for the first test, causing it to fail. It is a separate issue to address noted in #1338.

As for Defender, some of the ones in this run are due to issues with the webdriver timing out. A couple look like actual failures. I will look into those a bit more.

There still seems to be errors with the functional tests in Defender, Entra, and SharePoint that all seem unconnected to each other and any other pending PRs.

Will approve then for these current changes.

[schrolla]

@nanda-katikaneni Ready for merge.

[schrolla]

Ran another functional test run after rebasing with AAD fixes, and none of the Entra tests fail now. SharePoint is also passing with fixes. Defender individual tests still fail, and I will investigate, although it's possible the concurrency workflow PR may prevent errors there since this branch still runs multiple tests in parallel on the same tenant. Either way, the remaining test issues are not relevant to this particular branches changes.