jimfuqian/BB2-3216 Investigate adding coverage report to tests

[JFU-NAVA-PBC]

JIRA Ticket:
BB2-3216

User Story or Bug Summary:

Part of the quarterly QASP report deals with testing coverage of the code, and at this point, I haven't seen any baseline for our code coverage from the existing tests. We should investigate if any tooling is already in place to provide a code coverage report. If we do have tooling for this, then we need the team to be made aware of how to use it. If we do not, we should propose an approach to add tooling for this, either through Github, SonarQube, etc.

What Does This PR Do?

A python Coverage.py based coverage report integrated with a local sonarqube server and scanner (docker compose based) to demo the recommended BB2 code quality solution.

What Should Reviewers Watch For?

If you're reviewing this PR, please check these things, in particular:

• TODO

To check out the demo locally:

1. Checkout this PR to local
2. Start local sonarqube server (port 9000) by running: docker compose sonarqube up
3. Point browser to http://localhost:9000, login admin/admin
4. From sonarqube console, create a project e.g. BB2 with most settings in default, generate sonarqube token (for use at sonar scanner)
5. Open another term, and run: docker compose coverage, this will run all unit tests with python coverage, a coverage report will be written to current directory, coverage.xml
6. Run sonar scanner for project BB2: docker compose sonardemo
7. If sonardemo service run into error: Unauthorized check token or login/password, and the token in the docker-compose.yml sonardemo bash command line is good, then add -Dsonar.login=admin -Dsonar.password=
8. Go back to sonarqube console, and select project BB2, and navigate all the analysis including coverage.

What Security Implications Does This PR Have?

Submitters should complete the following questionnaire:

• If the answer to any of the questions below is Yes, then here's a link to the associated Security Impact Assessment (SIA), security checklist, or other similar document in Confluence: N/A.
  • Does this PR add any new software dependencies? Yes or No.
  • Does this PR modify or invalidate any of our security controls? Yes or No.
  • Does this PR store or transmit data that was not stored or transmitted before? Yes or No.
• If the answer to any of the questions below is Yes, then please add StewGoin as a reviewer, and note that this PR should not be merged unless/until he also approves it.
  • Do you think this PR requires additional review of its security implications for other reasons? Yes or No.

What Needs to Be Merged and Deployed Before this PR?

This PR cannot be either merged or deployed until the following pre-requisite changes have been fully deployed:

• CMSgov/some_repo#42

Any Migrations?

• Yes, there are migrations
  • The migrations should be run PRIOR to the code being deployed
  • The migrations should be run AFTER the code is deployed
  • There is a more complicated migration plan (downtime, etc)
• No migrations

Submitter Checklist

I have gone through and verified that...:

• This PR is reasonably limited in scope, to help ensure that:
  1. It doesn't unnecessarily tie a bunch of disparate features, fixes, refactorings, etc. together.
  2. There isn't too much of a burden on reviewers.
  3. Any problems it causes have a small "blast radius".
  4. It'll be easier to rollback if that becomes necessary.
• I have named this PR and its branch such that they'll be automatically be linked to the (most) relevant Jira issue, per: https://confluence.atlassian.com/adminjiracloud/integrating-with-development-tools-776636216.html.
• This PR includes any required documentation changes, including README updates and changelog / release notes entries.
• All new and modified code is appropriately commented, such that the what and why of its design would be reasonably clear to engineers, preferably ones unfamiliar with the project.
• All tech debt and/or shortcomings introduced by this PR are detailed in TODO and/or FIXME comments, which include a JIRA ticket ID for any items that require urgent attention.
• Reviews are requested from both:
  • At least two other engineers on this project, at least one of whom is a senior engineer or owns the relevant component(s) here.
  • Any relevant engineers on other projects (e.g. BFD, SLS, etc.).
• Any deviations from the other policies in the DASG Engineering Standards are specifically called out in this PR, above.
  • Please review the standards every few months to ensure you're familiar with them.

[jimmyfagan]

This takes a very long time to run locally, but seems to work pretty well, especially as a starting point. We'll need to probably work with Stephen and maybe others to figure out how to get this into Sonarqube to not have to run this locally, but I really like this POC. We should plan to demo this to the team during sprint demo, and also discuss at our next Eng Sync. Thanks for good work here!

[JFU-NAVA-PBC]

This takes a very long time to run locally, but seems to work pretty well, especially as a starting point. We'll need to probably work with Stephen and maybe others to figure out how to get this into Sonarqube to not have to run this locally, but I really like this POC. We should plan to demo this to the team during sprint demo, and also discuss at our next Eng Sync. Thanks for good work here!

Thanks Jimmy for taking time to review it and trying it out.

for super slow unit tests run (on Mac chip only), logged ticket: BB2-3236