Ref #681: Improve quality standards (#757)

* Add missing files

* Add logging to heartbeat checks

* Document how to pass arguments to pytest

* Mention Acoustic stage pod

* Standard README

* Mention access request in email channel

CODE_OF_CONDUCT.md

@@ -0,0 +1,15 @@
+# Community Participation Guidelines
+
+This repository is governed by Mozilla's code of conduct and etiquette guidelines. 
+For more details, please read the
+[Mozilla Community Participation Guidelines](https://www.mozilla.org/about/governance/policies/participation/). 
+
+## How to Report
+For more information on how to report violations of the Community Participation Guidelines, please read our '[How to Report](https://www.mozilla.org/about/governance/policies/participation/reporting/)' page.
+
+<!--
+## Project Specific Etiquette
+
+In some cases, there will be additional project etiquette i.e.: (https://bugzilla.mozilla.org/page.cgi?id=etiquette.html).
+Please update for your project.
+-->

CONTRIBUTING.md

@@ -0,0 +1,13 @@
+First off, thanks for taking the time to contribute! ❤️
+
+All types of contributions are encouraged and valued.
+
+Before doing so, here are a few guidelines:
+
+* You agree to license your contributions under the project [license](LICENSE).
+* Use pull-requests early so it's open for discussion, even if your
+  contribution isn't ready yet.
+* All pull requests should include tests, as they help us avoid regressions in
+  our code.
+* A pull-request adding functionality should also update the documentation
+  accordingly.

README.md

@@ -1,51 +1,23 @@
 # ConTact Management System (CTMS)
 
-While the app is running, interactive API documentation can found at the following relative paths: /docs, /redoc.
-
-OpenApiSpec(OAS) formatted in JSON can be found at the following path: /openapi.json
-
----
-
-[View All Docs](docs/README.md)
-
----
-
-## Prerequisites
-
-Please read the [Developer Setup documentation](docs/developer_setup.md) before developing \
-for this project to ensure correct environment setup.
-
----
-## Project Structure
-
-The project is structured with the following in mind:
-
-- [bin/*](bin/)
-    - Some scripts that have proven useful within the CTMS ecosystem
-- [docs/*](docs/)
-    - Documentation to guide others around the project interactions
-- [ctms/*](ctms/)
-    - Application logic lives within this directory
-    - [bin/*](ctms/bin/)
-        - Scripts intended for background machinery
-    - [schemas/*](ctms/schemas/)
-        - Pydantic Models for Data Modeling and Contract Validation
-- [migrations/*](migrations/)
-    - Alembic migrations that act as a changelog or version control system for implementing DB changes in an ordered fashion
-- [tests/unit/*](test/unit/)
-    - Test suite using pytest
-
----
-## Important Files
-
-Below are some files that are worth making note of:
-- [MAKEFILE](Makefile)
-    - Enabling commands such as: make {build | lint | setup | start | test | shell | db-only}
-- [ctms/app.py](ctms/app.py)
-    - FastAPI handling of HTTP Requests and routing to services
-- [ctms/bin/acoustic_sync.py](ctms/bin/acoustic_sync.py)
-    - Background job for synchronizing pending records to Acoustic
-- [ctms/config.py](ctms/config.py)
-    - Environment variables are initialized here
-- [ctms/models.py](ctms/models.py)
-    - SQLAlchemy models for ORM tool
+![Status Sustain](https://img.shields.io/badge/Status-Sustain-green)
+
+*CTMS* is an internal service at Mozilla in charge of managing marketing contacts.
+It consists of a REST API and a background synchronization process.
+
+## Usage
+
+Run the app with `make start`. Check [developer docs](docs/developer_setup.md) for setup.
+
+While the app is running, interactive API documentation is available at:
+
+- http://localhost:8000/docs
+- http://localhost:8000/redoc
+
+OpenApiSpec(OAS) formatted in JSON can be found at the following path: `/openapi.json`.
+
+## [Documentation](docs/README.md)
+
+## License
+
+*CTMS* is licensed under the MPLv2. See the `LICENSE` file for details.

SECURITY.md

@@ -0,0 +1,16 @@
+# Security Policy
+
+Mozilla has a [well-defined process for handling security vulnerabilities](https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/) based around responsible disclosure.
+
+## Supported Versions
+
+| Version | Supported          |
+| ------- | ------------------ |
+| v2.x.x  | :white_check_mark: |
+| < v2.0  | :x:                |
+
+## Reporting a Vulnerability
+
+If you believe you have found a security vulnerability, you should visit the [Mozilla bug bounty program](https://www.mozilla.org/en-US/security/bug-bounty/) for information on how to submit them.
+
+[This Bugzilla template](https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=--&bug_status=NEW&bug_type=defect&cf_fx_iteration=---&cf_fx_points=---&component=SRE&contenttypemethod=list&contenttypeselection=text%2Fplain&defined_groups=1&filed_via=standard_form&flag_type-4=X&flag_type-607=X&flag_type-674=X&flag_type-800=X&flag_type-803=X&flag_type-936=X&form_name=enter_bug&groups=releng-security&groups=mozilla-employee-confidential&groups=partner-confidential&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Infrastructure%20%26%20Operations&rep_platform=Unspecified&target_milestone=---&version=unspecified) will help you report a security vulnerability directly to our SRE team.

ctms/monitor.py

@@ -1,6 +1,7 @@
 """Application monitoring and health utilities"""
 
 import json
+import logging
 import os.path
 import time
 from datetime import datetime, timezone
@@ -11,13 +12,16 @@
 
 from ctms.crud import get_all_acoustic_records_count, get_all_acoustic_retries_count
 
+logger = logging.getLogger(__name__)
+
 
 def check_database(db_session, settings):
     """Check database availability and migration state."""
     start_time = time.monotonic()
     try:
         db_session.execute(select([func.now()])).first()
-    except SQLAlchemyError:
+    except SQLAlchemyError as exc:
+        logger.exception(exc)
         success = False
     else:
         success = True
@@ -34,7 +38,8 @@ def check_database(db_session, settings):
         end_time = datetime.now(tz=timezone.utc)
         count = get_all_acoustic_records_count(db_session, end_time, retry_limit)
         retry_count = get_all_acoustic_retries_count(db_session)
-    except SQLAlchemyError:
+    except SQLAlchemyError as exc:
+        logger.exception(exc)
         acoustic_success = False
     else:
         acoustic_success = True

ctms/routers/platform.py

@@ -1,3 +1,4 @@
+import logging
 from collections import defaultdict
 from typing import Optional
 
@@ -35,6 +36,9 @@
 router = APIRouter()
 
 
+logger = logging.getLogger(__name__)
+
+
 @router.get("/", include_in_schema=False)
 def root(request: Request):
     """GET via root redirects to /docs.
@@ -135,8 +139,16 @@ def heartbeat(
         max_retry_backlog = data["database"]["acoustic"]["max_retry_backlog"]
 
         if max_backlog is not None and max_backlog > backlog:
+            logger.error(
+                "Acoustic backlog size %s exceed maximum %s", backlog, max_backlog
+            )
             status_code = 503
         if max_retry_backlog is not None and max_retry_backlog > retry_backlog:
+            logger.error(
+                "Acoustic retry backlog size %s exceed maximum %s",
+                retry_backlog,
+                max_retry_backlog,
+            )
             status_code = 503
     return JSONResponse(content=data, status_code=status_code)
 

docs/README.md

@@ -1,4 +1,3 @@
-[...back to main README.md](../README.md/)
 - [Architecture Decision Records](adrs/)
 - [Diagrams](diagrams/)
 - [Configuration](configuration.md)

docs/developer_setup.md

@@ -170,6 +170,13 @@ git ls-files | xargs sed -i 's/A\.B\.C/X\.Y\.Z/g'
 
 Manually inspect the changes to filter out false positives.
 
+---
+## Access Acoustic STAGE
+
+- Request access to our Acoustic admins on `#email` Slack channel
+- Login via https://sso.mozilla.com/acoustic
+- Select *Pod 9*
+
 ---
 [View All Docs](./)
 

docs/testing_strategy.md

@@ -21,23 +21,24 @@ with the database.
 - The tests live in: `tests/unit/*.py`
 - The shared test fixtures live in: `tests/unit/conftest.py`
 
-### Running tests manually:
-Make sure you have dependencies installed and a postgres database running with
-all of the migrations run
+### Pass arguments to `pytest`
 
-And then run the installed ``pytest``:
-```sh
-pytest
+Set the `PYTEST_ADDOPTS` env var to pass arguments to `pytest`.
+
+Run in verbose mode (works for both unit and integration tests):
+
+```
+export PYTEST_ADDOPTS="-v"
 ```
 
 To stop on the first failure and drop into [pdb][pdb]:
 ```sh
-pytest -sx --pdb
+export PYTEST_ADDOPTS="-sx --pdb"
 ```
 
 To run a test or tests whose name matchs a substring:
 ```sh
-pytest -k "substring"
+export PYTEST_ADDOPTS='-k "substring"'
 ```
 
 View the [pytest] documentation for more options.