diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..498baa3f --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,15 @@ +# Community Participation Guidelines + +This repository is governed by Mozilla's code of conduct and etiquette guidelines. +For more details, please read the +[Mozilla Community Participation Guidelines](https://www.mozilla.org/about/governance/policies/participation/). + +## How to Report +For more information on how to report violations of the Community Participation Guidelines, please read our '[How to Report](https://www.mozilla.org/about/governance/policies/participation/reporting/)' page. + + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..dfb71055 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,13 @@ +First off, thanks for taking the time to contribute! ❤️ + +All types of contributions are encouraged and valued. + +Before doing so, here are a few guidelines: + +* You agree to license your contributions under the project [license](LICENSE). +* Use pull-requests early so it's open for discussion, even if your + contribution isn't ready yet. +* All pull requests should include tests, as they help us avoid regressions in + our code. +* A pull-request adding functionality should also update the documentation + accordingly. diff --git a/README.md b/README.md index 7c84ee31..37d79e74 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,23 @@ # ConTact Management System (CTMS) -While the app is running, interactive API documentation can found at the following relative paths: /docs, /redoc. - -OpenApiSpec(OAS) formatted in JSON can be found at the following path: /openapi.json - ---- - -[View All Docs](docs/README.md) - ---- - -## Prerequisites - -Please read the [Developer Setup documentation](docs/developer_setup.md) before developing \ -for this project to ensure correct environment setup. - ---- -## Project Structure - -The project is structured with the following in mind: - -- [bin/*](bin/) - - Some scripts that have proven useful within the CTMS ecosystem -- [docs/*](docs/) - - Documentation to guide others around the project interactions -- [ctms/*](ctms/) - - Application logic lives within this directory - - [bin/*](ctms/bin/) - - Scripts intended for background machinery - - [schemas/*](ctms/schemas/) - - Pydantic Models for Data Modeling and Contract Validation -- [migrations/*](migrations/) - - Alembic migrations that act as a changelog or version control system for implementing DB changes in an ordered fashion -- [tests/unit/*](test/unit/) - - Test suite using pytest - ---- -## Important Files - -Below are some files that are worth making note of: -- [MAKEFILE](Makefile) - - Enabling commands such as: make {build | lint | setup | start | test | shell | db-only} -- [ctms/app.py](ctms/app.py) - - FastAPI handling of HTTP Requests and routing to services -- [ctms/bin/acoustic_sync.py](ctms/bin/acoustic_sync.py) - - Background job for synchronizing pending records to Acoustic -- [ctms/config.py](ctms/config.py) - - Environment variables are initialized here -- [ctms/models.py](ctms/models.py) - - SQLAlchemy models for ORM tool +![Status Sustain](https://img.shields.io/badge/Status-Sustain-green) + +*CTMS* is an internal service at Mozilla in charge of managing marketing contacts. +It consists of a REST API and a background synchronization process. + +## Usage + +Run the app with `make start`. Check [developer docs](docs/developer_setup.md) for setup. + +While the app is running, interactive API documentation is available at: + +- http://localhost:8000/docs +- http://localhost:8000/redoc + +OpenApiSpec(OAS) formatted in JSON can be found at the following path: `/openapi.json`. + +## [Documentation](docs/README.md) + +## License + +*CTMS* is licensed under the MPLv2. See the `LICENSE` file for details. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..9156c459 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,16 @@ +# Security Policy + +Mozilla has a [well-defined process for handling security vulnerabilities](https://www.mozilla.org/en-US/about/governance/policies/security-group/bugs/) based around responsible disclosure. + +## Supported Versions + +| Version | Supported | +| ------- | ------------------ | +| v2.x.x | :white_check_mark: | +| < v2.0 | :x: | + +## Reporting a Vulnerability + +If you believe you have found a security vulnerability, you should visit the [Mozilla bug bounty program](https://www.mozilla.org/en-US/security/bug-bounty/) for information on how to submit them. + +[This Bugzilla template](https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody%40mozilla.org&bug_ignored=0&bug_severity=--&bug_status=NEW&bug_type=defect&cf_fx_iteration=---&cf_fx_points=---&component=SRE&contenttypemethod=list&contenttypeselection=text%2Fplain&defined_groups=1&filed_via=standard_form&flag_type-4=X&flag_type-607=X&flag_type-674=X&flag_type-800=X&flag_type-803=X&flag_type-936=X&form_name=enter_bug&groups=releng-security&groups=mozilla-employee-confidential&groups=partner-confidential&maketemplate=Remember%20values%20as%20bookmarkable%20template&op_sys=Unspecified&priority=--&product=Infrastructure%20%26%20Operations&rep_platform=Unspecified&target_milestone=---&version=unspecified) will help you report a security vulnerability directly to our SRE team. diff --git a/ctms/monitor.py b/ctms/monitor.py index 7b32b1e3..c259d138 100644 --- a/ctms/monitor.py +++ b/ctms/monitor.py @@ -1,6 +1,7 @@ """Application monitoring and health utilities""" import json +import logging import os.path import time from datetime import datetime, timezone @@ -11,13 +12,16 @@ from ctms.crud import get_all_acoustic_records_count, get_all_acoustic_retries_count +logger = logging.getLogger(__name__) + def check_database(db_session, settings): """Check database availability and migration state.""" start_time = time.monotonic() try: db_session.execute(select([func.now()])).first() - except SQLAlchemyError: + except SQLAlchemyError as exc: + logger.exception(exc) success = False else: success = True @@ -34,7 +38,8 @@ def check_database(db_session, settings): end_time = datetime.now(tz=timezone.utc) count = get_all_acoustic_records_count(db_session, end_time, retry_limit) retry_count = get_all_acoustic_retries_count(db_session) - except SQLAlchemyError: + except SQLAlchemyError as exc: + logger.exception(exc) acoustic_success = False else: acoustic_success = True diff --git a/ctms/routers/platform.py b/ctms/routers/platform.py index 9b083f84..a14bd961 100644 --- a/ctms/routers/platform.py +++ b/ctms/routers/platform.py @@ -1,3 +1,4 @@ +import logging from collections import defaultdict from typing import Optional @@ -35,6 +36,9 @@ router = APIRouter() +logger = logging.getLogger(__name__) + + @router.get("/", include_in_schema=False) def root(request: Request): """GET via root redirects to /docs. @@ -135,8 +139,16 @@ def heartbeat( max_retry_backlog = data["database"]["acoustic"]["max_retry_backlog"] if max_backlog is not None and max_backlog > backlog: + logger.error( + "Acoustic backlog size %s exceed maximum %s", backlog, max_backlog + ) status_code = 503 if max_retry_backlog is not None and max_retry_backlog > retry_backlog: + logger.error( + "Acoustic retry backlog size %s exceed maximum %s", + retry_backlog, + max_retry_backlog, + ) status_code = 503 return JSONResponse(content=data, status_code=status_code) diff --git a/docs/README.md b/docs/README.md index 54729890..e72981a1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,4 +1,3 @@ -[...back to main README.md](../README.md/) - [Architecture Decision Records](adrs/) - [Diagrams](diagrams/) - [Configuration](configuration.md) diff --git a/docs/developer_setup.md b/docs/developer_setup.md index 416ac5ea..585bb3b5 100644 --- a/docs/developer_setup.md +++ b/docs/developer_setup.md @@ -170,6 +170,13 @@ git ls-files | xargs sed -i 's/A\.B\.C/X\.Y\.Z/g' Manually inspect the changes to filter out false positives. +--- +## Access Acoustic STAGE + +- Request access to our Acoustic admins on `#email` Slack channel +- Login via https://sso.mozilla.com/acoustic +- Select *Pod 9* + --- [View All Docs](./) diff --git a/docs/testing_strategy.md b/docs/testing_strategy.md index f2bd7c86..859074e6 100644 --- a/docs/testing_strategy.md +++ b/docs/testing_strategy.md @@ -21,23 +21,24 @@ with the database. - The tests live in: `tests/unit/*.py` - The shared test fixtures live in: `tests/unit/conftest.py` -### Running tests manually: -Make sure you have dependencies installed and a postgres database running with -all of the migrations run +### Pass arguments to `pytest` -And then run the installed ``pytest``: -```sh -pytest +Set the `PYTEST_ADDOPTS` env var to pass arguments to `pytest`. + +Run in verbose mode (works for both unit and integration tests): + +``` +export PYTEST_ADDOPTS="-v" ``` To stop on the first failure and drop into [pdb][pdb]: ```sh -pytest -sx --pdb +export PYTEST_ADDOPTS="-sx --pdb" ``` To run a test or tests whose name matchs a substring: ```sh -pytest -k "substring" +export PYTEST_ADDOPTS='-k "substring"' ``` View the [pytest] documentation for more options.