Skip to content

Commit

Permalink
Adding workflows for unit tests and linting (#11)
Browse files Browse the repository at this point in the history
## Description
Adding 4 different GitHub workflows to help with managing code checks.
1. unit tests
2. linting
3. code vulnerabilities
4. dependency vulnerabilities

## Related Issues
closes #7 

## Additional Notes
Also removing the `scripts/vulnerability_checks.sh` script as we're
going to try using GH tools (ie CodeQL and Dependabot) for managing
those checks and won't need an independent script for it anymore.
  • Loading branch information
ericbuckley committed Sep 16, 2024
1 parent 66791e3 commit ba43da1
Show file tree
Hide file tree
Showing 10 changed files with 136 additions and 23 deletions.
9 changes: 9 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
commit-message:
prefix: "[deps] "
open-pull-requests-limit: 3
39 changes: 39 additions & 0 deletions .github/workflows/check_code_vulnerabilities.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: code vulnerabilities check

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
codeql:
runs-on: ubuntu-latest

permissions:
packages: read
actions: read
contents: read
security-events: write

strategy:
matrix:
# Using a matrix in case we need to test Javascript code in the future
language: ['python']

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
config: |
# only scan the code in the src directory
paths: ["src"]
31 changes: 31 additions & 0 deletions .github/workflows/check_lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: lint check

# When the workflow will be triggered
on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
lint:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install '.[dev]'
- name: Run lint checks
run: |
ruff check src/
50 changes: 50 additions & 0 deletions .github/workflows/check_unit_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: unit tests check

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest

services:
postgres:
image: postgres:13
ports:
- 5432:5432
env:
POSTGRES_PASSWORD: pw
POSTGRES_DB: testdb

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install '.[dev]'
- name: Wait for PostgreSQL to be ready
run: |
until pg_isready -h localhost -U postgres; do sleep 1; done
- name: Run unit tests
env:
MPI_DB_TYPE: postgres
MPI_DBNAME: testdb
MPI_HOST: localhost
MPI_PORT: 5432
MPI_USER: postgres
MPI_PASSWORD: pw
run: |
pytest --cov=recordlinker --cov-report=xml tests/unit
8 changes: 0 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,6 @@ To run a single unit test, use the following command:
./scripts/test_unit.sh tests/unit/test_linkage.py::test_link_record_against_mpi
```

### Building the Docker Image

To build the Docker image for the record linkage service from source code instead of downloading it from the DIBBs repository follow these steps.
1. Ensure that both [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) and [Docker](https://docs.docker.com/get-docker/) are installed.
2. Clone the DIBBs repository with `git clone https://github.com/CDCgov/phdi`.
3. Navigate to `/phdi/containers/record-linkage/`.
4. Run `docker build -t record-linkage .`.

## Standard Notices

### Public Domain Standard Notice
Expand Down
3 changes: 1 addition & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,8 @@ dependencies = [
[project.optional-dependencies]
dev = [
"pytest>=8.3",
"pytest-cov",
"ruff",
"pip-audit",
"bandit",
"mypy",
"pyarrow",
"httpx",
Expand Down
11 changes: 0 additions & 11 deletions scripts/vulnerability_check.sh

This file was deleted.

2 changes: 1 addition & 1 deletion src/recordlinker/config.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
from functools import lru_cache
from typing import Optional

from pydantic_settings import BaseSettings
from pydantic import Field
from pydantic_settings import BaseSettings


class Settings(BaseSettings):
Expand Down
1 change: 1 addition & 0 deletions src/recordlinker/linkage/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from pydantic_settings import BaseSettings


class DBSettings(BaseSettings):
mpi_db_type: str
mpi_dbname: str
Expand Down
5 changes: 4 additions & 1 deletion src/recordlinker/linkage/models.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import uuid

from sqlalchemy import orm, ForeignKey, String, JSON
from sqlalchemy import ForeignKey
from sqlalchemy import JSON
from sqlalchemy import orm
from sqlalchemy import String


class Base(orm.DeclarativeBase):
Expand Down

0 comments on commit ba43da1

Please sign in to comment.