diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..eecc79e --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,61 @@ +name: Publish to PyPi + +on: + release: + types: [published] + workflow_dispatch: + +jobs: + build: + name: Build distribution 📦 + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.x" + - name: Install pypa/build + run: >- + python3 -m + pip install + build + --user + - name: Build a binary wheel and a source tarball + run: python3 -m build + - name: Store the distribution packages + uses: actions/upload-artifact@v4 + with: + name: python-package-distributions + path: dist/* + + publish-to-pypi: + name: >- + Publish Python 🐍 distribution 📦 to PyPI + and Sign with Sigstore + if: startsWith(github.ref, 'refs/tags/') + needs: + - build + runs-on: ubuntu-latest + environment: + name: pypi + url: https://pypi.org/p/eppo-metrics-sync/ + permissions: + id-token: write + contents: write + + steps: + - name: Download all the dists + uses: actions/download-artifact@v4 + with: + name: python-package-distributions + path: dist/ + - name: Publish distribution 📦 to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + - name: Sign the dists with Sigstore + uses: sigstore/gh-action-sigstore-python@v2.1.1 + with: + inputs: >- + ./dist/*.tar.gz + ./dist/*.whl diff --git a/README.md b/README.md index c78e641..661f602 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,118 @@ -### Eppo Metrics Sync +# Eppo Metrics Sync -This Python package allows to post a directory of yaml files to Eppo's API. Documentation is available in Eppo's [documentation page](https://docs.geteppo.com/data-management/certified-metrics/). +[![PyPI version](https://badge.fury.io/py/eppo-metrics-sync.svg)](https://badge.fury.io/py/eppo-metrics-sync) +[![Tests](https://github.com/Eppo-exp/eppo-metrics-sync/actions/workflows/run_tests.yml/badge.svg)](https://github.com/Eppo-exp/eppo-metrics-sync/actions) -#### Deploying the package +A Python package for syncing metric definitions with Eppo's API. Manage your Eppo metrics as code using YAML files. Documentation is available in Eppo's [documentation page](https://docs.geteppo.com/data-management/certified-metrics/). -Note: this section is intended for Eppo package maintainers. -- Bump version in ```setup.py``` -- Install required tools:```python3 -m pip install --upgrade setuptools wheel twine``` -- Generate distribution packages:```python3 setup.py sdist bdist_wheel``` -- Use Twine to upload the package:```python3 -m twine upload dist/*``` -You'll be prompted for your PyPI username and password as well as the API key that is available in 1Password. \ No newline at end of file +## Features + +- Sync metrics and fact sources to Eppo +- Validate metric definitions locally +- Support for dbt models +- Dry-run capability for testing +- Prefix support for testing in shared workspaces + +## Installation + +```bash +pip install eppo-metrics-sync +``` + +## Usage + +### Basic usage + +1. Set required environment variables: + +```bash +export EPPO_API_KEY="your-api-key" + +export EPPO_SYNC_TAG="your-sync-tag" # optional +``` + +2. Create your metrics YAML files (see [Documentation](#documentation)) + +3. Run the sync: + +```bash +python -m eppo_metrics_sync path/to/yaml/directory +``` + +### CLI Options + +```bash +python -m eppo_metrics_sync [OPTIONS] DIRECTORY +``` + +Options: + +- `--dryrun` Validate files without syncing to Eppo +- `--schema` Schema type: eppo (default) or dbt-model +- `--sync-prefix` Prefix for fact/metric names (useful for testing) +- `--dbt-model-prefix` Warehouse/schema prefix for dbt models + +## Documentation + +For detailed information about metric configuration and available options, see Eppo's [documentation page](https://docs.geteppo.com/data-management/certified-metrics/). + +### Example YAML Configuration + +```yaml +fact_sources: + - name: Revenue + sql: | + SELECT ts, user_id, amount + FROM revenue_table + timestamp_column: ts + entities: + - entity_name: User + column: user_id + facts: + - name: Revenue + column: amount + +metrics: + - name: Total Revenue + description: Sum of Total Purchase Value in Purchases Fact Table + entity: User + numerator: + fact_name: Revenue + operation: sum +``` + +## Development + +### Setup + +#### Create a virtual environment + +```bash +python -m venv .venv +source .venv/bin/activate +``` + +#### Install dependencies + +```bash +pip install -r requirements.txt +``` + +### Running the tests + +```bash +pytest tests +``` + +### Building and Publishing + +For package maintainers: + +1. Update version in `pyproject.toml` +2. Build the package: + +```bash +python -m build +``` + +3. The package will be automatically published to PyPI when a new release is created on GitHub. diff --git a/pyproject.toml b/pyproject.toml index 638dd9c..9d0455d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,3 +1,18 @@ [build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta" + +[project] +name = "eppo_metrics_sync" +version = "0.1.3" +description = "Sync metrics to Eppo" +readme = "README.md" +requires-python = ">=3.7" +dependencies = [ + "PyYAML", + "jsonschema", + "requests", +] + +[tool.setuptools] +include-package-data = true diff --git a/setup.py b/setup.py deleted file mode 100644 index e19fcd7..0000000 --- a/setup.py +++ /dev/null @@ -1,11 +0,0 @@ -from setuptools import setup, find_packages - -setup( - name='eppo_metrics_sync', - version='0.1.2', - packages=find_packages(), - install_requires=[ - 'PyYAML', 'jsonschema', 'requests' - ], - include_package_data=True -)