Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Isolating distribution testing #910

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Isolating distribution testing #910

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
784e3ae
Isolating distribution testing
leahwicz Sep 11, 2024
469465e
Merge branch 'main' into leahwicz/splitEnv
mikealfare Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Under the Hood-20240911-192855.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Under the Hood
body: Isolating distribution testing
time: 2024-09-11T19:28:55.992897-04:00
custom:
Author: leahwicz
Issue: "874"
33 changes: 13 additions & 20 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repo is behind on some of the versions of Actions that are updated in the other repos. I figured I would update them while changing this file to match what we have everywhere else. The run still passes without issues on the upgrade

with:
python-version: '3.8'

Expand Down Expand Up @@ -83,7 +83,7 @@ jobs:
persist-credentials: false

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

Expand Down Expand Up @@ -125,7 +125,7 @@ jobs:
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: '3.8'

Expand Down Expand Up @@ -156,13 +156,13 @@ jobs:
if [[ "$(ls -lh dist/)" == *"a1"* ]]; then export is_alpha=1; fi
echo "is_alpha=$is_alpha" >> $GITHUB_OUTPUT

- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

test-build:
name: verify packages / python ${{ matrix.python-version }} / ${{ matrix.os }}
name: verify packages / python ${{ matrix.python-version }} / ${{ matrix.os }} / ${{ matrix.dist-type }}

if: needs.build.outputs.is_alpha == 0

Expand All @@ -173,41 +173,34 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-22.04, macos-12, windows-2022]
os: [ubuntu-latest, macos-12, windows-latest]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, I remember wanting to keep on the latest images (this isn't the case for Mac, not sure why). So the other adapter repos use latest instead of a hard coded version so updating this to keep parity as well. I can revert this if it isn't wanted

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We determined that it's better to hard pin and then upgrade as we can. The primary issue is that the latest tag eventually bumps and the new image may not have all of the python versions installed. We ran into this on macos in particular, which is probably what we used to pin that in the past.

python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
dist-type: ['whl', 'gz']

steps:
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install python dependencies
run: |
python -m pip install --user --upgrade pip
python -m pip install --user --upgrade pip
python -m pip install --upgrade wheel
python -m pip --version

- uses: actions/download-artifact@v3
- uses: actions/download-artifact@v4
with:
name: dist
path: dist/

- name: Show distributions
run: ls -lh dist/

- name: Install wheel distributions
run: |
find ./dist/*.whl -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/

- name: Check wheel distributions
run: |
python -c "import dbt.adapters.redshift"

- name: Install source distributions
- name: Install ${{ matrix.dist-type }} distributions
run: |
find ./dist/*.gz -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/
find ./dist/*.${{ matrix.dist-type }} -maxdepth 1 -type f | xargs python -m pip install --force-reinstall --find-links=dist/

- name: Check source distributions
- name: Check ${{ matrix.dist-type }} distributions
run: |
python -c "import dbt.adapters.redshift"
Loading