Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: fix wheel building #352

Merged
merged 16 commits into from
Feb 3, 2025
53 changes: 21 additions & 32 deletions .github/workflows/pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,18 @@ on:
push:
branches: [main]
tags: [v*]
pull_request:

jobs:
test:
uses: pyapp-kit/workflows/.github/workflows/test-pyrepo.yml@v2
with:
hatch-build-hooks-enable: true
coverage-upload: none
# test:
# uses: pyapp-kit/workflows/.github/workflows/test-pyrepo.yml@v2
# with:
# hatch-build-hooks-enable: true
# coverage-upload: none

deploy-sdist:
needs: test
if: success() && startsWith(github.ref, 'refs/tags/v')
# needs: test
# if: success() && startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
permissions:
id-token: write
Expand All @@ -34,36 +35,26 @@ jobs:
python -m pip install build
python -m build

- name: 🚢 Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
# - name: 🚢 Publish to PyPI
# uses: pypa/gh-action-pypi-publish@release/v1

- uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
files: "./dist/*"
# - uses: softprops/action-gh-release@v2
# with:
# generate_release_notes: true
# files: "./dist/*"

deploy-wheel:
if: startsWith(github.event.ref, 'refs/tags/v')
needs: deploy-sdist
name: mypyc wheels (${{ matrix.name }})
# if: startsWith(github.event.ref, 'refs/tags/v')
# needs: deploy-sdist
name: mypyc wheels (${{ matrix.os }})
runs-on: ${{ matrix.os }}
permissions:
id-token: write
contents: write
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
name: linux-x86_64
- os: windows-latest
name: windows-amd64
- os: macos-13
name: macos-x86_64
macos_arch: "x86_64"
- os: macos-14
name: macos-arm64
macos_arch: "arm64"
os: [ubuntu-latest, windows-latest, macos-13, macos-latest]

steps:
- uses: actions/checkout@v4
Expand All @@ -74,14 +65,12 @@ jobs:
uses: pypa/[email protected]
with:
output-dir: dist
env:
CIBW_ARCHS_MACOS: "${{ matrix.macos_arch }}"

- name: Upload wheels as workflow artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.name }}-mypyc-wheels
name: ${{ runner.os }}-${{ runner.arch }}-mypyc-wheels
path: ./dist/*.whl

- name: 🚢 Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
# - name: 🚢 Publish to PyPI
# uses: pypa/gh-action-pypi-publish@release/v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you upload here instead of have separate steep to collect all wheels anbd upload only if all platfrom buildt successfully?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simplicity 🤷‍♂️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But you may then upload to pypi broken release that was revealed during build wheel.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, I'd call it a "partial" release. it is actually pretty easy to upload more wheels if only some of them fail to build. In any case, this whole thing is draft, and I'm actively working on it. Nothing you see here at the moment should be considered something I've thought hard about. please hang on a moment. I'll let you know when it's ready for review if you'd like to look

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, short of uncommenting out the pypi publish step, i think this is now better. let me know if it's following all the best practices you would suggest

34 changes: 20 additions & 14 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,6 @@ dependencies = [
# extras
# https://peps.python.org/pep-0621/#dependencies-optional-dependencies
[project.optional-dependencies]
dev = [
"ipython",
"mypy",
"mypy_extensions",
"pre-commit",
"PyQt5",
"pytest-mypy-plugins",
"rich",
"ruff",
"typing-extensions",
]
docs = [
"griffe==0.25.5",
"mkdocs-material==8.5.10",
Expand All @@ -55,19 +44,32 @@ docs = [
proxy = ["wrapt"]
pydantic = ["pydantic"]
test = [
"dask",
"dask[array]>=2024.0.0",
"attrs",
"numpy",
"numpy >1.21.6",
"pydantic",
"pyinstaller>=4.0",
"pytest>=6.0",
"pytest-cov",
"wrapt",
"msgspec; python_version < '3.13'",
"msgspec",
"toolz",
]
testqt = ["pytest-qt", "qtpy"]

dev = [
"psygnal[test, testqt]",
"PyQt6",
"ipython",
"mypy",
"mypy_extensions",
"pre-commit",
"pytest-mypy-plugins",
"rich",
"ruff",
"typing-extensions",
]

[project.urls]
homepage = "https://github.com/pyapp-kit/psygnal"
repository = "https://github.com/pyapp-kit/psygnal"
Expand Down Expand Up @@ -118,6 +120,10 @@ test-extras = ["test"]
test-command = "pytest {project}/tests -v"
test-skip = ["*-musllinux*", "cp312-win*", "*-macosx_arm64"]

[tool.cibuildwheel.macos]
# https://cibuildwheel.readthedocs.io/en/stable/faq/#apple-silicon
archs = ["x86_64", "arm64"]

[[tool.cibuildwheel.overrides]]
select = "*-manylinux_i686*"
before-all = "yum install -y python3-devel"
Expand Down
23 changes: 21 additions & 2 deletions src/psygnal/_signal.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,6 @@ def ensure_at_least_20(val: int):
from __future__ import annotations

import inspect
import sys
import threading
import warnings
import weakref
Expand Down Expand Up @@ -182,7 +181,14 @@ def ensure_at_least_20(val: int):

_NULL = object()
F = TypeVar("F", bound=Callable)
RECURSION_LIMIT = sys.getrecursionlimit()

# using 300 instead of sys.getrecursionlimit()
# in a mypyc-compiled program, hitting an actual RecursionError can cause
# a segfault (rather than raise a python exception), so we really MUST
# avoid it. Windows has a lower stack limit than other platforms, so we
# use 300 as a cross-platform "safe" limit, determined via testing. It's
# probably plenty large for most reasonable use-cases.
RECURSION_LIMIT = 300

ReemissionVal = Literal["immediate", "queued", "latest-only"]
VALID_REEMISSION = set(ReemissionVal.__args__) # type: ignore
Expand Down Expand Up @@ -1223,6 +1229,13 @@ def emit_fast(self, *args: Any) -> None:
self._args_queue.append(args)
return

if self._recursion_depth >= RECURSION_LIMIT:
raise RecursionError(
f"Psygnal recursion limit ({RECURSION_LIMIT}) reached when emitting "
f"signal {self.name!r} with args {args}"
)

self._recursion_depth += 1
try:
for caller in self._slots:
caller.cb(args)
Expand All @@ -1238,6 +1251,9 @@ def emit_fast(self, *args: Any) -> None:
)
# this comment will show up in the traceback
raise loop_err from cb_err # emit() call ABOVE || callback error BELOW
finally:
if self._recursion_depth > 0:
self._recursion_depth -= 1

def __call__(
self, *args: Any, check_nargs: bool = False, check_types: bool = False
Expand All @@ -1246,6 +1262,9 @@ def __call__(
return self.emit(*args, check_nargs=check_nargs, check_types=check_types)

def _run_emit_loop(self, args: tuple[Any, ...]) -> None:
if self._recursion_depth >= RECURSION_LIMIT:
raise RecursionError("Recursion limit reached!")

with self._lock:
self._emit_queue.append(args)
if len(self._emit_queue) > 1:
Expand Down
Loading