Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meson build, replacing setuptools #109

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Meson build, replacing setuptools #109

wants to merge 2 commits into from

Conversation

marktsuchida
Copy link
Member

@marktsuchida marktsuchida commented Jan 19, 2024

Limitation: The sdist (source distribution for PyPI) now contains the whole of mmCoreAndDevices and more (33 MiB). While technically functional, we should fix this before merging this change.

Update mmCoreAndDevices to latest (which has meson.build for MMCore and MMDevice).

Add Meson build file.

Use meson-python so that python -m build just works via pyproject.toml.

Move the single source of truth for the version number from _version.py (now generated) to meson.build.

The Meson build has several advantages:

  • Build details of MMDevice and MMCore come from their own build files, rather than being duplicated here in setup.py
  • Editable installs truly work (even if C++ files are edited) (Caveat: beware of importing pymmcore from the source root)
  • Since we are using a true C++ build system, controlling build options is much easier and cleaner than it was with setuptools

The main disadvantage is that MANIFEST.in can no longer be used to control what gets included in the sdist (meson-python uses meson dist to produce the sdist, which includes all version-controlled files). However, once we are ready to use MMDevice and MMCore from independent repositories, this will no longer be an issue (and ends up being simpler than the error-prone MANIFEST.in).


  • Build works on macOS
  • Build works on Linux
    • Need dependency('threads')
    • Upstream (MMCore) needs dependency('threads')
  • Build works on Windows
    • Prevent Meson from looking for static libpython
    • Get CI to use MSVC, not MinGW, for build
  • Disable MMDevice/MMCore unit tests (by default) when building pymmcore (to avoid including Catch2)
    • Upstream (MMDevice, MMCore) needs to have option to disable tests
  • sdist contains only the needed files

@marktsuchida
Copy link
Member Author

Rebased and squashed.

Regarding the remaining issue (cannot produce correct sdist that doesn't include the massive mmCoreAndDevices), there are 3 possible solutions I can think of:

  • Wait until we have MMDevice and MMCore in separate Git repositories (and it should just work)
  • Come up with some ad-hoc way to produce the sdist outside of the regular build (yuck, and also error-prone if not using automatic check on MANIFEST.in)
  • Come up with our own PEP-517 build backend that forwards the sdist build to setuptools and wheel builds to meson-python (more work than I'd like to do)

Given that this Meson-based build does not add any user-facing benefits, I'm inclined to take the first approach and keep this draft PR open and maintained until we're ready. It has already served to iron out at least some of the issues we'll likely encounter when moving MMCoreJ to Meson.

Limitation: The sdist (source distribution for PyPI) now contains the
whole of mmCoreAndDevices and more (33 MiB). While technically
functional, we should fix this before merging this change.

Update mmCoreAndDevices to latest (which has meson.build for MMCore and
MMDevice).

Add Meson build file.

Use meson-python so that `python -m build` just works via
pyproject.toml.

Move the single source of truth for the version number from _version.py
(now generated) to meson.build.

The Meson build has several advantages:
- Build details of MMDevice and MMCore come from their own build files,
  rather than being duplicated here in setup.py
- Editable installs truly work (even if C++ files are edited) (Caveat:
  beware of importing pymmcore from the source root)
- Since we are using a true C++ build system, controlling build options
  is much easier and cleaner than it was with setuptools

The main disadvantage is that MANIFEST.in can no longer be used to
control what gets included in the sdist (meson-python uses `meson dist`
to produce the sdist, which includes all version-controlled files).
However, once we are ready to use MMDevice and MMCore from independent
repositories, this will no longer be an issue (and ends up being simpler
than the error-prone MANIFEST.in).
@marktsuchida
Copy link
Member Author

marktsuchida commented Oct 8, 2024

Rebased to current main.

  • Remove the temporary enablement of wheel building in PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant