Skip to content

Commit

Permalink
0.6.0
Browse files Browse the repository at this point in the history 
- add support and CI checks for python `>=3.8` (Previously only `3.10, 3.11` were supported)
  - this mostly consisted of messing with a lot of type annotations and adding `from __future__ import annotations` in a number of places
- added `validate_type` function
  - make `SerializableDatclass` actually do type validation
- improvements to `SerializableDataclass` and related functions
  - much better docs and comments
  - move `SerializableField` and `serializable_field` to their own file
  - resolve #40 by adding `deserialize_fn` to `SerializableField`
  - moved `array_safe_eq` and `dc_eq` to `muutils.json_serialize.util`
  - added `array_b64_meta` mode to `muutils.json_serialize.array`, works pretty much the same as existing `array_hex_meta` but should take less space
- added `ErrorMode` enum 
  - special `.process()` method which takes a message (and optionally classes for warnings and errors, and an `except_from` parameter) and decides whether to throw warning, exception, or do nothing
  - switched error mode handling to this across the package
- switch to ruff for formatting and linting
- add cli interface to `muutils.sys_info`
- added `sanitize_name` function with `sanitize_fname` and `sanitize_identifier` as special cases
  - see `muutils.misc`
- big changes and improvements to CI/makefile
  - `uv` now used to install deps in CI, separate job for checking `poetry.lock` consistency
  - makefile recipes reworked, and major changes to internals. `make help` for more info
  - added many comments to the makefile
- many other small fixes and changes, including a large number of new tests
  • Loading branch information
@mivanit
mivanit authored Jun 21, 2024
2 parents 9dbf15a + 3dac41f commit 0e46cf1
Show file tree
Hide file tree
Showing 60 changed files with 3,188 additions and 1,044 deletions.
0 .lastversion → .github/.lastversion
View file
File renamed without changes.
60 changes: 60 additions & 0 deletions .github/dev-requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
asttokens==2.4.1 ; python_version >= "3.10" and python_version < "4.0"
click==8.1.7 ; python_version >= "3.8" and python_version < "4"
colorama==0.4.6 ; python_version >= "3.8" and sys_platform == "win32" and python_version < "4.0" or python_version >= "3.8" and python_version < "4" and platform_system == "Windows"
contourpy==1.1.1 ; python_version >= "3.8" and python_version < "4.0"
coverage-badge==1.1.1 ; python_version >= "3.8" and python_version < "4.0"
coverage==7.5.3 ; python_version >= "3.8" and python_version < "4.0"
coverage[toml]==7.5.3 ; python_version >= "3.8" and python_version < "4.0"
cycler==0.12.1 ; python_version >= "3.8" and python_version < "4.0"
decorator==5.1.1 ; python_version >= "3.10" and python_version < "4.0"
exceptiongroup==1.2.1 ; python_version >= "3.8" and python_version < "3.11"
executing==2.0.1 ; python_version >= "3.10" and python_version < "4.0"
fonttools==4.53.0 ; python_version >= "3.8" and python_version < "4.0"
importlib-metadata==7.2.0 ; python_version >= "3.8" and python_version < "3.10"
importlib-resources==6.4.0 ; python_version >= "3.8" and python_version < "3.10"
iniconfig==2.0.0 ; python_version >= "3.8" and python_version < "4.0"
ipython==8.25.0 ; python_version >= "3.10" and python_version < "4.0"
jaxtyping==0.2.19 ; python_version >= "3.8" and python_version < "4.0"
jedi==0.19.1 ; python_version >= "3.10" and python_version < "4.0"
kiwisolver==1.4.5 ; python_version >= "3.8" and python_version < "4.0"
libcst==1.1.0 ; python_version >= "3.8" and python_version < "4"
markdown-it-py==3.0.0 ; python_version >= "3.8" and python_version < "4"
matplotlib-inline==0.1.7 ; python_version >= "3.10" and python_version < "4.0"
matplotlib==3.7.5 ; python_version >= "3.8" and python_version < "4.0"
mdurl==0.1.2 ; python_version >= "3.8" and python_version < "4"
mypy-extensions==1.0.0 ; python_version >= "3.8" and python_version < "4.0"
mypy==1.10.0 ; python_version >= "3.8" and python_version < "4.0"
numpy==1.24.4 ; python_version >= "3.8" and python_version < "3.9"
numpy==1.26.4 ; python_version >= "3.9" and python_version < "4.0"
packaging==24.1 ; python_version >= "3.8" and python_version < "4.0"
parso==0.8.4 ; python_version >= "3.10" and python_version < "4.0"
pathspec==0.12.1 ; python_version >= "3.8" and python_version < "4"
pexpect==4.9.0 ; python_version >= "3.10" and python_version < "4.0" and (sys_platform != "win32" and sys_platform != "emscripten")
pillow==10.3.0 ; python_version >= "3.8" and python_version < "4.0"
plotly==5.22.0 ; python_version >= "3.8" and python_version < "4.0"
pluggy==1.5.0 ; python_version >= "3.8" and python_version < "4.0"
prompt-toolkit==3.0.47 ; python_version >= "3.10" and python_version < "4.0"
ptyprocess==0.7.0 ; python_version >= "3.10" and python_version < "4.0" and (sys_platform != "win32" and sys_platform != "emscripten")
pure-eval==0.2.2 ; python_version >= "3.10" and python_version < "4.0"
pycln==2.4.0 ; python_version >= "3.8" and python_version < "4"
pygments==2.18.0 ; python_version >= "3.8" and python_version < "4.0"
pyparsing==3.1.2 ; python_version >= "3.8" and python_version < "4.0"
pytest-cov==4.1.0 ; python_version >= "3.8" and python_version < "4.0"
pytest==8.2.2 ; python_version >= "3.8" and python_version < "4.0"
python-dateutil==2.9.0.post0 ; python_version >= "3.8" and python_version < "4.0"
pyyaml==6.0.1 ; python_version >= "3.8" and python_version < "4"
rich==13.7.1 ; python_version >= "3.8" and python_version < "4"
ruff==0.4.10 ; python_version >= "3.8" and python_version < "4.0"
shellingham==1.5.4 ; python_version >= "3.8" and python_version < "4"
six==1.16.0 ; python_version >= "3.8" and python_version < "4.0"
stack-data==0.6.3 ; python_version >= "3.10" and python_version < "4.0"
tenacity==8.4.1 ; python_version >= "3.8" and python_version < "4.0"
tomli==2.0.1 ; python_version >= "3.8" and python_full_version <= "3.11.0a6"
tomlkit==0.12.5 ; python_version >= "3.8" and python_version < "4"
traitlets==5.14.3 ; python_version >= "3.10" and python_version < "4.0"
typeguard==4.3.0 ; python_version >= "3.8" and python_version < "4.0"
typer==0.12.3 ; python_version >= "3.8" and python_version < "4"
typing-extensions==4.12.2 ; python_version >= "3.8" and python_version < "4.0"
typing-inspect==0.9.0 ; python_version >= "3.8" and python_version < "4"
wcwidth==0.2.13 ; python_version >= "3.10" and python_version < "4.0"
zipp==3.19.2 ; python_version >= "3.8" and python_version < "3.10"
17 changes: 17 additions & 0 deletions .github/lint-requirements.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
click==8.1.7 ; python_version >= "3.8" and python_version < "4"
colorama==0.4.6 ; python_version >= "3.8" and python_version < "4" and platform_system == "Windows"
libcst==1.1.0 ; python_version >= "3.8" and python_version < "4"
markdown-it-py==3.0.0 ; python_version >= "3.8" and python_version < "4"
mdurl==0.1.2 ; python_version >= "3.8" and python_version < "4"
mypy-extensions==1.0.0 ; python_version >= "3.8" and python_version < "4"
pathspec==0.12.1 ; python_version >= "3.8" and python_version < "4"
pycln==2.4.0 ; python_version >= "3.8" and python_version < "4"
pygments==2.18.0 ; python_version >= "3.8" and python_version < "4"
pyyaml==6.0.1 ; python_version >= "3.8" and python_version < "4"
rich==13.7.1 ; python_version >= "3.8" and python_version < "4"
ruff==0.4.10 ; python_version >= "3.8" and python_version < "4.0"
shellingham==1.5.4 ; python_version >= "3.8" and python_version < "4"
tomlkit==0.12.5 ; python_version >= "3.8" and python_version < "4"
typer==0.12.3 ; python_version >= "3.8" and python_version < "4"
typing-extensions==4.12.2 ; python_version >= "3.8" and python_version < "4"
typing-inspect==0.9.0 ; python_version >= "3.8" and python_version < "4"
99 changes: 80 additions & 19 deletions .github/workflows/checks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,49 +14,110 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Install linters
run: pip install pycln isort black
run: make setup-format RUN_GLOBAL=1

- name: Run Format Checks
run: make check-format
run: make check-format RUN_GLOBAL=1

check-deps:
name: Check dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'

- name: Install Poetry
run: curl -sSL https://install.python-poetry.org | python3 -

- name: Poetry Plugins
run: |
poetry self add poetry-plugin-export
poetry self show plugins
- name: Check poetry.lock and .github/dev-requirements.txt
run: make check-dep-dev

- name: Install uv
run: pip install uv

- name: Install dependencies
run: uv pip install -r .github/dev-requirements.txt --system --no-deps # we already should have all dependencies exported into .github/dev-requirements.txt

- name: Install torch (special)
run: uv pip install torch==2.3.1+cpu --system --extra-index-url https://download.pytorch.org/whl/cpu

- name: Install muutils (local)
run: uv pip install . --system

- name: Install zanj (requires muutils)
run: uv pip install zanj --system

test:
name: Test and Lint
runs-on: ubuntu-latest
# needs: [lint, check-deps]
strategy:
matrix:
versions:
- python: "3.10"
torch: "1.13.1"
- python: "3.10"
torch: "2.0.1"
- python: "3.11"
torch: "2.0.1"
- python: '3.8'
torch: '1.13.1'
- python: '3.9'
torch: '1.13.1'
- python: '3.10'
torch: '1.13.1'
- python: '3.10'
torch: '2.3.1'
- python: '3.11'
torch: '2.3.1'
- python: '3.12'
torch: '2.3.1'
steps:
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.versions.python }}

- name: Install uv
run: pip install uv

- name: Install dependencies
# install torch first to avoid pytorch index messing things up
run: |
curl -sSL https://install.python-poetry.org | python3 -
poetry lock --check
export CUDA_VISIBLE_DEVICES=0
poetry add torch@${{ matrix.versions.torch }}+cpu --source torch_cpu
poetry install --all-extras
uv pip install -r .github/dev-requirements.txt --system --no-deps
uv pip install torch==${{ matrix.versions.torch}}+cpu --system --extra-index-url https://download.pytorch.org/whl/cpu
- name: Install muutils
run: uv pip install . --system

- name: Install zanj (>=3.10 only)
# TODO: not yet available for python 3.8 and 3.9
if: ${{ matrix.versions.python != '3.8' && matrix.versions.python != '3.9' }}
run: uv pip install zanj --system

- name: tests
run: make test
run: make test RUN_GLOBAL=1

- name: tests in strict mode
# TODO: until zanj ported to 3.8 and 3.9
if: ${{ matrix.versions.python != '3.8' && matrix.versions.python != '3.9' }}
run: make test WARN_STRICT=1 RUN_GLOBAL=1

- name: lint
run: make lint
- name: check typing
run: make typing RUN_GLOBAL=1
14 changes: 12 additions & 2 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,26 @@
# local stuff (pypi token, commit log)
.github/local/**

# this one is cursed
tests/unit/validate_type/test_validate_type_GENERATED.py
# test notebook
_test.ipynb
# junk data
JUNK_DATA_PATH/
junk_data
.pypi-token
.commit_log

.vscode/
# caches
__pycache__/
**/__pycache__/
**/.mypy_cache/
**/.pytest_cache/

# coverage
.coverage
htmlcov/

# build
build/
dist/
muutils.egg-info/
Expand Down
4 changes: 2 additions & 2 deletions docs/coverage/coverage.svg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0e46cf1

Please sign in to comment.