diff --git a/.github/workflows/qiime2.yml b/.github/workflows/qiime2.yml index 19abcd59..a40ddc0a 100644 --- a/.github/workflows/qiime2.yml +++ b/.github/workflows/qiime2.yml @@ -12,7 +12,19 @@ jobs: matrix: # Test on multiple QIIME 2 versions. This is a nice way of making sure # that Qurro works on all of the QIIME 2 versions it actually, you - # know, supports. + # know, supports. As of writing, this includes QIIME 2 versions + # + # 1. 2020.11 + # 2. 2021.2 + # 3. 2021.4 + # 4. 2021.8 + # 5. 2021.11 + # 6. 2022.2 + # 7. 2022.8 + # + # We currently test just a subset of these versions, because each of + # these builds takes a while (and testing each of these would probs be + # overkill). # # We could use a matrix of just QIIME 2 version strings (e.g. # "2020.11", ...), but other stuff about the URLs change across QIIME 2 @@ -25,6 +37,8 @@ jobs: # for the heads up about this :) q2-yaml: [ "https://raw.githubusercontent.com/qiime2/environment-files/master/2020.11/release/qiime2-2020.11-py36-linux-conda.yml", + "https://raw.githubusercontent.com/qiime2/environment-files/master/2021.8/release/qiime2-2021.8-py38-linux-conda.yml", + "https://raw.githubusercontent.com/qiime2/environment-files/master/2022.2/release/qiime2-2022.2-py38-linux-conda.yml", "https://raw.githubusercontent.com/qiime2/environment-files/master/latest/staging/qiime2-latest-py38-linux-conda.yml" ] @@ -57,8 +71,16 @@ jobs: - name: Run tests run: conda run -n qiime2-dev make test - - name: Lint and stylecheck - run: conda run -n qiime2-dev make stylecheck + - name: Lint and stylecheck (not using Black to check Python code) + run: conda run -n qiime2-dev make stylecheck_not_black + + # It's possible to install packages that break QIIME 2's CLI (e.g. + # installing Click >= 8 into a Q2 2022.2 environment seems to break it -- + # this is discussed in the developer documentation, CONTRIBUTING.md). + # Verify that, by installing Qurro, we have not "broken" QIIME 2 in this + # way. + - name: Test that QIIME 2's CLI works + run: conda run -n qiime2-dev qiime info - name: Upload code coverage information to Codecov uses: codecov/codecov-action@v2 diff --git a/.github/workflows/standalone.yml b/.github/workflows/standalone.yml index 102a77d0..ffa049fd 100644 --- a/.github/workflows/standalone.yml +++ b/.github/workflows/standalone.yml @@ -26,6 +26,9 @@ jobs: - name: Install Qurro (and its pip dependencies) run: pip install -e .[dev] + + - name: Install Black, also (no QIIME 2 to worry about conflicts with) + run: pip install "black >= 22.3.0" - name: Set up Node.js uses: actions/setup-node@v1 @@ -33,13 +36,14 @@ jobs: - name: Install Qurro's Node.js development dependencies run: npm install -g mocha-headless-chrome nyc prettier@2.0.5 jshint - # Just run the tests; don't bother running stylechecking / linting, since - # that's already done in the QIIME 2 CI. - name: Run tests run: | make pytest_standalone make jstest + - name: Lint and stylecheck (including using Black to check Python code) + run: make stylecheck + # NOTE: For now, we only submit coverage reports from the QIIME 2 CI, # since the tests that that CI runs are a superset of those ran by this # "standalone" CI. (Also, Codecov wasn't merging them, which caused diff --git a/CHANGELOG.md b/CHANGELOG.md index 74b589b2..4117a1b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,9 @@ # Qurro changelog -## Qurro 0.8.0 (date TBD) +## Qurro 0.8.0 (October 19, 2022) ### Features added -- Add an option to Qurro's visualization interface to exclude metadata columns - from the "sample plot data" output TSV +- Added an option to Qurro's visualization interface to exclude metadata + columns from the "sample plot data" output TSV ([#306](https://github.com/biocore/qurro/issues/306), [#313](https://github.com/biocore/qurro/pull/313)). - This should make it easier to merge the sample log-ratios exported from @@ -12,25 +12,35 @@ [Gemelli](https://github.com/biocore/gemelli)'s tutorials.) ### Backward-incompatible changes - Switched the required pandas version from `>= 0.24.0, <1` to `>= 1`. - This makes Qurro now compatible with QIIME 2 versions `>= 2020.11`! + **This makes Qurro now compatible with QIIME 2 versions `>= 2020.11`!** - However, this change removes support for older QIIME 2 versions. (It may be possible to get this version of Qurro installed into an older QIIME 2 - environment, but we do not explicitly support this.) + environment, but we do not explicitly support this.) If you need to use + Qurro in older QIIME 2 environments, you can install an older version of + Qurro (e.g. `pip install "qurro==0.7.1"`). - Notably, Songbird has not yet made the shift to pandas `>= 1`, so Songbird and Qurro will need to be installed into separate environments; the "Red Sea" example notebook details this process. +- Updated a few other dependency versions to fix various problems (e.g. + explicitly requiring SciPy and pinning it to `scipy < 1.9.0` to fix + [this scikit-bio issue](https://github.com/biocore/scikit-bio/issues/1818), + removing Black from the development dependencies for reasons discussed in + [`CONTRIBUTING.md`](https://github.com/biocore/qurro/blob/master/CONTRIBUTING.md)). ### Bug fixes ### Performance enhancements ### Miscellaneous -- Improve various parts of Qurro's code to remove various warnings (for +- Updated various parts of Qurro's code to remove various warnings (for example, about certain things being deprecated). -- Update Qurro's example notebooks: +- Updated Qurro's example notebooks: - Updated the "Red Sea" notebook to explain how to use Songbird and Qurro in different conda environments. - Updated the "ALDEx2" notebook to explain how we recommend installing ALDEx2 nowadays. - Updated the documentation to refer to the published version of the Mackerel data's paper ([Minich et al. 2020](https://journals.asm.org/doi/full/10.1128/mSphere.00401-20)). +- Fixed various broken links in the documentation + ([#318](https://github.com/biocore/qurro/pull/318), + [#320](https://github.com/biocore/qurro/pull/320)). - Ported Qurro's continuous integration from Travis CI to GitHub Actions ([#316](https://github.com/biocore/qurro/issues/316)). - Improved Qurro's continuous integration in multiple ways: @@ -38,8 +48,9 @@ - Test the standalone Qurro functionality on multiple Python versions. - Test the standalone Qurro functionality in a non-QIIME-2 environment (similar to EMPress' "standalone" CI). -- Updated the development documentation regarding the minimum QIIME 2 version, - incompatibilities between `black` and `click`, etc. +- Updated the development documentation + ([`CONTRIBUTING.md`](https://github.com/biocore/qurro/blob/master/CONTRIBUTING.md)) + regarding the minimum QIIME 2 version, dependency version issues, etc. ## Qurro 0.7.1 (May 22, 2020) ### Features added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cf51afb9..fc34842d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,16 +10,25 @@ in this repository or email the Qurro development team ## Setting up a development environment +We'll use a QIIME 2 environment as the basis for creating our "development +environment" -- this is convenient because some of Qurro's tests rely on QIIME +2 being installed. (You could also create a development environment that does +not use QIIME 2, some of Qurro's tests would not be able to be run in such an +environment.) + 1. Fork Qurro. 1. Clone your fork of Qurro's source code to your computer. 2. Create a development conda environment for Qurro: 1. Install the latest version of QIIME 2 natively, - [as you would normally](https://docs.qiime2.org/2022.2/install/native/). - _You'll need to install a QIIME 2 version of at least 2020.11._ + [as you would normally](https://docs.qiime2.org/2022.8/install/native/). + _You'll need to install a QIIME 2 version of at least 2022.8._ 2. In a terminal, navigate to the folder to which you cloned your fork of Qurro's source code above. Run `pip install -e .[dev]` inside this folder to install Qurro along with its normal and development Python dependencies. - 3. Install the various Node.js requirements for testing Qurro's JavaScript + 3. Install Black: `pip install "black >= 22.3.0"`. We don't include this in + the development dependencies for reasons that are explained later in + this document. + 4. Install the various Node.js requirements for testing Qurro's JavaScript code. This can be done by running `npm install -g mocha-headless-chrome jshint prettier@2.0.5 nyc`. Note that this will install these programs globally on your system. @@ -53,22 +62,24 @@ generated by Qurro's Python tests, and is used to populate some of the JavaScript tests. Long story short, you'll just need to run `make test` (or just `make pytest`) before running `make jstest`. -### When running QIIME 2, Qurro, etc. I get `TypeError: __init__() got multiple values for argument 'obj'` +### When running `qiime`, `qiime qurro`, etc. from the CLI I get `TypeError: __init__() got multiple values for argument 'obj'` -This happens due to a [Click](https://click.palletsprojects.com/en/8.1.x/) +This happens due to a [Click](https://click.palletsprojects.com/) version being installed that is unsupported by QIIME 2. See [this q2cli issue](https://github.com/qiime2/q2cli/issues/259) for context. This can come up in development because one of Qurro's development -dependencies is -[Black](https://github.com/psf/black), and later versions of Black can rely -on Click 8 (and as of writing QIIME 2 doesn't seem to support Click 8 yet). - -If you run into this error, you can probably get around it by installing an -older version of Black, although earlier versions of Click might disagree with -the later versions that we use in the CI. There isn't a great solution for this -yet that I know of (short of pinning Black to an earlier version and -reformatting all the Python code accordingly...), but hopefully when QIIME 2 -supports Click 8 the problem will vanish. +dependencies (used for auto-formatting and format-checking the Python code) is +[Black](https://github.com/psf/black), and later versions of Black rely +on Click 8 (and versions of QIIME 2 before 2022.8 do not support Click 8). + +Pinning to an older version of Black causes another problem, because -- in this +case -- the newer version of Click used in QIIME 2 2022.8 environments will +break Black (with [this error](https://github.com/psf/black/issues/2964)). + +So, we get around this issue by removing Black from our development +dependencies completely. If you installed a version of QIIME 2 of at least +2022.8, then installing Black (in the directions above) should not cause +problems (or at least it should not cause these particular problems). ## Acknowledgements diff --git a/Makefile b/Makefile index e336092e..df9a8d12 100644 --- a/Makefile +++ b/Makefile @@ -32,11 +32,15 @@ jstest: nyc instrument qurro/support_files/js/ qurro/tests/web_tests/instrumented_js/ mocha-headless-chrome -f qurro/tests/web_tests/index.html -c js_coverage.json +stylecheck: stylecheck_not_black stylecheck_black + +stylecheck_black: + black --check -l 79 qurro/ setup.py + # Assumes this is being run from the root directory of the qurro repo # (since that's where the .jshintrc is located). -stylecheck: +stylecheck_not_black: flake8 --ignore=E203,W503 qurro/ setup.py - black --check -l 79 qurro/ setup.py jshint $(JSLOCS) prettier --check --tab-width 4 $(JSLOCS) $(HTMLCSSLOCS) diff --git a/README.md b/README.md index 27885deb..b03b52ae 100644 --- a/README.md +++ b/README.md @@ -58,11 +58,7 @@ Qurro can be used standalone (as a Python 3 script that generates a folder containing a HTML/JS/CSS visualization) or as a [QIIME 2](https://qiime2.org/) plugin (that generates a QZV file that can be visualized at [view.qiime2.org](https://view.qiime2.org/) or by using -`qiime tools view`). *Starting with Qurro v0.8.0, Qurro requires a QIIME 2 -version of at least 2020.11.* - -Qurro should work with most modern web browsers. Firefox or Chrome are -recommended. +`qiime tools view`). Qurro is still being developed, so backwards-incompatible changes might occur. If you have any bug reports, feature requests, questions, or if you just @@ -85,7 +81,18 @@ can be viewed online [here](https://biocore.github.io/qurro/demos/red_sea/index. ## Installation and Usage -You can install Qurro using [pip](https://pip.pypa.io/en/stable/) or [conda](https://docs.conda.io/en/latest/). In either case, a python version of at least 3.6 and less than 3.10 is required to use Qurro. +You can install Qurro using [pip](https://pip.pypa.io/en/stable/) +or [conda](https://docs.conda.io/en/latest/). + +### System requirements +**If you're using Qurro within QIIME 2,** you will need a QIIME 2 version of at +least 2020.11. + +**If you're using Qurro outside of QIIME 2,** you will need a Python version of +at least 3.6 and less than 3.10. + +In either case, Qurro should work with most modern web browsers; Firefox or Chrome are +recommended. ### Installing with `pip` ```bash @@ -203,8 +210,11 @@ For python testing/style checking, Qurro uses [pytest-cov](https://github.com/pytest-dev/pytest-cov), [flake8](http://flake8.pycqa.org/en/latest/), and [black](https://github.com/ambv/black). You'll also need to have QIIME 2 -installed to run most of the python tests (your QIIME 2 version should be at -least 2020.11; this is the version which switched to pandas >= 1). +installed to run most of the python tests (note that, due to click vs. black +vs. QIIME 2 dependency issues, you should use a QIIME 2 environment of at least +2022.8; see +[`CONTRIBUTING.md`](https://github.com/biocore/qurro/blob/master/CONTRIBUTING.md) +for details). For JavaScript testing/style checking, Qurro uses [Mocha](https://mochajs.org/), [Chai](https://www.chaijs.com/), diff --git a/screenshots/redsea_data.png b/screenshots/redsea_data.png index 93ab9bbc..72eee0e4 100644 Binary files a/screenshots/redsea_data.png and b/screenshots/redsea_data.png differ diff --git a/setup.py b/setup.py index 0a712d01..72cbd31d 100644 --- a/setup.py +++ b/setup.py @@ -100,9 +100,6 @@ "pytest >= 4.2", "pytest-cov >= 2.0", "flake8", - # If we don't pin black to being at least 22.3.0, then it and click - # will conflict w/ an error: https://github.com/psf/black/pull/2966 - "black >= 22.3.0", "nbconvert", ] },