From 21e4f46daa11f0185debcf6157ff445f1243ff3b Mon Sep 17 00:00:00 2001 From: Rain Date: Wed, 29 Nov 2023 19:37:05 -0800 Subject: [PATCH] [meta] update documentation for glob matcher + changelogs --- nextest-filtering/CHANGELOG.md | 10 ++++++ nextest-metadata/CHANGELOG.md | 9 ++++-- nextest-runner/CHANGELOG.md | 5 +++ site/src/CHANGELOG.md | 24 ++++++++++++++ site/src/book/filter-expressions.md | 50 ++++++++++++++++++++++++----- site/src/book/running.md | 19 ++++++----- 6 files changed, 99 insertions(+), 18 deletions(-) diff --git a/nextest-filtering/CHANGELOG.md b/nextest-filtering/CHANGELOG.md index 4c90a01e652..597e366da32 100644 --- a/nextest-filtering/CHANGELOG.md +++ b/nextest-filtering/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## [0.6.0] - 2023-12-03 + +### Added + +- Support for glob matchers. +- Support for a new `binary_id` predicate. + +For more information, see the changelog for [cargo-nextest 0.9.64](https://nexte.st/CHANGELOG.html#0964---2023-12-03). + ## [0.5.1] - 2023-10-22 ### Changed @@ -68,6 +77,7 @@ Initial release. +[0.6.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-filtering-0.6.0 [0.5.1]: https://github.com/nextest-rs/nextest/releases/tag/nextest-filtering-0.5.1 [0.5.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-filtering-0.5.0 [0.4.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-filtering-0.4.0 diff --git a/nextest-metadata/CHANGELOG.md b/nextest-metadata/CHANGELOG.md index 10760a518e7..53c6cf9d8f0 100644 --- a/nextest-metadata/CHANGELOG.md +++ b/nextest-metadata/CHANGELOG.md @@ -1,6 +1,11 @@ # Changelog -## Unreleased +## [0.9.3] - 2023-12-03 + +### Added + +- New method `RustBinaryId::from_parts` to construct a `RustBinaryId` from its constituent parts. +- Stabilized and documented [the binary ID format](https://docs.rs/nextest-metadata/latest/nextest_metadata/struct.RustBinaryId.html#method.from_parts). ## [0.9.2] - 2023-09-26 @@ -40,7 +45,6 @@ (This was not released due to a publishing issue.) - ## [0.8.0] - 2023-01-02 ### Added @@ -183,6 +187,7 @@ This release is compatible with cargo-nextest 0.9.10 and later. - Initial version, with support for listing tests. +[0.9.3]: https://github.com/nextest-rs/nextest/releases/tag/nextest-metadata-0.9.3 [0.9.2]: https://github.com/nextest-rs/nextest/releases/tag/nextest-metadata-0.9.2 [0.9.1]: https://github.com/nextest-rs/nextest/releases/tag/nextest-metadata-0.9.1 [0.9.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-metadata-0.9.0 diff --git a/nextest-runner/CHANGELOG.md b/nextest-runner/CHANGELOG.md index 8d2cfa8ab94..24162b8892a 100644 --- a/nextest-runner/CHANGELOG.md +++ b/nextest-runner/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## [0.51.0] - 2023-12-03 + +See the changelog for [cargo-nextest 0.9.64](https://nexte.st/CHANGELOG.html#0964---2023-12-03). + ## [0.50.0] - 2023-11-16 See the changelog for [cargo-nextest 0.9.63](https://nexte.st/CHANGELOG.html#0963---2023-11-17). @@ -400,6 +404,7 @@ Thanks to [Guiguiprim](https://github.com/Guiguiprim) for their contributions to - Initial version. +[0.51.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.51.0 [0.50.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.50.0 [0.49.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.49.0 [0.48.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.48.0 diff --git a/site/src/CHANGELOG.md b/site/src/CHANGELOG.md index 25a07e8651b..83e6b50d0df 100644 --- a/site/src/CHANGELOG.md +++ b/site/src/CHANGELOG.md @@ -3,6 +3,29 @@ This page documents new features and bugfixes for cargo-nextest. Please see the [stability policy](book/stability.md) for how versioning works with cargo-nextest. +## [0.9.64] - 2023-12-03 + +### Added + +- Stabilized and documented [the binary ID format](https://nexte.st/book/running#binary-ids). +- Support for glob matchers in [filter expressions](https://nexte.st/book/filter-expressions). For example, `package(foo*)` will match all tests whose names start with `foo`. +- A new `binary_id()` predicate matches against the binary ID. + +### Changed + +- Unit tests in proc-macro crates now have a binary ID that consists of just the crate name, similar to unit tests in normal crates. + +- The default string matcher for the following predicates has changed from _equality_ to _glob_: + + - Package-related matchers: `package()`, `deps()`, and `rdeps()` + - Binary-related matchers: `binary()` + + The new `binary_id()` predicate also uses the glob matcher by default. + +### Fixed + +- Fixed a regression with some Cargo nightly-only features: see [guppy-rs/guppy#174](https://github.com/guppy-rs/guppy/pull/174) for more details. + ## [0.9.63] - 2023-11-17 ### Fixed @@ -852,6 +875,7 @@ Supported in this initial release: - [Test retries](book/retries.md) and flaky test detection - [JUnit support](book/junit.md) for integration with other test tooling +[0.9.64]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.64 [0.9.63]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.63 [0.9.62]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.62 [0.9.61]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.61 diff --git a/site/src/book/filter-expressions.md b/site/src/book/filter-expressions.md index f9ee211e992..cb3de8969d2 100644 --- a/site/src/book/filter-expressions.md +++ b/site/src/book/filter-expressions.md @@ -31,6 +31,7 @@ cargo nextest run -E 'test(my_test) + package(my-crate)' ### Examples - `package(serde) and test(deserialize)`: every test containing the string `deserialize` in the package `serde` +- `deps(nextest*)`: all tests in packages whose names start with `nextest`, and all of their (possibly transitive) dependencies - `not (test(/parse[0-9]*/) | test(run))`: every test name not matching the regex `parse[0-9]*` or the substring `run` > **Note:** If you pass in both a filter expression and a standard, substring-based filter, tests @@ -62,6 +63,7 @@ This section contains the full set of operators supported by the DSL. - `binary(name-matcher)`: include all tests in binary names matching `name-matcher`. - For tests of kind `lib` and `proc-macro`, the binary name is the same as the name of the crate. - Otherwise, it's the name of the integration test, benchmark, or binary target. +- `binary_id(name-matcher)`: include all tests in [binary IDs](running.md#binary-ids) matching `name-matcher`. - `platform(host)` or `platform(target)`: include all tests that are [built for the host or target platform](running.md#filtering-by-build-platform), respectively. - `none()`: include no tests. @@ -79,18 +81,22 @@ This section contains the full set of operators supported by the DSL. ### Name matchers -- `~string`: match a package or test name containing `string` -- `=string`: match a package or test name that's equal to `string` -- `/regex/`: match a package or test name if any part of it matches the regular expression `regex`. To match the entire string against a regular expression, use `/^regex$/`. The implementation uses the [regex](https://github.com/rust-lang/regex) crate. -- `string`: default matching strategy. - - For tests (`test()`), this is equivalent to `~string`. - - For packages (`package()`, `deps()` and `rdeps()`), binary kinds (`kind()`), and `platform()`, this is equivalent to `=string`. +- `=string`: *equality matcher*—match a package or test name that's equal to `string`. +- `~string`: *contains matcher*—match a package or test name containing `string`. +- `/regex/`: *regex matcher*—match a package or test name if any part of it matches the regular expression `regex`. To match the entire string against a regular expression, use `/^regex$/`. The implementation uses the [regex](https://github.com/rust-lang/regex) crate. +- `#glob`: *glob matcher*—match a package or test name if the full name matches the glob expression `glob`. The implementation uses the [globset crate](https://docs.rs/globset). +- `string`: Default matching strategy. + - For `test()` predicates, this is the *contains matcher*, equivalent to `~string`. + - For package-related predicates (`package()`, `deps()`, and `rdeps()`), this is the *glob matcher*, equivalent to `#string`. + - For binary-related predicates (`binary()` and `binary_id()`), this is also the *glob matcher*. + - For `kind()` and `platform()`, this is the *equality matcher*, equivalent to `=string`. -If you're constructing an expression string programmatically, it is recommended that you always use a prefix to avoid ambiguity. +If you're constructing an expression string programmatically, **always use a prefix** to avoid ambiguity. #### Escape sequences -The `~string` and `=string` name matchers can contain escape sequences, preceded by a backslash (`\`). +The *equality*, *contains*, and *glob* matchers can contain escape sequences, preceded by a +backslash (`\`). - `\n`: line feed - `\r`: carriage return @@ -101,6 +107,8 @@ The `~string` and `=string` name matchers can contain escape sequences, preceded - `\,`: comma - `\u{7FFF}`: 24-bit Unicode character code (up to 6 hex digits) +For the *glob matcher*, to match against a literal glob metacharacter such as `*` or `?`, enclose it in square brackets: `[*]` or `[?]`. + All other escape sequences are invalid. The _regular expression_ matcher supports the same escape sequences that [the regex crate does](https://docs.rs/regex/latest/regex/#escape-sequences). This includes character classes like `\d`. Additionally, `\/` is interpreted as an escaped `/`. @@ -130,3 +138,29 @@ Within a precedence group, operators bind from left to right. - `test(a) | test(b) & test(c)` is equivalent to `test(a) | (test(b) & test(c))`. - `test(a) & test(b) - test(c)` is equivalent to `(test(a) & test(b)) - test(c)`. - `not test(a) | test(b)` is equivalent to `(not test(a)) | test(b)`. + +## More information about filter expressions + +This section covers additional information that is of interest to nextest's developers and curious readers. + +
+Click to expand + +### Motivation + +Developer tools often work with some notion of sets, and many of them have grown some kind of domain-specific query language to be able to efficiently specify those sets. + +The biggest advantage of a query language is *orthogonality*: rather than every command having to grow a number of options such as `--include` and `--exclude`, developers can learn the query language once and use it everywhere. + +### Design decisions + +Nextest's filter expressions are meant to be specified at the command line as well as in configuration. This led to the following design decisions: + +* **No quotes:** Filter expressions do not have embedded quotes. This lets users use either single (`''`) or double quotes (`""`) to specify filter expressions, without having to worry about escaping them. +* **Minimize nesting of parens:** If an expression language uses parentheses or other brackets heavily (e.g. Rust's [`cfg()` expressions]), getting them wrong can be annoying when trying to write an expression. Text editors typically highlight matching and missing parens, but there's so such immediate feedback on the command line. +* **Infix operators:** Nextest's filter expressions use infix operators, which are more natural to read and write for most people. (As an alternative, Rust's [`cfg()` expressions] use the prefix operators `all()` and `any()`). +* **Operator aliases:** Operators are supported as both words (`and`, `or`, `not`) and symbols (`&`, `|`, `+`, `-`, `!`), letting users write expressions in the style most natural to them. Filter expressions are a small language, so there's no need to be particularly opinionated. + +[`cfg()` expressions]: https://doc.rust-lang.org/reference/conditional-compilation.html + +
diff --git a/site/src/book/running.md b/site/src/book/running.md index 068256840bb..0304157a9e9 100644 --- a/site/src/book/running.md +++ b/site/src/book/running.md @@ -14,14 +14,9 @@ In the output above: - Tests are marked **`PASS`** or **`FAIL`**, and the amount of wall-clock time each test takes is listed within square brackets. In the example above, **`test_list_tests`** passed and took 0.052 seconds to execute. - Tests that take more than a specified amount of time are marked **SLOW**. The timeout is 60 seconds by default, and can be [changed through configuration](configuration.md). -- The part of the test in purple is the _test binary_. A test binary is either: +- The part of the test in purple is the _binary ID_ for a unit test binary (see [Binary IDs](#binary-ids) below). - - a _unit test binary_ built from tests inline within `lib.rs`. These test binaries are shown by nextest as just the crate name, without a `::` separator inside them. - - an _integration test binary_ built from tests in the `[[test]]` section of `Cargo.toml` (typically tests in the `tests` directory.) These tests are shown by nextest in the format `crate-name::bin-name`[^bin-example]. - - For more about unit and integration tests, see [the documentation for `cargo test`](https://doc.rust-lang.org/cargo/commands/cargo-test.html). - -- The part after the test binary is the _test name_, including the module the test is in. The final part of the test name is highlighted in bold blue text. +- The part after the binary ID is the _test name_, including the module the test is in. The final part of the test name is highlighted in bold blue text. `cargo nextest run` supports all the options that `cargo test` does. For example, to only execute tests for a package called `my-package`: @@ -31,7 +26,15 @@ cargo nextest run -p my-package For a full list of options accepted by `cargo nextest run`, see `cargo nextest run --help`. -[^bin-example]: `bin` and `example` targets can also contain tests. Those are represented as `crate-name::bin/bin-name` and `crate-name::example/example-name`, respectively. +## Binary IDs + +A test binary can be any of: + +- A _unit test binary_ built from tests within `lib.rs` or its submodules. The binary ID for these are shown by nextest as just the crate name, without a `::` separator inside them. +- An _integration test binary_ built from tests in the `[[test]]` section of `Cargo.toml` (typically tests in the `tests` directory.) The binary ID for these is has the format `crate-name::bin-name`. +- Some other kind of test binary, such as a benchmark. In this case, the binary ID is `crate-name::kind/bin-name`. For example, `nextest-runner::bench/my-bench` or `quick-junit::example/show-junit`. + +For more about unit and integration tests, see [the documentation for `cargo test`](https://doc.rust-lang.org/cargo/commands/cargo-test.html). ## Filtering tests