Append generated test macro so next test macros are aware of it

[kezhuw]

This way test macros following #[rstest] can decide whether or not to generate test macro to avoid duplicate test runs.

It is an attempt to improve capabilities among test macros. Currently, following test from googletest(main branch at 2025-01-16) will run twice.

#[rstest]
#[case(1)]
#[gtest]
fn paramterised_test_should_work_with_rstest_first(#[case] value: u32) -> Result<()> {
    verify_that!(value, eq(value))
}

See: tokio-rs/tokio#6497, d-e-s-o/test-log#46, frondeus/test-case#143, kezhuw/stuck#53.
Refs: rust-lang/rust#67839, rust-lang/rust#82419.

[la10736]

Ok the issue here is another: rstest doesn't recognize gtest as a valid test attribute. The rule is: last segment of your test attribute should start with test.

So, in this case you can work around to it with something like follow:

pub mod google {
    pub use gtest as test;
}

#[rstest]
#[case(1)]
#[google::test]
fn paramterised_test_should_work_with_rstest_first(#[case] value: u32) -> Result<()> {
    verify_that!(value, eq(value))
}

[kezhuw]

So, in this case you can work around to it with something like follow:

googletest did this already, it has googletest::test.

The motivation from my side is to ease users of libraries. Attribute macros can only see macros following them. If we are going to generate macros in-place, then we will encounter this again for xtest, ytest. Then, users of test libraries will have to deal with duplicated test runs if they are ever aware of them.

By conforming to rules:

1. Don't generate test macro if it already exists.
2. Append but not prepend test macro so following macros can detect newly generated test macros.

Then, we resolve the problem in libraries but not spread it to users of libraries.

[la10736]

Attribute macros can only see macros following them

Suddenly that is false 😢 ... And the sad part is that there isn't any no hacking way to identify what is previous and what is after: you can just try to guess it by code info in span but are just heuristic: that was the state of art 6 moth ago when I try to get the list of the attributes that precede rstest and the one that are after.

[la10736]

1. Don't generate test macro if it already exists.

That's what rstest do when recognize a test attribute with that rule that I wrote before

https://docs.rs/rstest/latest/rstest/attr.rstest.html#inject-test-attribute

[kezhuw]

That's what rstest do when recognize a test attribute with that rule that I wrote before

Yeh, that is great. This pr completes the second part.

Suddenly that is false 😢 ... And the sad part is that there isn't any no hacking way to identify what is previous and what is after: you can just try to guess it by code info in span but are just heuristic: that was the state of art 6 moth ago when I try to get the list of the attributes that precede rstest and the one that are after.

I am not sure what have you experienced. May be we have some misunderstandings here. But from what I known(rust-lang/rust#67839 (comment), rust-lang/rust#82419), ItemFn.attrs is attribute macros following the processing macro. From my understanding(might be wrong though), procedure of test macro never see preceding macros.

Take the following code as an example.

#[rstest]
#[gtest]
fn rstest_with_googletest() -> Result<()> {
  verify_that!(1, eq(1))
}

After #[rstest] processed, the generated code is similar to following.

#[test]
#[gtest]
fn rstest_with_googletest() -> Result<()> {
  verify_that!(1, eq(1))
}

googletest generates test macro in-place only if there is no test attribute which is similar to how rstest behaves. Sadly #[test] is prepended before #[gtest], so the procedure of #[gtest] is not aware of it. Then, #[gtest] generate yet another test macro. This results in duplicated test runs.

[la10736]

Ok, is little bit wired and is not simple to catch because normally you are thinking in terms of custom attributes, but I dumped the attributes received by custom procedural macro inspect_attrs in the follow example:

#[ignore]
/// Doc comment before
/* Multi line
comment */
// Simple comment before
#[attr_before::inspect_attrs(some)]
// Simple comment
/// Doc comment

#[ignore]
#[after]
fn ignore_attr() {
    println!("done");
}

Attribute { pound_token: Pound, style: AttrStyle::Outer, bracket_token: Bracket, meta: Meta::Path { leading_colon: None, segments: [PathSegment { ident: Ident { ident: "ignore", span: #0 bytes(36..42) }, arguments: PathArguments::None }] } },
Attribute { pound_token: Pound, style: AttrStyle::Outer, bracket_token: Bracket, meta: Meta::NameValue { path: Path { leading_colon: None, segments: [PathSegment { ident: Ident { ident: "doc", span: #0 bytes(44..66) }, arguments: PathArguments::None }] }, eq_token: Eq, value: Expr::Lit { attrs: [], lit: Lit::Str { token: " Doc comment before" } } } },
Attribute { pound_token: Pound, style: AttrStyle::Outer, bracket_token: Bracket, meta: Meta::NameValue { path: Path { leading_colon: None, segments: [PathSegment { ident: Ident { ident: "doc", span: #0 bytes(171..186) }, arguments: PathArguments::None }] }, eq_token: Eq, value: Expr::Lit { attrs: [], lit: Lit::Str { token: " Doc comment" } } } },
Attribute { pound_token: Pound, style: AttrStyle::Outer, bracket_token: Bracket, meta: Meta::Path { leading_colon: None, segments: [PathSegment { ident: Ident { ident: "ignore", span: #0 bytes(190..196) }, arguments: PathArguments::None }] } },
Attribute { pound_token: Pound, style: AttrStyle::Outer, bracket_token: Bracket, meta: Meta::Path { leading_colon: None, segments: [PathSegment { ident: Ident { ident: "after", span: #0 bytes(200..205) }, arguments: PathArguments::None }] } }

As you can see all 5 attribute are received ignore, doc comment before, doc comment after and the other two that came after.

[la10736]

I am not sure what have you experienced. May be we have some misunderstandings here. But from what I known(rust-lang/rust#67839 (comment), rust-lang/rust#82419), ItemFn.attrs is attribute macros following the processing macro. From my understanding(might be wrong though), procedure of test macro never see preceding macros.

You are right but... not all attributes are macros 😢

[la10736]

Anyway, I'm not against your PR (beside that's braking some tests 😄 ) ... I'm just a little bit scared because that's a breaking change. and in general rstest is trying to solve it with the convention that I mentioned before and and the possibility to ignore some arguments in order to have a better compatibility with other macros.

[kezhuw]

in general rstest is trying to solve it with the convention that I mentioned before and and the possibility to ignore some arguments in order to have a better compatibility with other macros.

Do you means "That's what rstest do when recognize a test attribute with that rule that I wrote before" ?

If it is, this pr does not change this, actually depends on this.

that's braking some tests 😄
I'm just a little bit scared because that's a breaking change.

Let me figure out what it breaks. And let's see whether it matters.

[kezhuw]

Seems that the test fails due to some concurrent issues in test setup. It passed on my fork with 2 runs (https://github.com/kezhuw/rstest/actions/runs/12812929918/job/35727841633).

I saw similar failure in https://github.com/la10736/rstest/actions/runs/12177266364/job/33964724311.

[kezhuw]

I read some codes and think add_dependency is not concurrent safe with compile and other io functions.

[la10736]

Seems that the test fails due to some concurrent issues in test setup. It passed on my fork with 2 runs (https://github.com/kezhuw/rstest/actions/runs/12812929918/job/35727841633).

I saw similar failure in https://github.com/la10736/rstest/actions/runs/12177266364/job/33964724311.

You are right, it's a well know issue. It's quite impossible to solve it without make the CI slower (de facto removing parallelism). This happen only on really slow machine like github runners but I was never able to reproduce it on my laptops.

The issue is that the test use a workspace to generate the projects for the acceptance tests and sharing the compiled dependencies, so in this context a test can modify a workspace crate while another one read the same crate to compile its project. While I'm writing this I was thinking that a way could be save the workspace Cargo.toml on another file and leverage on the atomic move to remove the issue.... I'll try this approach.

[la10736]

When I say that's a breaking change I mean that some users can leverage on that order, but never mind I don't think that's a really issue.

[la10736]

LGTM, Just add a line in chengelog please.

[la10736]

LGTM!!!

Thank!!!!