Expose the max number of retries to formatters

[jan-molak]

Hey @davidjgoss and @charlierudolph!

Would it be possible to make the information about the max number of retries specified via --retry parameter available to Cucumber formatters?

At the moment Cucumber messages provide information about the current attempt number, for example:

{"testCaseStarted":{"attempt":1,"timestamp":{"seconds":"1609892232","nanos":945000000},"testCaseId":"d6c24ef0-66c4-42f7-a62e-2d6b976a40ff","id":"9b5d1c83-09a3-466d-8c14-e3c2b14918ea"}}

However, they don't tell how many attempts there are left, which makes reporting a bit inconvenient.

Perhaps this information could be added to IFormatterOptions either to the main options object itself or under parsedArgvOptions. I think the behaviour could be similar to how the RerunFormatter works.

Is this a feature you'd be willing to add / review a PR for?

[davidjgoss]

Moved this to the main repo, as I think it would only be practically achievable by adding to the standard messages - the test cases that can be retried could be a subset because of —retry-tag-filter so a global setting would seem off to me.

I think this would need to live on the TestCase message as eg “maximum_attempts”. Balanced against that, it’d have impact across implementations and is perhaps a bit niche.

@charlierudolph @mpkorstanje @aslakhellesoy wdyt?

[luke-hill]

Again I've not dived into the protobuf stuff much, but what I would think is that if a specific scenario failed once but passed 2nd time around, that wouldn't affect another scenario. As such to save weight (And possibly time), couldn't this be something that wraps the whole message structure at the top level?

[davidjgoss]

@luke-hill the issue with putting something like maximum_attempts at the top level would be that it only applies to some test cases (depending on --retry-tag-filter).

[luke-hill]

As in it only applies if the original test failed? Or if the param was input?

Because if the param is input then you can hard-code the value to 1 But maybe I'm misunderstanding.