Mark failed tests Skipped for a class of errors

[tgsmith61591]

We have a very large suite of cucumber tests driven by cucumber-rs. There are a few live integration tests of third party services we depend on that can fail for HTTP 500 errors (for example, amongst others).

There is a subset of upstream errors that we do not want to allow to block our CI pipeline, but would like to mark Skipped, which will keep our development team unblocked for any flakiness that is outside their control.

We have managed this internally by creating a custom writer, where for a failed step we inspect the error and call self.step_skipped rather than self.step_failed for certain circumstances.

Pseudo code:

pub trait MarkSkipped {
    fn skip(&self, &StepError) -> bool;
}

#[derive(Clone, Debug, Deref, DerefMut)]
pub struct CustomWriter<M: MarkSkipped = crate::NoSkip, Out: io::Write = io::Stdout> {
    #[deref]
    #[deref_mut]
    output: Out,
    styles: Styles,
    indent: usize,
    lines_to_clear: usize,
    verbosity: Verbosity,
    terse: bool,
    mark_skipped: M,
}

We would love to ditch the custom writer and get back to the cucumber-rs internal writers, since it allows us to more regularly upgrade to the latest version with greater ease.

I am happy to upstream this feature to the package, if the maintainers feel there is any value to it. I wanted to run it by the team before I put in the effort to create the feature!

[tyranron]

@tgsmith61591 sorry for getting back very late on this. I generally don't mind to upstream such functionality (though, bikeshedding is always a deal 😄), it seems to be helpful for many cases. In our codebases, we deal with such flakiness in other way and haven't thought about such solution, so it could benefit us too.