Allow customising the bisect formatter

[tubaxenor]

Currently bisect runner has 2 formatters: https://github.com/rspec/rspec-core/blob/main/lib/rspec/core/invocations.rb#L46-L47
Since they are not configurable, currently we can only parse the result from either stdout or monkey patch the formatter's bisect_repro_command method to get the repro from bisect notifier: https://github.com/rspec/rspec-core/blob/main/lib/rspec/core/bisect/coordinator.rb#L35

Is it currently possible to add customized formatters to bisect runner? Or is it ok to put up a PR to support this feature?

[pirj]

Can you monkey patch Bisect#bisect_formatter_klass_for and return your own formatter class from it?

[tubaxenor]

That's still monkey patch... Just wondering if there is a more official way or should be an official support for this 😅

[pirj]

Eventually, if there's demand. I can't really remember this request popping up while I contribute to RSpec.
It's a relatively small change code-wise, but that formatter part of the private API would have to be made public and documented.
I encourage you to monkey-patch for now, since there's low risk of this code to change, and if there's more people asking for to use custom bisect formatters, we'll consider accepting a PR.

It would be wonderful to describe your use case.

[tubaxenor]

It would be wonderful to describe your use case.

@pirj thanks for your reply. The use case for us is to build a flaky test analyzer, which we first took out failed spec batches and put each one of them to our CI nodes with --bisect options to catch the issued spec example/file. At the end of the process, we would like to gather the end result after bisecting, which basically is the repro returning after bisect runner execution.

It would be nice if the bisect runner can have the same ability as normal runner to take formatters to register on different notifications and act on them accordingly. But since I am the only one request it for now, it's reasonable to just monkey patch the code on our end 👍

[JonRowe]

I think this is a reasonable request, the bisect formatter is not the same api as our normal formatters, but the same principles could be used to make this happen. I'm not sure it will be a priority for anyone on the core team but if you'd like to work on it @tubaxenor I'm happy to help review it.

[tubaxenor]

I can try spend some time and work on this later this month, thanks for reopen this 🙇