Add TreatFocusedAs option?

[stevladimir]

There is TreatPendingAs option, which per my understanding is roughly an equivalent to --fail-on=pending option of hspec. Is it possible to have an equivalent for --fail-on=focused?

More general question... Probably it is feasible to have smth like --hspec-opt, which will allow to pass options whatever hspec accepts? Obviously, some options may be incompatible/useless, but it might be more robust to explicitly warn (or just mention in docs) on such options than implement all potentially useful options on ad hoc basis. At first glance one can re-use https://hackage.haskell.org/package/hspec-core-2.11.7/docs/Test-Hspec-Core-Runner.html#v:readConfig.

I can give it a try and submit PR if such changes will be supported

[mitchellwrosen]

I think an equivalent for --fail-on=focused makes sense. I don't think its name should be TreatFocusedAs though, because "treat focused as success" doesn't really make sense – it's more like just toggling whether having anything focused at all triggers an immediate failure.

When I made up the name TreatFocusedAs, I was unaware of the command-line flag --fail-on=pending. It would have been good to be consistent there.

I'm a little less sure about trying to wire together hspec's argument parser and tasty's options mechanism. For starters, that readConfig probably won't do because it does IO.

I propose the following:

1. We standardize on a prefix for hspec options so that they don't clash with any other tasty options (e.g. --hspec-*)
2. We deprecate treat-pending-as in favor of fail-on, matching hspec (so the tasty test suite command-line flag would be --hspec-fail-on=...)

[mitchellwrosen]

Hrm, I've thought a bit more carefully about this and am starting to reconsider.

hspec already has a rather elaborate dance involving starting with an initial Config, modifying that config from within the spec itself, reading command line options, environment variables, and config files to further modify the config... it would be good to reuse all of that functionality in tasty-hspec.

However, I'm not totally sure about what to do about the command-line options. What are our options?

[stevladimir]

However, I'm not totally sure about what to do about the command-line options. What are our options?

Well, I'm not at a stage to say something concrete. I need to dive into both hspec and tasty-hspec internals to see what is reachable. As I've mentioned initially my main concern (ignoring implementation complexity) is that many of hspec options might either duplicate or conflict with already existing own tasty options or be useless at all (e.g. output formatting options). So the most general idea of --hspec-opt <list-of-hspec-arguments> may not work well in practice.
Still it may be reasonable to re-use existing hspec machinery to not reinvent the wheel. E.g. we have reduced set of --hspec-, which we collect, convert into --<some-hspec-option> and feed that to hspec.readConfig or similar. Thus on the tasty-hspec side we only need to do the job of collecting those args.
Will explore options list for both and then come back with more well-thought proposals.