Tag scoped step definitions

[derSchtefan]

🤔 What's the problem you're trying to solve?

Bind step definition to tags (e.g. 'When I press "Save"' multiple definitions for different tags).

Many steps are worded identical, but refer to different operations of the system. Consider a web interface, where multiple ways to display the interface exist, multiple pages, or dialogs, all with a "Save" button. They might need different implementation on how the save action is triggered in the test glue code, but right now you would always need to specify something clumsy like When I press "Save" (on page X), which is not only odd to read but raises questions.

In the glue code you would have multiple implementations, all with slightly different wordings, making it a huge mess to come up with variants on how you write the same action.

Another very common occurrence is the "Then I expect no error to be shown" steps. This is usually easily scopeable with a tag in a feature file talking about whatever way errors are displayed or returned in that particular part of the system. However we find ourselves always having to write the error validations absurdly verbose.

✨ What's your proposed solution?

SpecFLOW (for .NET) has solved this quite elegantly: A step definition can be bound to one or multiple tags. It is then only relevant for Scenarios or features with that specific tag. This way the wording can be sharper and shorter, and the glue logic is more readable.

⛏ Have you considered any alternatives or workarounds?

Right now we have to pre or postfix steps that are worded the same but have to trigger two different actions in the test implementation, and it does actually work against the idea of Cucumber: to make the specification independent from the implementation

[mpkorstanje]

Moving this to common. This would affect more then one implementation.

[SabotageAndi]

Hi, I am from the SpecFlow Team.
Yes, we have this feature in SpecFlow, but if I would write SpecFlow new from scratch, I would put it at the very end of the backlog for it. This is my personal opinion, not of the SpecFlow Team.

This feature increases the complexity of navigation in editors a lot. A lot more code needs to be parsed to find the right location to jump to.
At least in SpecFlow, there are other ways to have "different behavior" of the step depending on tags, as you could check the Tag list yourself and call then different methods.

[derSchtefan]

@SabotageAndi I can't follow your reasoning. Checking the tag list yourself sounds like a hack, not a feature. It would require coordination between developers, centralizing the implementation of a step that is profoundly different between areas, just to read a tag and then re-route the method call. THAT is messy regarding "go to source". Guess what will happen next? A developer will say "can't this be automated"?

SpecFlow right now has this beautifully implemented, as the steps and the implementation is loosely coupled, and different areas do not interfere with each other. Gherkin/SpecFlow saved us from test coupling hell.

If a person wants to navigate to the source, and multiple implementations exist, and it is hard to find the right one, I don't expect the IDE to be smart. Simply show a list of all implementing methods and let the user choose. For what it is worth, most IDEs can't even resolve overloads when navigating statically typed source code, and we are all fine with it.

[SabotageAndi]

My reasoning is from how often I see tag scoping used and how much work it is to support it everywhere. Not from a user perspective, simply from a maintainer perspective.
For me personally, the time spent on bringing this to work, is too much, as there are workarounds. But it is here in SpecFlow and finally working in Visual Studio (after years of instability issues).

I don't expect the IDE to be smart. Simply show a list of all implementing methods and let the user choose. For what it is worth, most IDEs can't even resolve overloads when navigating statically typed source code, and we are all fine with it.

I have other experience with .NET developers and our users. MS is setting the bar with Visual Studio very high and so our users expect that this is working.

And I have no say if this gets implemented in Cucumber or not. @mattwynne asked me yesterday, that I write a comment about my experience with this in SpecFlow here.
And again, this is my opinion. @gasparnagy has a different one on this feature.

[jenisys]

@derSchtefan
Could you provide slightly more details what you really need instead of the solution you thought out.
For example:

• Do you need different step implementations in different directory scopes ?
• Or: Are the variants of this step-implementation needed in the same directory ?
• Etc.

NOTE:
You are basically asking for a multi-method implementation of a list of step-implementations with the same pattern (or slightly different patterns) in different contexts. This context could be selected by tags, directory scope, etc.

[gasparnagy]

Although I have found the tag-scoping useful is some specific case, I agree to @SabotageAndi that the maintenance complexity probably outweighs the benefits, especially because there are workarounds.

[mattwynne]

We've always resisted adding this kind of functionality to Cucumber. As @richardlawrence has pointed out many times before, there's a great power in Cucumber's adherence to a ubiquitous language: wherever you see the same phrase in the Gherkin, you know it always means the same thing. I'm really not convinced by the examples given that it would be a good idea to break this axiom.

If you really need this, you can built it yourself using polymorphism in your support code: wrap the code that interacts with each page in an object with a common pressSave() or validateNoErrorsDisplayed() method on it, and store the current page in a field on the step definition class.

[gasparnagy]

@mattwynne While I agree to your conclusion, I would like to highlight that ubiquitous language is not about this. Ubiquitous language is about a language that everyone understands. This is what we also call the business language, because that describes the requirements in a way that they are understood by everyone.

The scoped step definitions do not change the meaning of the steps! They only provide different automation solution to achieve the same. From this point of view, they are a simple implementation of the hexagonal architecture, where you just use different adapters for the same thing. So with the same argument you would need to sacrifice the hexagonal architecture as well.

When I used tag scoped steps, I have used it to achieve a test automation pyramid like shape in a way that a few "happy path" scenario of a feature have been automated on the outside interface of the app, but the most of them have been automated on the domain level. But the scenarios used the same steps as they were describing the same: the requirements. I'm not saying that I could not have solved it with the workarounds you mention, but it was clean and useful.

[mattwynne]

@gasparnagy I don't think ubiquitous language is only about the ubiquity of who uses it - although that's an important aspect. In it's essence, for me, it's about "using the same words for stuff" wherever you are in the stack - from user guides to the database schema (at least, within one bounded context). This includes being specific about distinct domain concepts.

Perhaps what I missed in my mention of ubiquitous language is my experience is that the constraint of having to use unique terms in your Gherkin steps for unique actions against the system in your step definitions can help force you to think about what your ubiquitous language is.

For example, if you can just say When I press save all over the place, and use tag scoping to change what that does underneath, you never get forced to really think about what the difference is between pressing save in what could be two very different contexts. If, however, you accept the constraint that you have to find unique terms for those distinct actions, you're actually forced to think about how you want to describe them.

So for example, When I press save in the context of a user profile dialog might become When I update my user profile and When I press save in the context of the new recording pane might become When I publish the new recording. I find it positively valuable to be forced to think about these words.

Gaspar your example of using tagged scopes to run scenarios at different levels is really interesting (and sounds much more valuable than the OP's suggested use case to me) but I'd prefer to see that implemented in something like Behat's suites rather than being mixed up in a single test run.

[gasparnagy]

@mattwynne I agree, but unfortunately the test runs are (usually) managed by the execution platform (in SpecFlow's case that would be the .NET xUnit runner, for example), where the possibilities might be limited or not in our control. In our case there is no easy way to define such suites there.

[mattwynne]

@gasparnagy interesting, but probably a separate discussion!

(ref: cucumber/cucumber-ruby#821, cucumber/cucumber-js#316)

[govindasharma1986]

Hi, I am in support of the feature: tags at step level but with a different reason.

I am automating a highly dynamic application.
The order of pages, number of pages, fields of each page and number of fields in each page are never fixed.

Hence, I decided test each fields in separate scenario so that I can skip tests using tags(scenario level) for those fields which need not be covered for current screen.

environment.py
------------------
def before_scenario(context, scenario):
    if 'skip' in scenario.tags:
        scenario.skip("Scenario was Skipped. Reason: Scenario was marked as @skip")

    tags_with_fields = [x for x in scenario.tags if 'field-' in x or 'fields-' in x]

    if len(tags_with_fields) > 1 and 'fields-all' in tags_with_fields:
        scenario.skip('Scenario was Skipped. Reason: Cannot use `fields-all` along with `field-{field_name}')

    count_match = 0
    for field_name in context.property.fields_under_test:  # context.property.fields_under_test is a list and setup in config.ini file
        for tag in tags_with_fields:
            if field_name in tag:
                count_match += 1
    
    if len(tags_with_fields) - count_match > 0 and 'fields-all' not in tags_with_fields:
        scenario.skip('Scenario was Skipped. Reason: Fields covered in the scenario are out of scope for the given template')

@field-mobile
    Scenario: Testing a particular field of the page
        ...

However, there are some tests(scenarios) which test for all fields in the page. The test would not be meaningful if I segregated them field wise. Hence I thought, if I could skip steps inside a scenario then it would have been very flexible to impl. such case.

@field-zipcode
    Scenario: To validate that when user enters valid zipcode then city and state are auto-fetched
        Then In Local Storage, key "city" should have value "Santa Clara"
        And In Local Storage, key "stateCode" should have value "CA"
        @field-city  # would like to skip this step if field city is not present in current page. (sometime a field would never be visible in any page)
        And field "city" should have value "Santa Clara"
        @field-state  # would like to skip this step if field state is not present in current page. (sometime a field would never be visible in any page)
        And field "state" should have value "CA"

Otherwise, my coverage would be less.

Let me know if this helps.

Note: I am not looking to reverse target scenarios/features using tags applied in steps
for eg: I don't intent to use tags at step in command line.
python -m behave --tags=@field-name

I hope this helps.

Ref: behave/behave#1057

[govindasharma1986]

@mpkorstanje @jenisys

Have added details in the above comment. Please let me know the way forward.