Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Info annotation #959

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

sapanparikh18
Copy link

@sapanparikh18 sapanparikh18 commented Dec 17, 2018

An annotation, FeatureInfo, to add information about some of the basic test attributes.
This annotation then can be integrated with Spock Reports to add the following

  1. Id (Need to identify every feature uniquely)
  2. Impact (Impact of the feature, if it fails)
  3. The frequency of Failure (Probability of a feature failing)
  4. Component (Which component it may be associated with. Example: shopping cart, product page etc )

These params then cab be used to create reports like ACC (https://testing.googleblog.com/2011/10/google-test-analytics-now-in-open.html)


This change is Reviewable

…c test attributes.

This annotation then can be integrated with Spock Reports to add the following
1. Id (Need to identify every feature uniquely)
2. Impact (Impact of the feature, if it fails)
3. Frequency of Failure (Probability of a feature failing)
4. Component (Which component it may be associated with. Example: shopping cart, product page etc )

These params then cab be used to create reports like ACC (https://testing.googleblog.com/2011/10/google-test-analytics-now-in-open.html)
@codecov
Copy link

codecov bot commented Dec 17, 2018

Codecov Report

Merging #959 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #959   +/-   ##
=========================================
  Coverage     75.99%   75.99%           
  Complexity     3534     3534           
=========================================
  Files           377      377           
  Lines         10746    10746           
  Branches       1367     1367           
=========================================
  Hits           8166     8166           
  Misses         2103     2103           
  Partials        477      477

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 528b25d...009d964. Read the comment docs.

@leonard84
Copy link
Member

Hi, thanks for your contribution. However, I don't think that this will be a good fit for spock(-core).

  • Spock does nothing with this annotation, so by itself it is a useless addition for users. If you have some reporting software that would access it, then it would make more sense provide it in a separate jar provided by that software.
  • It has the same name as an internal class, so mixups might be possible.
  • It targets a very specific kind of reporting/methodology
  • re 1) it seems cumbersome to declare unique ids by hand, how should it work for data driven tests, do they all get the same id?
  • re 3) isn't that an external metric?

@sapanparikh18
Copy link
Author

Hi @leonard84
Please find my responses below.

Hi, thanks for your contribution. However, I don't think that this will be a good fit for spock(-core).

  • Spock does nothing with this annotation, so by itself it is a useless addition for users. If you have some reporting software that would access it, then it would make more sense provide it in a separate jar provided by that software.

Identifying test cases by eye is also an important feature. I think very similar examples in Spock already exists like the @issue annotation and @Narrative annotation. I am not sure if they are used outside of any report.

  • It has the same name as an internal class, so mixups might be possible.

Should I resubmit the PR with @id or @testcase or @tc annotation?

  • It targets a very specific kind of reporting/methodology

That's right, but at least an ID is needed in any testing methodology. We have a real problem of identifying test cases between 8K+ test cases and 20 automation developers. Any documentation about writing test cases will indicate that having an ID is an important aspect of test case documentation, especially when we don't look at Spock just a Unit test framework but also a great documentation framework which documents the behavior of a system. But point well taken, will a new PR with just ID annotation work?

  • re 1) it seems cumbersome to declare unique ids by hand, how should it work for data-driven tests, do they all get the same id?

Yes, test data is independent of the test case being tested. Each test case should have unique identification but not the execution.

  • re 3) isn't that an external metric?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants