Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit data value name length in unrolled test #1702

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

T45K
Copy link
Contributor

@T45K T45K commented Jun 24, 2023

Background

We're using Spock for testing of our product.
The other day, we found some tests using large size images in where clause were stacked and didn't finish at all.
I investigated the root cause, and it was the following reason.

  1. Spock shows data name and value in test case name when we use where clause.
  2. Even if we use objects which are represented by too long string in where clause, Spock tries to render the name.
  3. Then, stacking will occur because showing too long string takes much time.

Proposal

Limiting data value name length to avoid unintentional stacking.

Sample project

I prepared https://github.com/T45K/spock-stack-sample
You can reproduce my result by cloning it and importing into IDE (I'm using IntelliJ IDEA).

On master branch, original Spock is used, and you will find AppTest.stack because test target will be shown in test name will take much time.

On solution branch, I set Spock built in this PR. You will find the test is much faster.

Others

I used 1024 as max length. This is just my preference (it is easy to understand because 2 ^ 10). If you have any recommendation, I'll follow.

Thank you.

@codecov
Copy link

codecov bot commented Jun 24, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (e08fe5c) 78.18% compared to head (ba306d6) 78.18%.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1702   +/-   ##
=========================================
  Coverage     78.18%   78.18%           
- Complexity     4041     4042    +1     
=========================================
  Files           428      428           
  Lines         13080    13083    +3     
  Branches       1687     1688    +1     
=========================================
+ Hits          10226    10229    +3     
  Misses         2222     2222           
  Partials        632      632           
Impacted Files Coverage Δ
...rk/runtime/DataVariablesIterationNameProvider.java 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@T45K T45K marked this pull request as draft June 24, 2023 13:38
@T45K T45K marked this pull request as ready for review June 24, 2023 13:47
@Vampire
Copy link
Member

Vampire commented Jun 24, 2023

Did you consider to just use a custom test naming for that test? That the data variables are rendered into the iteration name is just the default. But you can customize the default pattern used on all features and you can just use a custom pattern on any individual feature.

@T45K
Copy link
Contributor Author

T45K commented Jun 25, 2023

Thank you for checking this PR.

Yes. Currently, I'm using the feature for our product to avoid stacking.

The main reason why I raised this PR is that it was too hard (at least for me) to find out reasons why the tests are not completed.
I guess that users need only first few characters of data value name to identify each test cases in almost all cases, and they want to avoid unintentional stacking.

Copy link
Member

@leonard84 leonard84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this only cause problems in IntelliJ or also in other environments?

@@ -51,7 +51,11 @@ public String getName(IterationInfo iteration) {
dataVariables.forEach((name, value) -> {
String valueString;
try {
valueString = toStringOrDump(value);
valueString = RenderUtil.toStringOrDump(value);
int maxDataValueStringLength = 1024;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Did you try to figure out which length of method names where problematic?

❓ Does this really fix the problem comprehensively? What happens if you have more than 1 problematic parameter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants