Concept hooks return status response

[PiotrNestor]

@sriv
Adding management of concept hooks returning status response.

The related updates are:

• gauge-proto: https://github.com/system-verification/gauge-proto/tree/return-concept-status
• gauge-dotnet: https://github.com/system-verification/gauge-dotnet/tree/return-concept-status

For some reason there is no response / result returned by 'e.runner.ExecuteAndGetStatus' in notifyBeforeConcept, notifyAfterConcept even if the 'gauge-dotnet' has been updated to return a result.

grpcRunner executeMessage (that is called by ExecuteAndGetStatus) returns response == nil

What might be the reason for that?
What to check?

[github-actions]

Benchmark Results

java_gradle_multithreaded.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	10%	108120	0:23.65	0
6475093	9%	114264	0:23.61	0
e1b8dd9	10%	119588	0:22.49	0
3adaae8	10%	115416	0:23.17	0

java_maven_serial.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	84%	216004	0:16.96	0
6475093	83%	260176	0:17.12	0
e1b8dd9	75%	234640	0:19.04	0
3adaae8	80%	249332	0:17.78	0

java_simple_multithreaded.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	38%	67900	0:09.27	0
6475093	30%	68364	0:12.14	0
e1b8dd9	34%	65960	0:10.23	0
3adaae8	26%	67648	0:13.15	0

java_gradle_serial.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	9%	118740	0:25.15	0
6475093	9%	117392	0:25.12	0
e1b8dd9	9%	120024	0:26.14	0
3adaae8	9%	123552	0:24.22	0

java_simple_parallel.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	24%	72212	0:22.78	0
6475093	22%	72064	0:23.98	0
e1b8dd9	24%	72016	0:22.87	0
3adaae8	22%	74272	0:24.23	0

java_gradle_parallel.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	5%	115740	0:40.07	0
6475093	5%	116668	0:46.13	0
e1b8dd9	5%	120188	0:41.12	0
3adaae8	5%	122104	0:39.72	0

java_maven_parallel.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	43%	231220	0:31.14	0
6475093	44%	243516	0:29.99	0
e1b8dd9	40%	237048	0:33.03	0
3adaae8	42%	236704	0:31.20	0

java_simple_serial.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	52%	66024	0:11.35	0
6475093	51%	65716	0:11.90	0
e1b8dd9	53%	67896	0:11.58	0
3adaae8	43%	65560	0:14.26	0

java_maven_multithreaded.csv

Commit	CPU	Memory	Time	ExitCode
b55b734	67%	234752	0:16.79	0
6475093	58%	202856	0:19.71	0
e1b8dd9	65%	245688	0:17.74	0
3adaae8	75%	237916	0:15.44	0

Notes

• The results above are generated by running against seed projects in https://github.com/getgauge/gauge-benchmark
• These results are not persisted, but on merging to master the benchmark will be rerun.
• These benchmark are run in Github Actions' agents, which are virtualized. Results are not to be taken as actual benchmarks.Rather, these are indicative numbers and make sense for comparison.

---

See Workflow log for more details.

[PiotrNestor]

@sriv
OK, I found a problem in 'invokeServiceFor' not returning the correct response :(
The question now is what to do in 'executeConcept' when there is a Concept start / end response error

[sriv]

The question now is what to do in 'executeConcept' when there is a Concept start / end response error

I guess we can mark the concept execution result as failed like how its done here: https://github.com/getgauge/gauge/blob/master/execution/result/conceptResult.go#L53-L66

I am fairly certain that spec/scenario hooks behave similarly, i.e. mark them as failed when the hooks have failures.

[PiotrNestor]

@sriv
I have now an implementation that handles Concept hook result with an error.
I have verified the status and error with the following reporters:

  "Plugins": [
    "reportportal",
    "html-report",
    "xml-report"
  ],

When there is a Concept hook error, the reporters

• correctly indicate that the scenario has failed
• there is an error message and stack info
• there is presently no screenshot attached- should this be added
• there are some details that the reporters could improve

[PiotrNestor]

@sriv

Simulating a 'BeforeConcept' hook results in the following html-report
The error information is provided but it's not entirely clear that the concept is failing

Simulating a 'AfterConcept' hook results in the following html-report
The error information is provided but it's not entirely clear that the concept is failing

[sriv]

Thanks for testing this out and the feedback. I believe your feedback is around the placement of the error message?

html-report renders concepts as an expandable step with the actual steps listed under it. I guess you would like the errors in the concept hooks should be adjacent to this expandable concept?

This should be possible by changing the template for html-report. The default template lies here https://github.com/getgauge/html-report/blob/master/themes/default/views/partials.tmpl

This uses golang's stdlib templating and the concept container is here: https://github.com/getgauge/html-report/blob/master/themes/default/views/partials.tmpl#L687-L698

If you want to take a stab at this please feel free.

[PiotrNestor]

@sriv
Thanks for the explanation and pointing out the template functionality.
My understanding is then that there is no additional logic required in the gauge runner in order to correctly handle the concept hook status response.
In that case, can we proceed with reviewing and accepting the related gauge-proto and gauge-dotnet PRs?

[sriv]

Done, merged the corresponding gauge-dotnet and gauge-proto PRs

[PiotrNestor]

@sriv
This PR is ready ...

[gaugebot]

@PiotrNestor Thank you for contributing to gauge. Your pull request has been labeled as a release candidate 🎉🎉.

Merging this PR will trigger a release.

Please bump up the version as part of this PR.

Instructions to bump the version can found at CONTRIBUTING.md

If the CONTRIBUTING.md file does not exist or does not include instructions about bumping up the version, please looks previous commits in git history to see what changes need to be done.