[JENKINS-64516] Add support for Robot Framework 4.0 (#40)

* Add methods to get "skip" status * Remove redundant list type references * Add skip to RobotCaseResult * Remove unnecessary variable * Remove redundant toString() call * Apply better Java syntax * Apply better Java syntax * Parse RF4.0 styled tags * Add unit tests for skip status * Parse "SKIP" status correctly * Ignore skipped tests when counting pass percentage * Add "Skipped" trend to graph and add a column to list view * Add "skip" support for graphs, blue ocean, and list view * Add skip info to results overview * Replace output.xml with more complex * Change colour to orange * Add test to parse tags from RF 4.x output properly * Add documentation to some test output.xml files * Proper syntax for skipped cases in table * Parse tags properly * Add more skip status indications * Small refactor for parsing arguments and add "tag" to list of caught elements * Ignore skipped test cases when getting all failed cases * Replace StringBuffers with StringBuilders * Add unit test to test stacktrace parsing * Properly parse arguments and args * Add instructions for RF 4 configuration * Update documentation * Fix getFailedSince() * Move keyword parsing to a separate method * Refactor kw parsing and add for-loop parsing * Add unit test for RF4.0 nested for loop parsing * Refactor BlueOcean tests * Fix stacktrace parsing * Delete commented lines * Add test to parse if branch * Add if-branch parsing * Fix for loops and ifs with no contents and prettify indentations * Add test to check empty for loop * Add tests to parse stacktrace * Fix test expected values * Fix stacktrace newlines * Fix keyword and arg parsing * Add default to switch case * Add comment for clarity * Update doc/README.md Co-authored-by: Tatu Kairi <[email protected]> * Update doc/README.md Co-authored-by: Tatu Kairi <[email protected]> * Update src/main/java/hudson/plugins/robot/RobotParser.java Make kw the default Co-authored-by: Tatu Kairi <[email protected]> * Unify structures * Add explicit typing for clarity * Add braces for clarity * Add braces to if Co-authored-by: Tatu Kairi <[email protected]>
jenkinsci · Mar 12, 2021 · f4c36a9 · f4c36a9
1 parent 8fc0df6
commit f4c36a9
Show file tree

Hide file tree

Showing 37 changed files with 1,604 additions and 187 deletions.
diff --git a/doc/README.md b/doc/README.md
@@ -107,10 +107,10 @@ build. Conversely, the project view side panel link won't go away until
 after configuration change/resave in project. We're working on a
 solution to this.
 
-![](images/loghtml_sidepanel.png)
+![sidepanel link](images/sidepanel.png)
 *side panel link*
 
-![](images/robo_loglink_projectpage.png)  
+![summary link](images/robot_4_table.png)
 *Summary link*
 
 #### Configuring Robot overall pass/fail to show in the project list
@@ -128,7 +128,7 @@ solution to this.
     [http://YOURJENKINSHOST/configure](http://yourjenkinshost/configure)
     )and select your newly created view to be the default one.
 
-![](images/robot_view_column.png)
+![List view](images/robot_view_column.png)
 *List view column in action*
 
 #### Using token macros in e-mail report
@@ -178,25 +178,36 @@ and [here](https://content-security-policy.com/) how to change you CSP settings
 but be aware that **Changing CSP settings will potentially expose you Jenkins instance for
 security vulnerabilities**.
 
-### Overall Screenshots
+### Robot Framework 4.x compatibility
 
-##### Config
+The plugin supports both Robot Framework 3.x and 4.x output files. However, in order to support both, the plugin
+shows some extra information for both. [In Robot Framework 4.0 test criticality was removed and "SKIP" status was added](https://github.com/robotframework/robotframework/blob/master/doc/releasenotes/rf-4.0.rst). So for 3.x the
+results overview will show a `Skipped` column, which will always be 0 and for Robot Frameowork 4.x output files
+the `Critical tests` row will always be 0.
 
-![](images/config1.png)
+Skipped tests aren't taken into account when calculating pass percentage, but they are calculated to the total
+amount of tests.
 
-##### Project view
+Because criticality was removed in Robot Framework 4.0, having the `Use thresholds for critical tests only` checkbox
+checked will always result in a passing step (because pass percentage is always considered to be 100% when there are
+0 tests). In order to have set build status correctly, you **must** uncheck the checkbox or use `onlyCritical: false`
+in your pipeline when you call `robot`.
 
-![](images/project_page.png)
+We are planning of dropping support for Robot Framework 3.x and earlier some time in distant future.
+
+### Overall Screenshots
+
+
+##### Project view
 
-##### Build view
+![Build page overview](images/build_page.png)
 
-![](images/build_page.png)
 
 ##### Detailed build view
 
-![](images/detailed.png)
+![Detailed results view](images/detailed.png)
 
 ## Bugs
 
 [List of open bugs on
-robot-plugin](https://issues.jenkins-ci.org/issues/?jql=component%20%3D%20robot-plugin%20AND%20resolution%20is%20EMPTY%20ORDER%20BY%20updated%20DESC)
+robot-plugin](https://issues.jenkins-ci.org/issues/?jql=component%20%3D%20robot-plugin%20AND%20resolution%20is%20EMPTY%20ORDER%20BY%20updated%20DESC)
diff --git a/doc/images/build_page.png b/doc/images/build_page.png
diff --git a/doc/images/detailed.png b/doc/images/detailed.png
diff --git a/doc/images/loghtml_sidepanel.png b/doc/images/loghtml_sidepanel.png
diff --git a/doc/images/robo_loglink_projectpage.png b/doc/images/robo_loglink_projectpage.png
diff --git a/doc/images/robot_4_table.png b/doc/images/robot_4_table.png
diff --git a/doc/images/robot_view_column.png b/doc/images/robot_view_column.png
diff --git a/doc/images/sidepanel.png b/doc/images/sidepanel.png
diff --git a/src/main/java/hudson/plugins/robot/AggregatedRobotAction.java b/src/main/java/hudson/plugins/robot/AggregatedRobotAction.java
@@ -109,7 +109,7 @@ public static class AggregatedRobotResult extends RobotResult {
 
 		private static final long serialVersionUID = 1L;
 		private final transient AggregatedRobotAction parent;
-		private int passed, failed, criticalPassed, criticalFailed;
+		private int passed, failed, skipped, criticalPassed, criticalFailed;
 
 		public AggregatedRobotResult(AggregatedRobotAction parent) {
 			this.parent = parent;
@@ -120,6 +120,7 @@ public void addResult(RobotResult result) {
 			criticalPassed += result.getCriticalPassed();
 			failed += result.getOverallFailed();
 			passed += result.getOverallPassed();
+			skipped += result.getOverallSkipped();
 		}
 
 		@Override
@@ -147,9 +148,14 @@ public long getOverallFailed() {
 			return failed;
 		}
 
+		@Override
+		public long getOverallSkipped() {
+			return  skipped;
+		}
+
 		@Override
 		public long getOverallTotal() {
-			return failed + passed;
+			return failed + passed + skipped;
 		}
 
 		@Override
@@ -162,6 +168,11 @@ public int getPassed() {
 			return (int) getOverallPassed();
 		}
 
+		@Override
+		public int getSkipped() {
+			return (int) getOverallSkipped();
+		}
+
 		@Override
 		public AbstractBuild getOwner() {
 			return parent.getOwner();