I966 results compare fixes #998
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use natural sort compare for json/tsv fields, so x1 appears before x[10] Add compare of problem information for scoreboard. Add constants to control if certain conditions will cause a comparison difference (addresses CDS issue and PC2 issue) For now, clean up problem id field by converting it back to a string from an elementid. Do not abort entire compare process if a file is missing. note the missing file in the output as a failure, but continue doing the other files. CI: add line number if errors occur during TSV processing (this was an omission in original code) Change order of presentation of failures in the report file to coincide with the number of failures printed in the beginning. Add natural compare routine to StringUtilities.
Order results by compare state, then key, then field. Putting compare state in there groups all errors of the same type together. Added comments where they were deficient. If a record (key) exists in one file but not the other, do not display each field that is missing. It is enough to state the entire key is missing from one file. Clean up generated report (Pluralize, comments, highlight the filename). Add GUI flag to ignore empty awards (no teams for the award). The CDS adds awards that have no teams. Awards with no teams are meaningless, so this flag will allow you to ignore them, cleaning up the results compare report. Remove redundant info from the compare report. Some lines had the same thing (key) repeated 3 times. Updated Junit to use newer data files and test new features.
Neglected to update some copyright dates. These are using for tests.
SamanwaySadhu
approved these changes
Aug 12, 2024
kkarakas
approved these changes
Aug 22, 2024
Compare time fields if non-zero or ignore zero flag is not set.
It's possible that one CCS just puts zeroes in for the time. Previously only checked if shadow had 0 times, now we check for both primary and shadow.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of what the PR does
Issue which the PR addresses
Fixes #966
Environment in which the PR was developed (OS,IDE, Java version, etc.)
Windows 11
java version "1.8.0_321"
Java(TM) SE Runtime Environment (build 1.8.0_321-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.321-b07, mixed mode)
Precise steps for testing the PR (i.e., how to demonstrate that it works correctly)
The JUnit "
FileComparisonUtilitiesTest" does an excellent job of testing various cases.From the GUI:
Bring up a completed contest such as WF46 Finals, or bring up a clean contest (you can use the JUnit test data from WF46)
On an administrator, go to the Run Contest->Results Compare tab
Fill in (or browse to):
...\pc2v9\testdata\FileComparisonUtilitiesTest\resultscompwork\results\domjudgefor the Primary CCS Results DirectoryFill in (or browse to):
...\pc2v9\testdata\FileComparisonUtilitiesTest\resultscompwork\results\pc2for the PC2 Results DirectoryCheck the "Include details in comparison"
The state of the "Ignore awards with no teams" checkbox will not matter for this test
Press the "View Report" button. There should be 0 differences.
Close the View Report window.
Change the Primary CCS Results Directory to:
C:\MyNetProjects\ICPC\pc2v9\testdata\FileComparisonUtilitiesTest\resultscompwork\results\cdsCheck both the "Include details in comparison" and the "Ignore awards with no teams" checkboxes.
Press the "View Report" button.
You will see it reports there is no
results.tsvfile in the primary (cds) folder (correct).You will see it reports 40 differences in the
awards.jsonfile. This is because the CDS uses different citations (note the differences). In addition, the CDS has some additional awards for the resolver that the CCS's are unaware of. These are reported as "No record with that key was found in the PC2 awards file" (for each missing award)Close the View Report window.
Uncheck the "Ignore awards with no teams" checkbox
Press the "View Report" button again.
You will notice a bunch more comparison errors for several awards in the CDS file where there are no teams assigned. This tests the "Ignore awards with no teams" checkbox. We do not need to have empty records on the CCS results, in fact, it's probably wrong that the CDS has them, but Tim doesn't seem to care.