Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Tes Compliance Tests to run on both 1.0.0 and 1.1.0 #459

Open
yuliadub opened this issue Oct 12, 2023 · 6 comments
Open
Labels
Spec Compliance TES Priority: P3 Groomed to a Priority 3 issue

Comments

@yuliadub
Copy link
Contributor

Problem:
Today Tes compliance tests only run tests for TES version 1.0.0

Solution:
Today tes compliance tests only run on version 1.0.0 after a TES integration test is run. After this issue is complete tes compliance tests should run on version 1.1.0. Both results should be recorded appropriately and report a warning on the release if they fail.
This can be run as a separate step in the release as 1.0.0 or in the same step.

Code dependencies
Will this require code changes in:

  • TES - adjusting compliance test script
  • Integration tests?

Additional context
Original task that set up compliance tests: #26

@MattMcL4475 MattMcL4475 added the TES Priority: P3 Groomed to a Priority 3 issue label Dec 11, 2023
@vsmalladi
Copy link
Contributor

@yuliadub
Copy link
Contributor Author

yuliadub commented Apr 3, 2024

ability to run tests on both frameworks is being added in this PR: https://github.com/microsoft/ga4gh-tes/pull/653/files

in this case, the openapi-test-runner will run on one, then another version back to back, and then the results will be copied into a file. The complexity with pulling this data out is due to the fact that today the script runs inside a docker image. All we do today is we check to see if any results have failed, and they do, produce a warning on the ADO specific pipeline run. The next step to this is to display the results as badges inside the ga4gh-tes repo, that would show if the tests are passing or not. The thoughts on how to do this are going to captured in the following issue: 660

@vsmalladi
Copy link
Contributor

@yuliadub can we revist this?

@yuliadub
Copy link
Contributor Author

yuliadub commented Sep 16, 2024

@vsmalladi didnt you add functionality to run on both frameworks already? Looking here.

or are you looking to add the badge to show if they are passing or not?

@vsmalladi
Copy link
Contributor

ah yes @yuliadub i think we can close this then and move to the badge issue right?

@yuliadub
Copy link
Contributor Author

@vsmalladi up to you (or Nikita and Matt?), I dont have a strong opinion -- the badges will be valuable once we actually start passing the compliance tests, not sure if we are yet or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Spec Compliance TES Priority: P3 Groomed to a Priority 3 issue
Projects
None yet
Development

No branches or pull requests

4 participants