Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added spec with x-ignorable:true to check Evaluation operations #762

Merged
merged 3 commits into from
Jan 5, 2025

Conversation

Tokesh
Copy link
Collaborator

@Tokesh Tokesh commented Jan 3, 2025

Description

I'm adding a spec with the parameter x-ignorable:true to see if it will appear in operations and evaluated_operations on the existing tests.
debt from #759

Issues Resolved

[#748]
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Tokesh Tokesh added the skip-changelog No need to update CHANGELOG. label Jan 3, 2025
@Tokesh Tokesh marked this pull request as ready for review January 3, 2025 13:28
Copy link
Contributor

github-actions bot commented Jan 3, 2025

Changes Analysis

Commit SHA: 8238c0c
Comparing To SHA: 3d88921

API Changes

Summary

NO CHANGES

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12610531929/artifacts/2385752607

API Coverage

Before After Δ
Covered (%) 609 (59.65 %) 609 (59.65 %) 0 (0 %)
Uncovered (%) 412 (40.35 %) 412 (40.35 %) 0 (0 %)
Unknown 43 43 0

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I merge this test without your fix in #759 no tests will fail. So add a JavaScroipt/code test specifically for the "it will appear in operations and evaluated_operations on the existing tests" part.

@Tokesh
Copy link
Collaborator Author

Tokesh commented Jan 3, 2025

If I merge this test without your fix in #759 no tests will fail. So add a JavaScroipt/code test specifically for the "it will appear in operations and evaluated_operations on the existing tests" part.

Hi!
I deliberately reverted the code to the previous version. The issue arises because we don’t generate evaluations dynamically but instead define it manually as an array.
image

The tests don’t fail because the code dynamically works only with operations, which already ignores x-ignorable:true. If we remove that parameter, the tests will fail.

As far as I understand, when running unit tests, we’re not starting an OpenSearch instance, correct? If so, implementing this on the current infrastructure would be challenging. It might be necessary to create a special type of test that runs alongside OpenSearch.

@dblock
Copy link
Member

dblock commented Jan 3, 2025

For a unit test you want something a lot lower level and simpler and intentional. The method you changed was TestResults.evaluatied_operations(). So modify/add to https://github.com/opensearch-project/opensearch-api-specification/blob/main/tools/tests/tester/TestResults.test.ts a test that looks something like

descrive('evaluatied_operations', () => {
  test('does not include x-ignorable APIs', () => {
    expect(test_results.evaluated_operations()).to ...
  })
})

@Tokesh Tokesh requested a review from dblock January 4, 2025 13:05
@dblock dblock merged commit 97ab829 into opensearch-project:main Jan 5, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-changelog No need to update CHANGELOG.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants