[Breaking Change] Cross-Version Runtime Exception: Data does not match any schemas from 'oneOf' (AutoRest v2 -> v3)

[mikekistler]

The Breaking Changes check in PR 26672 failed with a runtime exception with "ERROR: Schema violation: Data does not match any schemas from 'oneOf'.

The exception points to an "x-ms-example":

          "NameSpaceCreate": {
            "$ref": "./examples/NameSpaces/EHNamespaceFailover.json"
          }

for an operation with a 202 response that includes an "Azure-AsyncOperation" and a "location" response header. And the example included both of these headers:

  "responses": {
    "202": {
      "headers": {
        "azure-asyncoperation": "http://azure.async.operation/status",
        "location": "eastus"
      }
    }
  }

Through trial and error, I determined that the "location" header was causing the exception, and with a little more experimenting I found that the value had to be a URL. That makes sense, except that the OpenAPI only defined "location" as a "string". Strange!

Anyway, fixing the value of "location" in the example to be a valid URL fixed the runtime exception.

[konrad-jamrozik]

Related investigation:

• [Breaking Change] ERROR: Schema violation: Data does not match any schemas from 'oneOf' #6036

[konrad-jamrozik]

More discussion of oneOf in this Teams thread

Key observation from Timothee Guerin about oneOf being unhelpful error message:

Mentioned earlier but with autorest v3 I migrated to a newer json schema which allows if then which gives a much better error
Like similar to what editor.swagger.io does

[mikekistler]

Another occurrence: https://github.com/Azure/azure-rest-api-specs/pull/27515/checks?check_run_id=20811157978

[konrad-jamrozik]

Re the breaking change labelling issue in:

• Update Custom Voice API to a new version (2024-02-01-preview). azure-rest-api-specs#27515

Mike wrote:

Approving breaking changes because #7278. Manual inspection of the diff shows no breaking changes.

Recall the failure was AutoRest error:

ERROR: Schema violation: No enum match for: operation-location

Per my comment here:

Key observation from Timothee Guerin about oneOf being unhelpful error message:

Mentioned earlier but with autorest v3 I migrated to a newer json schema which allows if then which >> gives a much better error
Like similar to what editor.swagger.io does

Because AutoRest invoked by openapi-diff failed, the downstream automation (in openapi-alps repo) that classifies breaking change failures didn't run. That downstream automation can quell the check if all the failures are benign. Basically, this kind of observed failure is imprecise and currently requires manual effort. We could work on updating to AutoRest v3, which hopefully would make the error better and would allow easier manual investigation and maybe even better automated determination what is the issue. But then we need to prioritize for it, and it is unclear how hard it would be to do the migration. It may impact what and when fails validation. We use AutoRest also in LintDiff, ModelValidation and SemanticValidation. If we don't update it there, this might lead to inconsistent behavior across checks.

Doc on AutoRest migration from v2 to v3:

• https://github.com/Azure/autorest/blob/main/docs/migrate/readme.md

[konrad-jamrozik]

Additional context on Teams here

about this PR:

• https://github.com/Azure/azure-rest-api-specs/pull/25168/checks?check_run_id=22192854520