webhooks: JSON schemas for webhook event payloads

[hawkw]

As I discussed in #7277, we should have some mechanism for communicating the JSON schema of the payload sent for each webhook event class. There are a few obvious ways to do this:

1. OpenAPI v3.1.x has a way to specify webhook payloads as well as push-based APIs. However, we presently only implement Openapi v3.0 in the dropshot/progenitor cinematic universe (see e.g. Support OpenAPI 3.1.0 progenitor#762). Per @ahl, implementing v3.1 will require a substantial engineering effort, so we may not be able to use this in the near-term.
2. We also have an endpoint for listing event classes on a running system. We could add a JSON schema either to the entries in this list and/or to a new GET /v1/webhooks/event-classes/{class} endpoint.
3. Some kind of bespoke thing where we generate some JSON schemas at build-time, cat them together, and get that into the docs site somehow.

I think we should probably pursue at least two of these approaches. Using the OpenAPI v3.1 thing feels like the ideal thing, but using that depends on other work that may take some time, so we should probably do at least one of the others as an interim solution.

Potential Trickiness

I anticipate a couple issues with this that are probably worth thinking about now, both of which relate to the potential for webhook payload schemas to be variable.

The primary concern is versioning: as we release new versions of the control plane, we may be able to add new data to existing webhook payloads (or otherwise change them). Because webhook delivery is an RPW and event payloads are stored in CRDB when the event is created, it's possible for the control plane to be updated to a new version between when an event is published internally and when it is successfully delivered to a receiver. In this case, if the new release added fields to that event schema, it's possible that we may not be able to send the new schema when delivering that event, if data required to produce those fields was not recorded in the previous version of the event's schema. So, we would have to send the previous version. We'll need to make sure that whatever schemas we publish for event classes make it clear to the consumer that previous versions may be sent, and to include versioning information in the payload so that the receiver can know how to interpret it. Probably the move is to represent every event payload as an enum with the version as a tag, at the top level, or something.

In RFD 538's "future work" appendix, we propose a system for filtering the data included in webhook payloads based on the receiver's permissions. At present, this is not implemented, as all webhook receivers operate with the "fleet.viewer" role. However, when we implement webhook receivers with more restrictive permissions, we will want to make it clear that some data in the payload schema may or may not be included based on the receiver's permissions. For instance, if a receiver is only a project viewer, the information about the silo in which an event occurred will be excised. This is probably not too hard; we just have to make those fields nullable in the schema, and document why they may or may not be included.

[ahl]

I would suggest a pretty simple approach: an expectorate test that poops out a json file. We'll then move that json file downstream into docs, oxide.rs, oxide.go, oxide.ts. From there, we'll generate types and include those for use in the SDKs for example.

[hawkw]

starting internals for this: #8107

i'll probably go and make it dump out a schema.json somewhere that we can use for now.