stabilize order of properties in grammars

[mmoskal]

I got started at https://github.com/microsoft/llguidance/blob/main/parser/src/json/README.md

@hudson-ai please check and beef it up!

[hudson-ai]

You've honestly covered pretty much everything. Will think about what else may be useful to add...

[mmoskal]

I think we should specify the intersection order, this is what you seem to be doing:

Intersection of two sets of properties P1 and P2 is P1 + (P2 \ P1) (the order of P1 is maintained and remaining properties from P2 are added at the end in order of P2).

The sibling keys are always processed first (as P1). Then elements of AllOf are processed in order.

On reflection, this seems the only reasonable option (one could argue siblings could go at the end, but I think front makes more sense).

[hudson-ai]

This aligns with my mental model, yes!

A few additions:

• When required lists keys not in properties, we first handle properties in order and then the remaining keys of required in order
• We have a set precedence for allOf, oneOf, anyOf, and $ref, all handled after sibling keys (or parent keys, depending on how you look at it). Alternatively, it would be reasonable to process these in order of appearance.
• If we decide to handle allOf, etc. in order of appearance, it would be reasonable to handle precedence of these applicators vs sibling keys also in a "in order of appearance" manner.

I think the set precedence vs "in order of appearance" decision is the one big thing I would like to make an intentional decision on before calling this "stabilized".

[mmoskal]

I like "in order of appearance", even if that makes the sibling keys impl a bit more difficult.

[hudson-ai]

I think I agree. Stabilizing this feels relatively high priority, so I'll start drafting something