Merge branch 'main' into meinte/correctness

Author: dslmeinte

metametamodel/metametamodel.adoc

@@ -98,12 +98,12 @@ Language elements contained in a Language are allowed to refer to these other La
 * within declared dependencies
 * within _transitive_ dependencies.
 +
-Example: if Language B depends on Language A, and Language C depends on B, then Language C can also refer to members of Language A without explicitly declaring a dependency.
+Example: if Language _B_ depends on Language _A_, and Language _C_ depends on _B_, then Language _C_ can also refer to members of Language _A_ without explicitly declaring a dependency.
 
 A Language CAN declare a transitive dependency explicitly.
-In the example above, Language C CAN declare an explicit dependency on Language A.
+In the example above, Language _C_ CAN declare an explicit dependency on Language _A_.
 
-##TODO: Dependency on builtins {fn-org153}##
+As any Language depends (at least transitively and <<Concept.extends, implicitly>>) on <<stdlib>> elements, a Language CAN declare a dependency on builtins -- but _does not_ need to.{fn-org153}
 
 [[Concept]]
 ==== Concept
@@ -139,12 +139,22 @@ A Concept `implements` zero or more <<ConceptInterface, ConceptInterfaces>>.
 A Concept can have any number of <<Classifier.features, `features`>>, given it is a <<Classifier>>.
 
 .Constraints
-TBD
+A Concept MUST NOT extend itself, or form circles via `extends`.
 
 [[Annotation]]
 ==== Annotation
-An Annotation is a limited[*] additional, orthogonal information attached to potentially any node, sharing the node’s lifecycle.
+An Annotation is an additional piece of information attached to potentially any node, sharing the node’s lifecycle.
 The annotated node (or its concept) does not need to know about the annotation.{fn-org13}
+Annotations CAN be attached to other Annotations (although this structure is hard to comprehend, and should be used with caution).
+
+Annotations should only have limited content because the more complex the annotation is, the more likely it should not be part of the annotated node's lifecycle -- We might want to reuse the complex annotation somewhere else. +
+Example: In MPS, the editor of a concept can be seen as an annotation of that concept.
+But even if we deleted the concept, we might want to reuse the editor for another (similar) concept.
+
+Annotations' contents should be orthogonal to the annotated node, because actual content should be part of the original concepts, and unrelated contents should be somewhere else. +
+Example: Let's assume our Language defines a `Date` PrimitiveType.
+We might annotate `Date` with `@JavaImplementation(java.util.Date)` and `@TypeScriptImplementation(JsJoda.LocalDate)`.
+Our core model stays platform-independent, but we maintain the implementation details at the right place -- if we ever deleted `Date`, all related information would be gone without further cleanup.
 
 .Example
 We design a fancy documentation language, and can annotate any other node with such documentation.
@@ -166,13 +176,13 @@ An Annotation is a <<Classifier>>, and indirectly a <<LanguageEntity>> (as it is
 Each Annotation specifies which <<Classifier, `Classifiers`>> it `annotates`.
 If it should be applicable to any node, it annotates <<Node>>.
 
-As Annotations can specify which Concept or Annotation it [[Annotation.extends, Annotation.extends]]`extends` and which interfaces it [[Annotation.implements, Annotation.implements]] `implements`.
-An Annotation can have any number of <<Classifier.features, `features`>>, given it is a <<Classifier>>.
+Annotations can specify which Annotation it [[Annotation.extends, Annotation.extends]]`extends` and which interfaces it [[Annotation.implements, Annotation.implements]] `implements`.
+An Annotation CAN have any number of <<Classifier.features, `features`>>, given it is a <<Classifier>>.
 
 .Constraints
 We CANNOT redefine `multiple` and `annotates` in a sub-annotation (i.e. an annotation that `extends` another).{fn-org154}
 
-We can attach an instance of an Annotation only to instances of the Concept the Annotation `annotates` (and sub-concepts thereof).
+We MUST attach an instance of an Annotation only to instances of the Concept the Annotation `annotates` (and sub-concepts thereof).
 
 .Annotation examples
 --
@@ -293,10 +303,6 @@ CompanyAddress <-[#red]- CompanyRole
 ----
 --
 
-.Remarks
-[*] "limited" because the more complex the annotation is, the more likely it should not be part of the annotated node's lifecycle -- We might want to reuse the complex annotation somewhere else.
-Example: In MPS, the editor of a concept can be seen as an annotation of that concept. But even if we deleted the concept, we might want to reuse the editor for another (similar) concept.
-
 [[ConceptInterface]]
 ==== ConceptInterface
 A ConceptInterface represents a category of entities sharing some similar characteristics.

serialization/serialization.adoc

@@ -1,9 +1,14 @@
 include::../shared/issue-footnotes.adoc[]
 
 :m3: ../metametamodel/metametamodel
+:refarch: ../reference-architecture/reference-architecture
 :fn-mof: footnote:mof[https://en.wikipedia.org/wiki/Meta-Object_Facility[Meta-Object Facility], also known as M3 model]
 
 = LIonWeb Serialization Format
+:toc: preamble
+:toclevels: 3
+
+This document describes the serialization format for LIonWeb chunks.
 
 == Conventions used in this document
 * _italic_ words refer to concepts defined by JSON.
@@ -17,6 +22,9 @@ We do not take any measures to reduce the amount of transmitted data.{fn-org73}
 == Description
 LIonWeb node serialization format is defined in JSON (https://datatracker.ietf.org/doc/html/rfc8259[RFC 8259]).
 
+We follow the advice of RFC 8259 to be "interoperable", i.e. we assume object keys are unique, and their order is undefined.
+Any violations SHOULD be reported as error.{fn-org159}
+
 == Overview of structures
 
 plantuml::serialization.puml[format=svg]
@@ -29,18 +37,18 @@ Root level MUST be an _object_ with three members, called *serialization chunk*.
 ##TODO: Are more members allowed?{fn-org67}##
 
 [[SerializationChunk.serializationFormatVersion]]
-The first member MUST be _key_ `serializationFormatVersion` with a _string_ _value_.{fn-org58}
+The first member SHOULD be _key_ `serializationFormatVersion` with a _string_ _value_.{fn-org58}{fn-org159}
 The value MUST be a decimal integer (without leading or trailing whitespace) describing the serialization format version used to create the processed document, according to <<versions>>.
 
 [[SerializationChunk.languages]]
-The second member MUST be _key_ `languages` with an _array_ _value_.{fn-org76}{fn-org78}
+The second member SHOULD be _key_ `languages` with an _array_ _value_.{fn-org76}{fn-org78}{fn-org159}
 Each _element_ in the value array MUST adhere to <<language>>.
 The order of _elements_ is undefined.
 _elements_ MUST contain all language/version referred to by any <<meta-pointer>> in the processed document.
 Each _element_ must be unique with respect to all its _members_.
 
 [[SerializationChunk.nodes]]
-The third member MUST be _key_ `nodes` with an _array_ _value_.{fn-org33}
+The third member SHOULD be _key_ `nodes` with an _array_ _value_.{fn-org33}{fn-org159}
 Each _element_ in the value array MUST adhere to <<node>>.
 The order of _elements_ is undefined.
 Each _element_ must be unique with respect to the value of its _key_ `id`.
@@ -49,9 +57,11 @@ Each _element_ must be unique with respect to the value of its _key_ `id`.
 === Language structure
 [[UsedLanguage]]
 Each *used language* MUST be an _object_.{fn-org129}
-##TODO: include builtins in used language? {fn-org153}##
 The order of _members_ is undefined.
 
+NOTE: If the chunk describes a language (M2), it might include instances of builtins' language entities.
+In this case, builtins MUST be listed as *used language* like any other language.{fn-org153}
+
 The _object_ MUST contain the following _members_:{fn-org76}
 
 * [[UsedLanguage.key]] _key_ `key` with _string_ _value_, adhering to <<language-key>>.
@@ -96,13 +106,13 @@ The _object_ MUST contain the following _members_:{fn-org59}{fn-java33}{fn-org55
 * [[Node.concept]] _key_ `concept`{fn-org37-name} with _object_ _value_, adhering to <<meta-pointer>>.
   The *meta-pointer*'s ``key``'s _value_ refers to the <<{m3}.adoc#IKeyed.key, *key*>> of the <<{m3}.adoc#Concept, *Concept*>> this *node* is an instance of.
 * [[Node.properties]] _key_ `properties` with _array_ _value_, each _element_ adhering to <<property>>.
-The order of _elements_ is undefined.
+The order of _elements_ is undefined.{fn-org156}
 * [[Node.children]] _key_ `children`{fn-org55-name-children} with _array_ _value_, each _element_ adhering to <<child>>.
-The order of _elements_ is undefined.
+The order of _elements_ is undefined.{fn-org156}
 * [[Node.references]] _key_ `references`{fn-org55-name-references} with _array_ _value_, each _element_ adhering to <<reference>>.
-The order of _elements_ is undefined.
+The order of _elements_ is undefined.{fn-org156}
 * [[Node.annotations]] _key_ `annotations`{fn-org150} with _array_ _value_, each _element_ adhering to <<annotation>>.
-The order of _elements_ is undefined.
+The order of _elements_ MUST be maintained as set by <<{refarch}.adoc#client, model client>>.{fn-org157}
 * [[Node.parent]] _key_ `parent` with _string_ or _null_ _value_, adhering to <<parent>>.
 
 ##TODO: How to store invalid text?{fn-org62}##
@@ -143,7 +153,7 @@ The _object_ MUST contain the following _members_:
 The *meta-pointer*'s ``key``'s _value_ refers to the <<{m3}.adoc#IKeyed.key, *key*>> of the <<{m3}.adoc#Containment, *Containment*>> this *child* is an instance of.
 * [[Child.children]] _key_ `children` with _array_ _value_ with _string_ _elements_.
 Each _element_ adheres to <<{m3}.adoc#identifiers, Identifier spec>>, and refers to the *id* of the contained *node*.
-The order of _elements_ is undefined.
+The order of _elements_ MUST be maintained as set by <<{refarch}.adoc#client, model client>>.{fn-org157}
 +
 NOTE: Each *child* element is the inverse relation of a *parent*.
 +
@@ -159,7 +169,8 @@ The _object_ MUST contain the following _members_:
 
 * [[Reference.reference]] _key_ `reference` with _object_ _value_, adhering to <<meta-pointer>>.
 The *meta-pointer*'s ``key``'s _value_ refers to the <<{m3}.adoc#IKeyed.key, *key*>> of the <<{m3}.adoc#Reference, *Reference*>> this *reference* is an instance of.
-* [[Reference.targets]] _key_ `targets` with __object_ _elements_.
+* [[Reference.targets]] _key_ `targets` with _array_ _value_ with __object_ _elements_.
+The order of _elements_ MUST be maintained as set by <<{refarch}.adoc#client, model client>>.{fn-org157}
 Each _element_ MUST have the following _members_ in undefined order:{fn-org55-name-references}
 ** [[Reference.reference.resolveInfo]] _key_ `resolveInfo`{fn-org36} with _value_ as one of:
 *** _string_ containing *resolveInfo*, a textual hint that might be used to find the target *node* of this reference.
@@ -178,17 +189,11 @@ NOTE: The referred *node* CAN be contained in the processed document, but also C
 [[annotation]]
 ==== Annotation
 [[Annotation]]
-Each *annotation* MUST be an _object_.
-The order of _members_ is undefined.
-
-The _object_ MUST contain the following _members_:
+Each *annotation* MUST be a _string_.
+It adheres to <<{m3}.adoc#identifiers, Identifier spec>>, and refers to the *id* of the contained annotation *node*.
 
-* [[Annotation.annotations]] _key_ `annotations` with _array_ _value_ with _string_ _elements_.
-Each _element_ adheres to <<{m3}.adoc#identifiers, Identifier spec>>, and refers to the *id* of the contained *node*.
-The order of _elements_ is undefined.
-+
 NOTE: Each *annotation* element is the inverse relation of a *parent*.
-+
+
 NOTE: The annotation *node* CAN be contained in the processed document, but also CAN be outside the processed document (i.e. not contained in the processed document).
 
 [[parent]]
@@ -211,6 +216,7 @@ NOTE: The referred *node* CAN be contained in the processed document, but also C
 All property values MUST be serialized as JSON _string_.{fn-org34}{fn-org9}.
 An unset property CAN be serialized as JSON _null_.
 
+[[string]]
 ==== String
 <<{m3}.adoc#String, LIonCore Strings>> might be any string, of any length, including (but not limited to):
 
@@ -220,6 +226,7 @@ An unset property CAN be serialized as JSON _null_.
 * containing extended Unicode characters: `"😐"`
 * containing escaped Unicode characters: `"\uD83D\uDE10"`
 
+[[boolean]]
 ==== Boolean
 <<{m3}.adoc#Boolean, LIonCore Booleans>> MUST be encoded as exactly one of these JSON _strings_:
 
@@ -228,6 +235,7 @@ An unset property CAN be serialized as JSON _null_.
 
 Booleans MUST NOT be encoded with leading or trailing whitespace, uppercase characters, short forms (like `t` or `f`), or decimal representation (like `1`, `0`, `-1`).
 
+[[integer]]
 ==== Integer
 <<{m3}.adoc#Integer, LIonCore Integers>> MUST be encoded as JSON _string_.
 
@@ -262,6 +270,7 @@ Booleans MUST NOT be encoded with leading or trailing whitespace, uppercase char
 * `" 5"`
 * `"-6 "`
 
+[[json]]
 ==== JSON
 <<{m3}.adoc#JSON, LIonCore JSON>> MUST be encoded as JSON _string_.
 All double quotes, line breaks, etc. MUST be escaped to form a proper JSON _string_.
@@ -273,6 +282,7 @@ The value MUST adhere to JSON spec (RFC 8259).
 .Invalid example
 `{ "key": "my value", "myArray": [1, -2, true] }`
 
+[[literal]]
 ==== Enumeration literals
 <<{m3}.adoc#EnumerationLiteral, LIonCore Enumeration literals>> MUST be encoded as JSON _string_ _value_ according to <<{m3}.adoc#keys, Key spec>>.
 MUST refer to the <<{m3}.adoc#IKeyed.key, key>> of an <<{m3}.adoc#EnumerationLiteral, EnumerationLiteral>> of the <<{m3}.adoc#Enumeration, Enumeration>> defined as <<{m3}.adoc#Property.type, type>> of this *Property*.{fn-org128}

serialization/serialization.puml

@@ -19,6 +19,7 @@ SerializationChunk *--> "0..*" Node: nodes
 class Node {
     id: Id
     parent: Id
+    annotations: Id[] <<ordered>>
 }
 
 Node *--> "1" MetaPointer: concept
@@ -31,7 +32,7 @@ class MetaPointer {
 
 Node *--> "0..*" Child: children
 class Child {
-  children: String[]
+  children: Id[] <<ordered>>
 }
 Child *--> "1" MetaPointer: containment
 
@@ -45,15 +46,10 @@ Property *--> "1" MetaPointer: property
 Node *--> "0..*" Reference: references
 class Reference
 Reference *--> "1" MetaPointer: reference
-Reference *--> "0..*" ReferenceTarget: targets
+Reference *--> "0..*" ReferenceTarget: <<ordered>>\ntargets
 class ReferenceTarget {
     reference: Id
     resolveInfo: String
 }
 
-Node *--> "0..*" Annotation: annotations
-class Annotation {
-  annotations: String[]
-}
-
 @enduml

serialization/serialization.schema.json

@@ -177,7 +177,8 @@
         "additionalProperties": false,
         "minProperties": 7,
         "maxProperties": 7
-      }
+      },
+      "uniqueItems": true
     }
   },
   "required": [

shared/issue-footnotes.adoc

@@ -145,3 +145,6 @@
 :fn-org150: footnote:org150[https://github.com/LIonWeb-org/organization/issues/150[How to represent annotations in serialization #150]]
 :fn-org153: footnote:org153[https://github.com/LIonWeb-org/organization/issues/153[Details on builtin language #153]]
 :fn-org154: footnote:org154[https://github.com/LIonWeb-org/organization/issues/154[Details on Annotations #154]]
+:fn-org156: footnote:org156[https://github.com/LIonWeb-org/organization/issues/156[Keep serialization order between different features? #156]]
+:fn-org157: footnote:org157[https://github.com/LIonWeb-org/organization/issues/157[Maintain order in serialization of each containment / reference / annotation #157]]
+:fn-org159: footnote:org159[https://github.com/LIonWeb-org/organization/issues/159[Don't rely on valid, but ambiguous JSON structures #159]]