Ideas on derivation #235

enikao · 2024-02-04T16:26:31Z

based on #167

Very unfinished, just scribbling down my thoughts.
This draft PR is meant as a place to collect comments.

…o discussions

… call

enikao · 2024-03-12T18:39:27Z

derived/derivation.adoc

@@ -90,6 +90,11 @@ IDerived "0..*" -> "1..*"  Node: base
 Each derived node references one or more base nodes.
 Any (base) node can have none or more derived nodes.

+[horizontal,labelwidth=12]
+QUESTION:: Should a derived node also be able to refer to features inside a node? Or do we leave that for specific classes implementing `IDerived`? 


In the example below, the PropertyViolation points to a specific feature.

Assume the following concepts:

concept Person [id=P, key=pers] property firstName: string [id=PF, key=firstN] property lastName: string [id=PL, key=lastN] concept EmptyNameViolation implements IDerived, IViolation [id=ENV, key=emptyNameV] property message: string [id=ENVM, key=msg] reference property: Property [id=ENVP, key=prop]

Original model contains instance:

Person A [id=2] firstName = "John" lastName = ""

Validation derived model contains instance:

EmptyNameViolation B [id=5] message = "lastName is empty!" base = --> A[id=2] property = --> lastName[id=PL]

enikao · 2024-03-12T18:40:29Z

derived/derivation.adoc

@@ -90,6 +90,11 @@ IDerived "0..*" -> "1..*"  Node: base
 Each derived node references one or more base nodes.
 Any (base) node can have none or more derived nodes.

+[horizontal,labelwidth=12]
+QUESTION:: Should a derived node also be able to refer to features inside a node? Or do we leave that for specific classes implementing `IDerived`? 
+QUESTION:: The assumption is that derived nodes can have children that are not derived nodes.


Derived nodes can have any kind of features, including children that might or might not implement IDerived.

enikao · 2024-03-12T18:43:19Z

derived/derivation.adoc

@@ -134,6 +139,9 @@ We need to omit the parameter if we don't want to limit the depth.
 ==== Result
 {chunk} containing all derived nodes according to `nodeIds`, `languages`, and `depthLimit` parameters.

+[horizontal,labelwidth=12]
+QUESTION:: The `depth:Limit` describes the depth of the retrieved nodes (derived nodes may be full trees), the next line suggests that `depthLimit` applies to the node id's sent. Needs clarification.


The idea is:

We select the same original nodes as with Bulk API retrieve

For all these nodes, we find their derivations (if available)

For all derivations, we return the derivations and all descendants thereof with infinite depth.
The assumption is that derived nodes only contain other nodes that are part of the derived information.

enikao · 2024-03-12T18:45:19Z

derived/derivation.adoc

@@ -143,6 +151,8 @@ Does NOT include the definition of <<{serialization}.adoc#UsedLanguage, UsedLang


 == Possible derivation backends
+[horizontal,labelwidth=12]
+QUESTION:: The repository that is referred to is this always the repository where the opriginal model is stored, or could oit be a different (i.e. derivation processor specific)  repository?


Formally the derivation backend implementation is independent of the repository implementation, i.e. they don't need to be the same "service". (In practice in makes a lot of sense to couple them, though.)

Derived models are part of the same repository as the original model in the sense that they share one id-space, and can refer to each other.

enikao · 2024-03-12T18:45:30Z

derived/derivation.adoc

@@ -270,6 +280,9 @@ package M3 {
 }
 ----

+[horizontal,labelwidth=12]
+QUESTION:: Should the `PropertyValidation`  also point to the Property instance in the M2 model being validated, and not just to the node that contains the property?


See remark above.

ftomassetti · 2024-03-15T09:51:44Z

I am not sure if there is a better place to write these comments, but reading the current document on derived models I had these questions/comments:

Can other sources be involved in creating a derived node (e.g., external data)? So should we say that a derived node is produced by examining existing nodes (both "original nodes" and other derived nodes) plus, potentially, external sources?
Can we have a derived partition or just other nodes?
What about changing the term base node in source node?
Should derived nodes have some sort of marker to figure out which derivation produced them? For example, an annotation
Should derivations have a name or some identifier?
Why do we need the IDerived interface? Can't I derive any node I want? For example, I may want to use derive nodes to de-sugar code, without having to mark all of my expressions or statements as implementing IDerived

enikao · 2024-03-15T10:01:52Z

Can other sources be involved in creating a derived node (e.g., external data)? So should we say that a derived node is produced by examining existing nodes (both "original nodes" and other derived nodes) plus, potentially, external sources?

Yes, as long as the processor creating the derived nodes takes care of interacting with the external source.

Can we have a derived partition or just other nodes?

I'm not sure I understand. We can have a derivation of any node, not only partitions.

What about changing the term base node in source node?

I don't really mind; I think we used this term base already at other places for this idea.

Should derived nodes have some sort of marker to figure out which derivation produced them? For example, an annotation

Derivations are defined by their ava, i.e. a language. That might be enough of a marker -- or not.

Should derivations have a name or some identifier?

Derivations in sense of derived nodes are marked by implementing IDerived.
Derivations themselves are defined by their ava, i.e. a language.
Do we need more identification?

Why do we need the IDerived interface? Can't I derive any node I want? For example, I may want to use derive nodes to de-sugar code, without having to mark all of my expressions or statements as implementing IDerived

IDerived connects the derived node to its base. Without that, how do we know from what we derived it?
We can have an EvaluationResult implements IDerived that refers to the evaluated expression.
The EvaluationResult can contain arbitrary complex sub-nodes to e.g. describe the execution time of each sub-expression. These sub-nodes don't need to implement IDerived (even though it might make sense, because we can connect them uniquely to a base node).

dslmeinte · 2024-04-02T15:37:58Z

derived/processors.pptx

@joswarmer Can you also commit a PDF version of this?

There is a PNG version next to the PowerPoint

Sorry, figured the PowerPoint had more than the one figure :)

derived/processors.adoc

dslmeinte · 2024-04-05T09:08:17Z

derived/processors.adoc

+A base model can be an _original_ or a _derived_ model.
+
+processor::
+A _processor_ is a component that provides/calculates derived nodes for nodes in a base model.


{C|Sh}ouldn't a processor be associated to a set of languages that a base model is supposed to conform to? An “instance” of a processor would then be associated to a particular base model.

I think a processor should have a set of partitions for which it derived nodes. Which of the nodes in the partitions will havederived nodes is up to the processor.

Niko Stotz and others added 28 commits November 23, 2023 14:32

first draft of bulk API

eb0da42

first draft of bulk api completed

965f978

update after rebase

91b06fb

moved language api out of simple bulk api

c5a9900

removed "proposed api name" sections (redundant and confusing)

38fea09

replaced parameter retrieve.mode with retrieve.depthLimit according t…

d3f21a4

…o discussions

added working hypothesis according to discussions

7e5ce2b

formatting and clarification according to discussions

cc0db5b

Updated according to discussions

5b73777

wip

dc48760

added some thoughts about id mapping

8ce7873

added some thoughts about stable node URIs

5d51f84

rewrote bulk API store call from client's point of view; added delete…

5654518

… call

better ToC overview

b55d95c

Added some comment (labeled Jos)

27ad5f1

Fix typo

de06d4b

Comment

f09878f

Added comments

cd8f54e

result of discussions on 2023-11-24 (unedited)

ec3f3f5

moved trash into separate API

de67cdf

explicitly described which constraints to enforce and ignore

e4625db

moved Jos' remarks from file content to comments

5ec03b7

updated bulk / language / trash APIs to latest discussions

e38cc42

first sketch on derived models

0c7902c

updated with latest decisions

1b02ab2

added error cases

f391c34

added more error cases

10d53c7

Added some question to the derivation document.

db531f3

enikao commented Mar 12, 2024

View reviewed changes

joswarmer added 2 commits April 2, 2024 16:41

Definitions and questions about Processors and Derived Models

688e692

Typos

0db9eb8

dslmeinte reviewed Apr 2, 2024

View reviewed changes

dslmeinte reviewed Apr 5, 2024

View reviewed changes

derived/processors.adoc Show resolved Hide resolved

dslmeinte reviewed Apr 5, 2024

View reviewed changes

Niko Stotz added 2 commits May 7, 2024 13:45

completeness scenario as sequence diagram

433a2a5

added inboxes

3d5a879

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ideas on derivation #235

Ideas on derivation #235

enikao commented Feb 4, 2024

enikao Mar 12, 2024

enikao Mar 12, 2024

enikao Mar 12, 2024

enikao Mar 12, 2024

enikao Mar 12, 2024

ftomassetti commented Mar 15, 2024

enikao commented Mar 15, 2024

dslmeinte Apr 2, 2024

joswarmer Apr 2, 2024

dslmeinte Apr 2, 2024

dslmeinte Apr 5, 2024

joswarmer Apr 16, 2024

Ideas on derivation #235

Are you sure you want to change the base?

Ideas on derivation #235

Conversation

enikao commented Feb 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ftomassetti commented Mar 15, 2024

enikao commented Mar 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment