Skip to content

Commit

Permalink
Merge branch 'finos:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
rafaelbey authored Nov 29, 2024
2 parents 07d9f6f + 39e5c0c commit b40d220
Show file tree
Hide file tree
Showing 1,195 changed files with 20,756 additions and 13,050 deletions.
2 changes: 1 addition & 1 deletion .gs-project.yml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
productGuid: "product::315850"
productGuid: "product::776807"
76 changes: 76 additions & 0 deletions docs/compiler/compiler-extension-processor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Defining Compiler Extension Processor

``` JAVA
public static <T extends PackageableElement> Processor<T> newProcessor(
Class<T> elementClass,
Collection<? extends Class<? extends PackageableElement>> prerequisiteClasses,
BiFunction<? super T, CompileContext, org.finos.legend.pure.m3.coreinstance.meta.pure.metamodel.PackageableElement> firstPass,
BiConsumer<? super T, CompileContext> secondPass,
BiConsumer<? super T, CompileContext> thirdPass,
BiFunction<? super T, CompileContext, RichIterable<? extends org.finos.legend.pure.m3.coreinstance.meta.pure.metamodel.PackageableElement>> prerequisiteElementsPass)
```

## Element Class
Provide subclass of `PackageableElement` which is being compiled (e.g. `Database.class`).

## Prerequisite Classes
Provide a set of prerequisite classes which your element class depends on (e.g. `[Mapping.class, PackageableConnection.class]`).
The compiler sorts the dependencies based on these prerequisite classes and guarantees all elements in those classes are compiled before the elements in your class.

## First Pass
Define a function that performs the following operations:
* Create `Pure (M3)` objects and register them in Pure graph
* Set primitive values in the `Pure (M3)` objects
* **MUST NOT** reference other elements in the Pure graph

## Second Pass
Define a function that performs the following operations:
* Resolve content of its own element and references to other elements
* **MUST NOT** introspect content of other elements or check validity/correctness

## Third Pass
Define a function that performs the following operations:
* Resolve cross-dependencies
* Introspect other elements

## Prerequisite Elements Pass
Define a function that returns the prerequisite elements under the same element class that your element depends on.
The compiler sorts the dependencies based on these prerequisite elements and guarantees that those elements are compiled before your element.
For instance, `MappingA` depends on `MappingB` and `MappingC`, and therefore, `MappingB` and `MappingC` must be compiled first.

The compiler throws an `EngineException` if it finds circular dependencies in these elements. The following Pure grammar demonstrates an example of circular dependencies:

``` pure
###Relational
Database store::CovidDataStoreA
(
include store::CovidDataStoreB
)
Database store::CovidDataStoreB
(
include store::CovidDataStoreC
)
Database store::CovidDataStoreC
(
include store::CovidDataStoreA
Table DEMOGRAPHICS
(
FIPS VARCHAR(200),
STATE VARCHAR(200)
)
Table COVID_DATA
(
ID INTEGER PRIMARY KEY,
FIPS VARCHAR(200),
DATE DATE,
CASE_TYPE VARCHAR(200),
CASES INTEGER,
LAST_REPORTED_FLAG BIT
)
Join CovidDataDemographicsJoin(DEMOGRAPHICS.FIPS = COVID_DATA.FIPS)
)
```
2 changes: 1 addition & 1 deletion legend-engine-application-query/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<groupId>org.finos.legend.engine</groupId>
<artifactId>legend-engine</artifactId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>legend-engine-application-query</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<groupId>org.finos.legend.engine</groupId>
<artifactId>legend-engine-configuration</artifactId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>legend-engine-configuration-contract-extension-pure</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<groupId>org.finos.legend.engine</groupId>
<artifactId>legend-engine-configuration</artifactId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>legend-engine-configuration-plan-generation-serialization</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion legend-engine-config/legend-engine-configuration/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<groupId>org.finos.legend.engine</groupId>
<artifactId>legend-engine-config</artifactId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<artifactId>legend-engine-config</artifactId>
<groupId>org.finos.legend.engine</groupId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<artifactId>legend-engine-config</artifactId>
<groupId>org.finos.legend.engine</groupId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -418,6 +418,17 @@ protected Iterable<? extends Class<? extends CompilerExtension>> getExpectedComp
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.DataSpaceCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.TextCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.CoreCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.ProfileCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.EnumerationCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.ClassCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.MeasureCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.AssociationCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.FunctionCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.MappingCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.PackageableRuntimeCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.PackageableConnectionCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.SectionIndexCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.DataElementCompilerExtension.class)
.with(org.finos.legend.engine.language.pure.dsl.generation.compiler.toPureGraph.GenerationCompilerExtensionImpl.class)
.with(org.finos.legend.engine.language.pure.dsl.service.compiler.toPureGraph.ServiceCompilerExtensionImpl.class)
.with(org.finos.legend.engine.language.pure.compiler.toPureGraph.ExternalFormatCompilerExtension.class)
Expand Down Expand Up @@ -587,6 +598,7 @@ protected Iterable<String> getExpectedCodeRepositories()
.with("core_external_store_relational_sql_planning")
.with("core_external_store_relational_sql_dialect_translation")
.with("core_external_store_relational_sql_dialect_translation_duckdb")
.with("core_external_store_relational_postgres_sql_parser")
;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>org.finos.legend.engine</groupId>
<artifactId>legend-engine-repl</artifactId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>

<artifactId>legend-engine-repl-app-assembly</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<artifactId>legend-engine-repl</artifactId>
<groupId>org.finos.legend.engine</groupId>
<version>4.63.2-SNAPSHOT</version>
<version>4.67.3-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,33 +19,33 @@
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.Variable;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.AppliedFunction;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.AppliedProperty;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.AppliedQualifiedProperty;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.UnknownAppliedFunction;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CBoolean;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CByteArray;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CDateTime;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CDecimal;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CFloat;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CInteger;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CLatestDate;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CStrictDate;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CStrictTime;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.CString;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Class;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.AppliedQualifiedProperty;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.UnknownAppliedFunction;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CBoolean;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CByteArray;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CDateTime;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CDecimal;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CFloat;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CInteger;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CLatestDate;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CStrictDate;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CStrictTime;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.CString;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.Class;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.ClassInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Collection;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Enum;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.EnumValue;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.GenericTypeInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.HackedUnit;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.Enum;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.EnumValue;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.packageableElement.GenericTypeInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.HackedUnit;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.KeyExpression;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Lambda;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.MappingInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.PackageableElementPtr;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.PrimitiveType;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.UnitInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.MappingInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.packageableElement.PackageableElementPtr;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.PrimitiveType;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.datatype.UnitInstance;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.UnitType;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Whatever;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.deprecated.Whatever;

public class ValueSpecificationDefaultVisitor<T> implements ValueSpecificationVisitor<T>
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import org.finos.legend.engine.language.pure.compiler.toPureGraph.PureModel;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.ValueSpecification;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.AppliedFunction;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.PackageableElementPtr;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.packageableElement.PackageableElementPtr;
import org.finos.legend.engine.repl.autocomplete.Completer;
import org.finos.legend.engine.repl.autocomplete.CompletionItem;
import org.finos.legend.engine.repl.autocomplete.FunctionHandler;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.Variable;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.application.AppliedFunction;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.Lambda;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.PackageableElementPtr;
import org.finos.legend.engine.protocol.pure.v1.model.valueSpecification.raw.packageableElement.PackageableElementPtr;
import org.finos.legend.engine.repl.autocomplete.Completer;
import org.finos.legend.engine.repl.autocomplete.CompletionItem;
import org.finos.legend.engine.repl.autocomplete.FunctionHandler;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,13 +207,13 @@ else if (buffer)
}
if (content.contains("{") && !content.contains("}"))
{
return value + "}#";
return (value.contains(magicToken) ? value : value + magicToken) + "}#";
}
if (content.contains("{") && content.contains("}"))
{
return value + "#";
}
return value + "{}#";
return value + "{" + magicToken + "}#";
}
return value;

Expand Down
Loading

0 comments on commit b40d220

Please sign in to comment.