Skip to content

Commit

Permalink
Merge pull request #539 from jvalue/refactor-valuetype-to-value-type
Browse files Browse the repository at this point in the history
Use camel case for value type (valueType) internally
  • Loading branch information
georg-schwarz authored Apr 9, 2024
2 parents b851b83 + a54d948 commit 23dd62b
Show file tree
Hide file tree
Showing 131 changed files with 821 additions and 802 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,17 +19,17 @@ Data engineers can use Jayvee and its interpreter to clean and preprocess data f
Explore a glimpse of our upcoming features in the following list. This overview is broad and subject to evolution. We're excited to share our vision of the exciting journey ahead, and we invite you to accompany us on this adventure!

- ✅ Blocks and pipes
- ✅ Simple valuetypes and constraints
- ✅ Simple value types and constraints
- ✅ Natively support table-based data
- ✅ Column-based transformations
- ✅ Describe blocks via builtin blocktypes in Jayvee
- ✅ Compose logic of multiple blocks via composite blocktypes
- ⌛ Multi-file Jayvee to distribute programs over multiple files (see [RFC 0015](./rfc/0015-multi-file-jayvee/0015-multi-file-jayvee.md))
- ⌛ Improve the syntax of valuetypes (see [RFC 0014](https://github.com/jvalue/jayvee/pull/409))
- ⌛ Improve the syntax of value types (see [RFC 0014](https://github.com/jvalue/jayvee/pull/409))
- ⌛ Jayvee formatter
- ⌛ Further extractors and sinks
- 🤔 Reusable libraries (with a package manager)
- 🤔 Composite valuetypes (with multiple fields)
- 🤔 Composite value types (with multiple fields)
- 🤔 Natively support tree data (XML, JSON)
- 🤔 Valuetypes parsers (to read and write different formats)
- 🤔 Customizable invalid value handling (default value, average, median, interpolation, ...)
Expand Down
4 changes: 2 additions & 2 deletions apps/docs/docs/dev/04-guides/05-standard-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The following elements are part of the standard library:

The implementations of builtin contents are not expressed in Jayvee itself but on the TypeScript layer. Examples:

- **Builtin valuetypes**: These valuetypes are the base for defining user-defined valuetypes in Jayvee, e.g., `text`, `integer`, `decimal`, `boolean`.
- **Builtin valuetypes**: These valuetypes are the base for defining user-defined value types in Jayvee, e.g., `text`, `integer`, `decimal`, `boolean`.
- **Builtin iotypes**: These iotypes are used to describe in inputs and outputs of blocktypes, e.g., `Sheet`, `File`.
- **Builtin blocktypes**: These blocktypes are the very basic building blocks in Jayvee, e.g., `HttpExtractor`, `SqliteLoader`.
- **Builtin constraint types**: These constraint types are constraints with custom logic, e.g., `LengthConstraint`, `RegexConstraint`.
Expand All @@ -25,7 +25,7 @@ Builtin definitions are usually generated and added to the standard library from

The implementations of user-defined contents are expressed in Jayvee itself. Examples:

- **User-defined valuetypes**: These valuetypes are based on builtin or other user-defined valuetypes. Their definition is expressed natively in Jayvee, e.g., `Percent`.
- **User-defined valuetypes**: These valuetypes are based on builtin or other user-defined value types. Their definition is expressed natively in Jayvee, e.g., `Percent`.
- **User-defined blocktypes**: These blocktypes are based on builtin or other user-defined blocktypes. Their definition is expressed natively in Jayvee.

We use `jv` files to add user-defined valuetypes to the standard library (see below).
Expand Down
4 changes: 2 additions & 2 deletions apps/docs/docs/user/core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,10 @@ Blocks can be either defined as part of the language, called `builtin` or define
A `ValueType` is the definition of a data type of the processed data.
Some `Blocks` use `ValueTypes` to define logic (like filtering or assessing the data type in a data sink).
We differentiate the following types of `ValueTypes`:
- `Built-in ValueTypes` come with the basic version of Jayvee. See [Built-in Valuetypes](./valuetypes/builtin-valuetypes).
- `Built-in ValueTypes` come with the basic version of Jayvee. See [Built-in Valuetypes](./value-types/builtin-value-types).
- `Primitive ValueTypes` can be defined by the user to model domain-specific data types and represent a single value.
`Constraints` can be added to a `Primitive ValueType`.
See [Primitive Valuetypes](./valuetypes/primitive-valuetypes).
See [Primitive Valuetypes](./value-types/primitive-value-types).
- `Compound ValueTypes`: UPCOMING.

```jayvee
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/docs/user/expressions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Expressions in Jayvee are arbitrarily nested statements. They consist of:
- variables (e.g., declared by `from` properties in [Transforms](./transforms.md))
- operators (e.g., `*` or `sqrt`)

Expressions get evaluated at runtime by the interpreter to a [Built-in ValueType](./valuetypes/builtin-valuetypes).
Expressions get evaluated at runtime by the interpreter to a [Built-in ValueType](./value-types/builtin-value-types).

### Example

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
#
# SPDX-License-Identifier: AGPL-3.0-only

builtin-valuetypes.md
builtin-value-types.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"label": "Valuetypes",
"label": "ValueTypes",
"position": 5,
"link": {
"type": "generated-index",
"description": "Jayvee supports these different kinds of valuetypes."
"description": "Jayvee supports these different kinds of value types."
}
}
11 changes: 9 additions & 2 deletions apps/docs/generator/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,19 @@ function generateValueTypeDocs(
services: JayveeServices,
rootPath: string,
): void {
const docsPath = join(rootPath, 'apps', 'docs', 'docs', 'user', 'valuetypes');
const docsPath = join(
rootPath,
'apps',
'docs',
'docs',
'user',
'value-types',
);
const userDocBuilder = new UserDocGenerator(services);
const valueTypeDoc =
userDocBuilder.generateValueTypesDoc(PrimitiveValuetypes);

const fileName = `builtin-valuetypes.md`;
const fileName = `builtin-value-types.md`;
writeFileSync(join(docsPath, fileName), valueTypeDoc, {
flag: 'w',
});
Expand Down
12 changes: 6 additions & 6 deletions apps/docs/generator/src/user-doc-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
JayveeServices,
JayveeValueTypesDocGenerator,
MarkdownBuilder,
PrimitiveValuetype,
PrimitiveValueType,
PropertySpecification,
} from '@jvalue/jayvee-language-server';

Expand All @@ -27,20 +27,20 @@ export class UserDocGenerator
constructor(private services: JayveeServices) {}

generateValueTypesDoc(
valueTypes: Record<string, PrimitiveValuetype>,
valueTypes: Record<string, PrimitiveValueType>,
): string {
const builder = new UserDocMarkdownBuilder()
.docTitle('Built-in Valuetypes')
.docTitle('Built-in ValueTypes')
.generationComment()
.description(
`
For an introduction to valuetypes, see the [Core Concepts](../core-concepts).
Built-in valuetypes come with the basic version of Jayvee.
They are the basis for more restricted [Primitive Valuetypes](./primitive-valuetypes)
that fullfil [Constraints](./primitive-valuetypes#constraints).`.trim(),
They are the basis for more restricted [Primitive Valuetypes](./primitive-value-types)
that fullfil [Constraints](./primitive-value-types#constraints).`.trim(),
1,
)
.heading('Available built-in valuetypes', 1);
.heading('Available built-in value types', 1);

Object.entries(valueTypes)
// eslint-disable-next-line @typescript-eslint/no-unused-vars
Expand Down
10 changes: 5 additions & 5 deletions libs/execution/src/lib/blocks/composite-block-executor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ import {
EvaluationContext,
IOType,
InternalValueRepresentation,
Valuetype,
ValueType,
WrapperFactoryProvider,
createValuetype,
createValueType,
evaluateExpression,
evaluatePropertyValue,
getIOType,
Expand Down Expand Up @@ -121,11 +121,11 @@ export function createCompositeBlockExecutor(
context: ExecutionContext,
) {
properties.forEach((blocktypeProperty) => {
const valueType = createValuetype(blocktypeProperty.valueType);
const valueType = createValueType(blocktypeProperty.valueType);

assert(
valueType,
`Can not create valuetype for blocktype property ${blocktypeProperty.name}`,
`Can not create value type for blocktype property ${blocktypeProperty.name}`,
);

const propertyValue = this.getPropertyValueFromBlockOrDefault(
Expand All @@ -151,7 +151,7 @@ export function createCompositeBlockExecutor(

private getPropertyValueFromBlockOrDefault(
name: string,
valueType: Valuetype,
valueType: ValueType,
block: BlockDefinition,
properties: BlocktypeProperty[],
evaluationContext: EvaluationContext,
Expand Down
2 changes: 1 addition & 1 deletion libs/execution/src/lib/debugging/debug-log-visitor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ export class DebugLogVisitor implements IoTypeVisitor<void> {

const headers = [...table.getColumns().entries()]
.map(([columnName, column]) => {
return `${columnName} (${column.valuetype.getName()})`;
return `${columnName} (${column.valueType.getName()})`;
})
.join(' | ');
this.log(`[Header] ${headers}`);
Expand Down
12 changes: 6 additions & 6 deletions libs/execution/src/lib/execution-context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import {
PipelineDefinition,
PropertyAssignment,
TransformDefinition,
Valuetype,
ValueType,
type WrapperFactoryProvider,
evaluatePropertyValue,
isBlockDefinition,
Expand Down Expand Up @@ -88,19 +88,19 @@ export class ExecutionContext {

public getPropertyValue<I extends InternalValueRepresentation>(
propertyName: string,
valuetype: Valuetype<I>,
valueType: ValueType<I>,
): I {
const property = this.getProperty(propertyName);

if (property === undefined) {
return this.getDefaultPropertyValue(propertyName, valuetype);
return this.getDefaultPropertyValue(propertyName, valueType);
}

const propertyValue = evaluatePropertyValue(
property,
this.evaluationContext,
this.wrapperFactories,
valuetype,
valueType,
);
assert(propertyValue !== undefined);
return propertyValue;
Expand Down Expand Up @@ -132,15 +132,15 @@ export class ExecutionContext {

private getDefaultPropertyValue<I extends InternalValueRepresentation>(
propertyName: string,
valuetype: Valuetype<I>,
valueType: ValueType<I>,
): I {
const wrapper = this.getWrapperOfCurrentNode();
const propertySpec = wrapper.getPropertySpecification(propertyName);
assert(propertySpec !== undefined);

const defaultValue = propertySpec.defaultValue;
assert(defaultValue !== undefined);
assert(valuetype.isInternalValueRepresentation(defaultValue));
assert(valueType.isInternalValueRepresentation(defaultValue));

return defaultValue;
}
Expand Down
2 changes: 1 addition & 1 deletion libs/execution/src/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ export * from './transforms';
export * from './types';
export * from './util';
export * from './debugging';
export * from './types/valuetypes/visitors';
export * from './types/value-types/visitors';

export * from './execution-context';
export * from './extension';
Expand Down
Loading

0 comments on commit 23dd62b

Please sign in to comment.