Skip to content

Commit 13f280c

Browse files
aniravi24aniravi24
committed
docs: add more detailed README and JSDoc
1 parent cc24dd7 commit 13f280c

File tree

12 files changed

+342
-76
lines changed

12 files changed

+342
-76
lines changed

‎README.md

Lines changed: 171 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,186 @@
1-
# Welcome to typefusion!
1+
# Welcome to Typefusion!
22

33
<!-- TODO codecov badge -->
44

5+
## Table of Contents
6+
7+
- [Welcome to Typefusion!](#welcome-to-typefusion)
8+
- [Table of Contents](#table-of-contents)
9+
- [Introduction](#introduction)
10+
- [Getting Started](#getting-started)
11+
- [Usage](#usage)
12+
- [CLI](#cli)
13+
- [Library](#library)
14+
- [Typefusion Ref](#typefusion-ref)
15+
- [Examples](#examples)
16+
- [Docs](#docs)
17+
- [Have an issue?](#have-an-issue)
18+
- [Contributions](#contributions)
19+
520
## Introduction
621

7-
Typefusion is a tool inspired by DBT that lets you run typescript scripts, and the results are materialized into a database (currently PostgreSQL only).
22+
Typefusion lets you run TypeScript scripts, and the results are materialized into a database (currently PostgreSQL only). Scripts can reference each other's results, so you can build complex workflows. It's inspired by [Data Build Tool](https://www.getdbt.com/) (DBT).
23+
24+
## Getting Started
25+
26+
To get started with Typefusion, you need to install the package and set up your database configuration. Follow the steps below:
27+
28+
1. Install Typefusion using your preferred package manager:
29+
30+
```sh
31+
npm install typefusion
32+
```
33+
34+
```sh
35+
pnpm install typefusion
36+
```
37+
38+
```sh
39+
yarn add typefusion
40+
```
41+
42+
```sh
43+
bun add typefusion
44+
```
45+
46+
2. To setup your database configuration, you can may do one of two options:
47+
48+
- Have a full connection string in the `DATABASE_URL` environment variable.
49+
- Have `PGDATABASE`, `PGHOST`, `PGPORT`, `PGPASSWORD`, and `PGUSER` set as environment variables.
50+
51+
## Usage
52+
53+
1. First, create a directory to store your scripts.
54+
55+
2. Then, create a script file in the directory, for example, `main.ts`:
56+
57+
```ts
58+
import { pgType, TypefusionPgResult } from "typefusion";
59+
60+
export const mainSchema = {
61+
id: pgType.integer().notNull(),
62+
name: pgType.text().notNull(),
63+
age: pgType.integer().notNull(),
64+
email: pgType.text().notNull(),
65+
address: pgType.text().notNull(),
66+
};
67+
68+
export default async function main(): Promise<
69+
TypefusionPgResult<typeof mainSchema>
70+
> {
71+
console.log("running main");
72+
return {
73+
types: mainSchema,
74+
data: [
75+
{
76+
id: 1,
77+
name: "John Doe",
78+
age: 30,
79+
email: "[email protected]",
80+
address: "123 Main St",
81+
},
82+
],
83+
};
84+
}
85+
```
86+
87+
Then, you may either use the CLI or library to run your script.
88+
89+
### CLI
90+
91+
You can use the Typefusion CLI to run your scripts. Here is an example of how to run the CLI:
92+
93+
```sh
94+
npm run typefusion --help # The CLI mode has some nifty features, check them out!
95+
```
96+
97+
A common setup in your package.json scripts (presuming you have dotenv setup with a `.env` file in your project):
98+
99+
```json
100+
"scripts": {
101+
"run-typefusion": "dotenv -- typefusion ./scripts"
102+
}
103+
```
104+
105+
Then you can run your scripts with:
106+
107+
```sh
108+
npm run run-typefusion
109+
```
110+
111+
You should now see the output of your script in your database! There should be a table called `main` with the data you returned from the script above.
112+
113+
### Library
114+
115+
You can also use Typefusion as a library in your TypeScript projects. Here is an example of how to use it:
116+
117+
```ts
118+
import typefusion from "typefusion";
119+
120+
await typefusion({
121+
directory: "./scripts",
122+
});
123+
```
124+
125+
## Typefusion Ref
126+
127+
Typefusion Refs allow you to reference the results of a script in another script. It is useful for building complex workflows. Following the usage of the library above, here is an example of how to use Typefusion Ref to reference the results of the `main` script:
128+
129+
```ts
130+
import { pgType, typefusionRef, TypefusionPgResult } from "typefusion";
131+
import main from "./main.js";
132+
133+
const smallSchema = {
134+
small: pgType.text().notNull(),
135+
};
136+
137+
export default async function typefusion_ref(): Promise<
138+
TypefusionPgResult<typeof smallSchema>
139+
> {
140+
const result = await typefusionRef(main);
141+
console.log("typefusion ref main result", result);
142+
return {
143+
types: smallSchema,
144+
data: [
145+
{
146+
small: "smallString" as const,
147+
},
148+
],
149+
};
150+
}
151+
```
152+
153+
You'll notice that the `main` script is run first, and then the `typefusion_ref` script is run. The result of the `main` script is then used in the `typefusion_ref` script. `result` is fully type-safe depending on the types you used in the `main` script.
154+
155+
As long as there are no circular dependencies, you can reference any script from any other script, allowing for complex workflows.
156+
157+
If you want to run a query of some kind and want to just grab the table name without getting the full data, you can use the `typefusionRefTableName` function. This is useful if you want to run a query of some kind and want to just grab the table name from the other script without the full data.
158+
159+
```ts
160+
import { typefusionRefTableName } from "typefusion";
161+
import main from "./main.js";
162+
163+
const tableName = await typefusionRefTableName(main);
164+
console.log("typefusion ref table name", tableName); // should print out "main"
165+
```
166+
167+
## Examples
168+
169+
Here are some examples of how to use Typefusion:
170+
171+
1. [Main Example](packages/typefusion/example/main.ts)
172+
2. [Ref Example](packages/typefusion/example/options/typefusion_pg_result.ts)
173+
174+
You can find the remaining examples [here](packages/typefusion/example).
8175

9176
## Docs
10177

11-
Reference docs can be found at https://aniravi24.github.io/typefusion
178+
Reference docs can be found at [https://aniravi24.github.io/typefusion](https://aniravi24.github.io/typefusion). JSDoc comments are used to document more details not covered in this README.
12179

13180
## Have an issue?
14181

15182
Open a GitHub issue [here](https://github.com/aniravi24/typefusion/issues/new)
16183

17184
## Contributions
18185

19-
Please open an issue to document a bug or suggest a feature request before opening a PR!
186+
Please open an issue to document a bug or suggest a feature request before opening a PR.

‎packages/typefusion/package.json

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,17 @@
2525
"typefusion": "./dist/cli.js"
2626
},
2727
"exports": {
28-
".": "./dist/index.js",
29-
"./effect": "./dist/effect.js"
28+
".": {
29+
"import": "./dist/index.js",
30+
"require": "./dist/index.cjs"
31+
},
32+
"./effect": {
33+
"import": "./dist/effect.js",
34+
"require": "./dist/effect.cjs"
35+
}
3036
},
31-
"main": "dist/index.js",
37+
"main": "./dist/index.cjs",
38+
"module": "./dist/index.js",
3239
"types": "dist/index.d.ts",
3340
"scripts": {
3441
"build": "tsup",

‎packages/typefusion/src/cli.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ const command = Command.make(
3030

3131
const cli = Command.run(command, {
3232
name: "Typefusion CLI",
33-
version: "v0.0.1",
33+
version: "IGNORE_VERSION_OUTPUT_HERE_CHECK_YOUR_PACKAGE_JSON",
3434
});
3535

3636
Effect.suspend(() => cli(process.argv)).pipe(

‎packages/typefusion/src/db/postgres/types.ts

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,17 @@
1+
import { Data, Effect } from "effect";
2+
13
export type Nullable<T> = T | null;
24

5+
/**
6+
* This is a simple wrapper class to represent a Postgres type that will be used to define a table.
7+
* It also provides a fluent API to set properties of the column (e.g. nullability).
8+
* You can easily create your own custom types by instantiating this class.
9+
*
10+
* @example
11+
* ```ts
12+
* const myCustomType = new PgType<Nullable<string>>("myCustomType");
13+
* ```
14+
*/
315
export class PgType<T> {
416
public _type: T;
517
private _nullable: boolean;
@@ -76,3 +88,55 @@ export const pgType = {
7688
xml: () => new PgType<Nullable<string>>("xml"),
7789
bytea: () => new PgType<Nullable<Uint8Array>>("bytea"),
7890
};
91+
92+
export class UnsupportedJSTypePostgresConversionError extends Data.TaggedError(
93+
"UnsupportedJSTypePostgresConversionError",
94+
)<{
95+
cause: unknown;
96+
message: string;
97+
}> {}
98+
99+
// People shouldn't really use this function, it's only useful for the simplest of types
100+
export const valueToPostgresType = (value: unknown) =>
101+
Effect.gen(function* () {
102+
if (value === null || value === undefined) {
103+
return "TEXT";
104+
}
105+
if (value instanceof Date) {
106+
return "TIMESTAMP WITH TIME ZONE";
107+
}
108+
switch (typeof value) {
109+
case "object":
110+
return "JSONB";
111+
case "string":
112+
return "TEXT";
113+
case "bigint":
114+
return "BIGINT";
115+
case "number":
116+
if (Number.isInteger(value)) {
117+
// PostgreSQL INTEGER range: -2,147,483,648 to +2,147,483,647
118+
const MIN_INTEGER = -2147483648;
119+
const MAX_INTEGER = 2147483647;
120+
if (value >= MIN_INTEGER && value <= MAX_INTEGER) {
121+
return "INTEGER";
122+
}
123+
return "BIGINT";
124+
}
125+
return "DOUBLE PRECISION";
126+
case "boolean":
127+
return "BOOLEAN";
128+
default:
129+
return yield* Effect.fail(
130+
new UnsupportedJSTypePostgresConversionError({
131+
cause: null,
132+
message: `Unsupported type for value provided in script result: ${typeof value}`,
133+
}),
134+
);
135+
}
136+
});
137+
138+
export const postgresIdColumn = (type?: PgType<unknown>) => {
139+
const idType =
140+
type?.getPostgresType() || "BIGINT GENERATED ALWAYS AS IDENTITY";
141+
return `id ${idType} PRIMARY KEY` as const;
142+
};

‎packages/typefusion/src/effect.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ import { typefusion } from "./typefusion.js";
22

33
export { typefusionRefEffect, typefusionRefTableNameEffect } from "./lib.js";
44

5+
export { typefusion };
6+
57
export default typefusion;

‎packages/typefusion/src/helpers.ts

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,11 @@ export class ModuleExecutionError extends Data.TaggedError(
5050
message: string;
5151
}> {}
5252

53+
/**
54+
* Runs a module and inserts the result into the database.
55+
* @param leaf - The relative path of the module to run.
56+
* @returns void
57+
*/
5358
export function runModule(leaf: string) {
5459
return Effect.gen(function* () {
5560
const path = `../${leaf}`;

‎packages/typefusion/src/index.ts

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
import { Effect } from "effect";
2-
import { typefusion, TypefusionConfig } from "./typefusion.js";
2+
import {
3+
typefusion as typefusionEffect,
4+
TypefusionConfig,
5+
} from "./typefusion.js";
36

47
export {
58
DependencyGraphGenerationError,
@@ -8,20 +11,23 @@ export {
811

912
export { typefusionRef, typefusionRefTableName } from "./lib.js";
1013

14+
export { UnsupportedJSTypePostgresConversionError } from "./db/postgres/types.js";
15+
1116
export {
1217
TypefusionPgResult,
1318
TypefusionResult,
1419
TypefusionResultDataOnly,
1520
TypefusionResultUnknown,
1621
ConvertDataToSQLDDLError,
17-
DatabaseGetError,
22+
DatabaseSelectError,
1823
DatabaseInsertError,
19-
UnsupportedTypeError,
2024
} from "./store.js";
2125

2226
export { ModuleExecutionError, ModuleImportError } from "./helpers.js";
2327

2428
export * from "./db/postgres/types.js";
2529

26-
export default (config: TypefusionConfig) =>
27-
typefusion(config).pipe(Effect.runPromise);
30+
export const typefusion = (config: TypefusionConfig) =>
31+
typefusionEffect(config).pipe(Effect.runPromise);
32+
33+
export default typefusion;

‎packages/typefusion/src/lib.ts

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,17 @@
11
import { Effect } from "effect";
2-
import { DatabaseGetError, dbSelect, TypefusionScriptResult } from "./store.js";
2+
import {
3+
DatabaseSelectError,
4+
dbSelect,
5+
TypefusionScriptResult,
6+
} from "./store.js";
37
import { SqlLive } from "./db/postgres/client.js";
48
import { ConfigError } from "effect/ConfigError";
59

10+
/**
11+
* Get the data from a module (i.e. the result of one of your Typefusion scripts).
12+
* @param module - The module to get the data from.
13+
* @returns The data from the associated table for that module.
14+
*/
615
export const typefusionRef = async <
716
T extends (...args: any[]) => PromiseLike<TypefusionScriptResult>,
817
>(
@@ -26,6 +35,9 @@ export const typefusionRef = async <
2635
) as any;
2736
};
2837

38+
/**
39+
* Analogous to {@link typefusionRef} but for use in Effect.
40+
*/
2941
export const typefusionRefEffect = <
3042
T extends (...args: any[]) => PromiseLike<TypefusionScriptResult>,
3143
>(
@@ -42,7 +54,7 @@ export const typefusionRefEffect = <
4254
: Awaited<ReturnType<T>>["data"] extends Array<infer U>
4355
? U
4456
: never,
45-
DatabaseGetError | ConfigError
57+
DatabaseSelectError | ConfigError
4658
> => {
4759
return dbSelect(module).pipe(Effect.provide(SqlLive)) as any;
4860
};
@@ -61,6 +73,9 @@ export const typefusionRefTableName = async <
6173
return module.name;
6274
};
6375

76+
/**
77+
* Analogous to {@link typefusionRefTableName} but for use in Effect.
78+
*/
6479
export const typefusionRefTableNameEffect = <
6580
T extends (...args: any[]) => PromiseLike<TypefusionScriptResult>,
6681
>(

0 commit comments

Comments
 (0)