Skip to content

Commit

Permalink
docs: rewrite data synchronization intro
Browse files Browse the repository at this point in the history
  • Loading branch information
doodlewind committed Jan 9, 2024
1 parent d54a817 commit 62232b4
Show file tree
Hide file tree
Showing 17 changed files with 158 additions and 149 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ This can be illustrated as the diagram below:
In addition to extending custom blocks, here are what you can also conveniently achieve with BlockSuite:

- Writing type-safe complex editing logic based on the [command](https://blocksuite.io/command.html) mechanism, similar to react hooks designed for document editing.
- Persistence of documents and compatibility with various third-party formats (such as markdown and HTML) based on block [snapshot](https://blocksuite.io/data-persistence.html#snapshot-api) and transformer.
- Incremental updates, real-time collaboration, local-first state management, and even decentralized data synchronization based on the [provider](https://blocksuite.io/data-persistence.html#provider-based-state-management) mechanism of the document.
- Persistence of documents and compatibility with various third-party formats (such as markdown and HTML) based on block [snapshot](https://blocksuite.io/data-synchronization.html#snapshot-api) and transformer.
- Incremental updates, real-time collaboration, local-first state management, and even decentralized data synchronization based on the [provider](https://blocksuite.io/data-synchronization.html#provider-based-state-management) mechanism of the document.
- State scheduling across multiple documents and reusing one document in multiple editors.

> 🚧 BlockSuite is currently in its early stage, with some extension capabilities still under refinement. Hope you can stay tuned, try it out, or share your feedback!
Expand Down
2 changes: 1 addition & 1 deletion packages/docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ export default defineConfig({
},
],
},
{ text: 'Data Persistence', link: '/data-persistence' },
{ text: 'Data Synchronization', link: '/data-synchronization' },
],
},
{
Expand Down
2 changes: 1 addition & 1 deletion packages/docs/adapter.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Adapter

Adapter works as a bridge between different formats of data and the BlockSuite [`Snapshot`](./data-persistence#snapshot-api) (i.e., the JSON-serialized block tree). It enables you to import and export data from and to BlockSuite documents.
Adapter works as a bridge between different formats of data and the BlockSuite [`Snapshot`](./data-synchronization#snapshot-api) (i.e., the JSON-serialized block tree). It enables you to import and export data from and to BlockSuite documents.

## Base Adapter

Expand Down
4 changes: 0 additions & 4 deletions packages/docs/block-schema.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Block Schema

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

In BlockSuite, all blocks should have a schema. The schema of the block describes the data structure of the block.

You can use the `defineBlockSchema` function to define the schema of the block.
Expand Down
4 changes: 0 additions & 4 deletions packages/docs/block-service.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Block Service

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

Each kind of block can register its own service, so as to define block-specific methods to be called during the editor lifecycle. The service is a class that extends the `BlockService` class:

```ts
Expand Down
4 changes: 0 additions & 4 deletions packages/docs/block-spec.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Block Spec

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

In BlockSuite, a `BlockSpec` defines the structure and interactive elements for a specific block type within the editor. BlockSuite editors are typically composed entirely of block specs, with the top-level UI often implemented as a dedicated block, usually of the `affine:page` type.

A block spec contains the following properties:
Expand Down
4 changes: 0 additions & 4 deletions packages/docs/block-view.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Block View

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

In BlockSuite, blocks can be rendered by any UI framework. A block should be rendered to a DOM element, and we use `view` to represent the renderer.

By default, we provide a [lit](https://lit.dev/) renderer called `@blocksuite/lit`. But it's still possible to use other UI frameworks. We'll introduce later about how to write custom block renderers.
Expand Down
4 changes: 0 additions & 4 deletions packages/docs/block-widgets.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Block Widgets

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

In BlockSuite, widgets are components that can be used to display helper UI elements of a block. Sometimes, you want to display a menu to provide some extra information or actions for a block. As another example, it's a common practice to display a toolbar when you select a block.

The widget is designed to provide this kind of functionalities. Similar to blocks, widgets also depends on UI frameworks. By default, we provide a [lit](https://lit.dev/) renderer called `@blocksuite/lit`. But it's still possible to use other UI frameworks. We'll introduce later about how to write custom block renderers.
Expand Down
4 changes: 0 additions & 4 deletions packages/docs/command.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Command

::: info
This document is preliminary and subject to refinement and updates for clarity and accuracy.
:::

Commands are the reusable actions for triggering state updates. Inside a command, you can query different states of the editor, or perform operations to update them. With the command API, you can define chainable commands and execute them.

## Command Chain
Expand Down
115 changes: 0 additions & 115 deletions packages/docs/data-persistence.md

This file was deleted.

86 changes: 86 additions & 0 deletions packages/docs/data-synchronization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Data Synchronization

::: info
🌐 This documentation has a [Chinese translation](https://insider.affine.pro/share/af3478a2-9c9c-4d16-864d-bffa1eb10eb6/xiObHbAC0yUb7HmX4-fjg).
:::

This guide explores several optimal ways to synchronize (in other words, save and load) documents in BlockSuite.

## Snapshot API

Traditionally, you might expect a JSON-based API that works somewhat like `editor.load()`. For such scenarios, BlockSuite indeed conveniently fulfills this need through its built-in snapshot mechanism:

```ts
import { Job } from '@blocksuite/store';

const { workspace } = page;

// A job is required for performing the tasks
const job = new Job({ workspace });

// Export current page content to snapshot JSON
const json = await job.pageToSnapshot(page);

// Import snapshot JSON to a new page
const newPage = await job.snapshotToPage(json);
```

The snapshot stores the JSON representation of the `page` block tree, preserving its nested structure. Additionally, BlockSuite has designed an [Adapter](./adapter) API on top of the snapshot to handle conversions between the block tree and third-party formats like markdown and HTML.

## Provider-Based State Management

Different from the classic mechanism above, BlockSuite natively supports a state management strategy that can be mentally paralleled with [React Server Component](https://www.joshwcomeau.com/react/server-components/). This allows the state of the block tree to be directly used as serializable data, streaming from the server (or local database) to the client.

In this case, **the document data stored on the server is no longer JSON, but always a binary representation of CRDT** (similar to protobuf or RSC payload). As the block tree in BlockSuite is natively implemented by CRDT, and the CRDT data is always updated first during state updates ([see this article](./crdt-native-data-flow)), the block tree state in the BlockSuite editor is always driven entirely by CRDT data. Therefore, compared to the RSC mindset:

```
ui = f(data)(state)
```

The BlockSuite mindset is always:

```
ui = f(data)
```

This is equivalent to updating the server first every time you update a todo list item, and then updating the state with the data returned from the server. However, with the ability of CRDT that automatically resolves conflicts, this process can be reliably completed locally and synchronized with remote documents.

In contrast, traditional editors typically only support APIs like `editor.load()`, which is more similar to a compromised `f(data)(state)` model, and has more complexity when dealing with real-time collaboration with multiple data sources.

In BlockSuite, the data-driven synchronization strategy is implemented through providers:

- When creating a new document, you only need to connect the `page` to a specific provider (or multiple providers) to expect the CRDT data of the block tree to be synchronized via these providers.
- Similarly, when loading an existing document, the method is to create a new empty `page` object and connect it to the corresponding provider. At this time, the block tree data will also flow in from the provider data source:

```ts
// IndexedDB provider from Yjs community
import { IndexeddbPersistence } from 'y-indexeddb';

// Let's start from an empty page
const { page } = createEmptyPage();

// `page.spaceDoc` is the underlying CRDT data structure.
// Here we connect the doc to the IndexedDB table named 'my-doc'
const provider = new IndexeddbPersistence('my-doc', page.spaceDoc);

// Case 1.
// If you are creating a new page,
// init here and the block will be automatically written to IndexedDB
page.load(() => {
page.addBlock('affine:page');
// ...
});

// Case 2.
// If you are loading an existing page,
// simply wait here and your content will be ready
await page.load();
```

In both cases, whether the document is **loaded** or **created**, the [`page.slots.ready`](/api/@blocksuite/store/classes/Page.html#ready-1) slot will be triggered, indicating that the document has been initialized.

Furthermore, by connecting multiple providers, documents can automatically be synchronized to a variety of different backends:

![pluggable-providers](./images/pluggable-providers.png)

This brings great flexibility and is the pattern currently being used in [AFFiNE](https://github.com/toeverything/AFFiNE).
6 changes: 6 additions & 0 deletions packages/docs/lit.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,9 @@
Intermediate layer for adapting the block tree to the [lit](https://lit.dev/) framework component tree UI.

BlockSuite uses lit as the default framework because lit components are native web components, avoiding synchronization issues between the component tree and DOM tree during complex editing.

The [`EditorHost`](/api/@blocksuite/lit/classes/EditorHost.html) is a lit component that works as the DOM host of the block tree, and the [`BlockElement`](/api/@blocksuite/lit/classes/BlockElement.html) and [`WidgetElement`](/api/@blocksuite/lit/classes/WidgetElement.html) are standard lit components for extending UI components of block and widget.

::: tip
Lit components extends `HTMLElement`, so all DOM-related properties are inherited.
:::
4 changes: 2 additions & 2 deletions packages/docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ This can be illustrated as the diagram below:
In addition to extending custom blocks, here are what you can also conveniently achieve with BlockSuite:

- Writing type-safe complex editing logic based on the [command](./command) mechanism, similar to react hooks designed for document editing.
- Persistence of documents and compatibility with various third-party formats (such as markdown and HTML) based on block [snapshot](./data-persistence#snapshot-api) and transformer.
- Incremental updates, real-time collaboration, local-first state management, and even decentralized data synchronization based on the document's [provider](./data-persistence#provider-based-state-management) mechanism.
- Persistence of documents and compatibility with various third-party formats (such as markdown and HTML) based on block [snapshot](./data-synchronization#snapshot-api) and transformer.
- Incremental updates, real-time collaboration, local-first state management, and even decentralized data synchronization based on the document's [provider](./data-synchronization#provider-based-state-management) mechanism.
- State scheduling across multiple documents and reusing one document in multiple editors.

To try out BlockSuite, refer to the [Quick Start](./quick-start) document and start with the preset editors in `@blocksuite/presets`.
Expand Down
4 changes: 4 additions & 0 deletions packages/docs/presets/doc-editor.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# `DocEditor`

::: info
The comprehensive API surface of this preset is still a work in progress. Please stay tuned!
:::

This editor preset is great at block-based rich text editng, with drag handle, slash menu, format toolbar and other built-in powerful widgets combined.

```ts
Expand Down
4 changes: 4 additions & 0 deletions packages/docs/presets/edgeless-editor.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# `EdgelessEditor`

::: info
The comprehensive API surface of this preset is still a work in progress. Please stay tuned!
:::

This editor preset is great at whiteboard graphics editing capabilites. It combines canvas-based graphics rendering and DOM-based block tree editing together. This facilitates both creative graphic design and structured document editing, catering to a wide range of user needs and workflows.

```ts
Expand Down
52 changes: 50 additions & 2 deletions packages/docs/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,56 @@ This package is the data layer for modeling collaborative document states. It's

## `Page`

TODO
In BlockSuite, a [`Page`](/api/@blocksuite/store/classes/Page.html) is the container for a block tree, providing essential functionalities for creating, retrieving, updating, and deleting blocks inside it. Under the hood, every page holds a Yjs [subdocument](https://docs.yjs.dev/api/subdocuments).

Besides the block tree, the [selection](./selection) state is also stored in the [`page.awarenessStore`](/api/@blocksuite/store/classes/Page.html#awarenessstore) inside the page. This store is also built on top of the Yjs [awareness](https://docs.yjs.dev/api/about-awareness).

## `Workspace`

TODO
In BlockSuite, a [`Workspace`](/api/@blocksuite/store/classes/Workspace.html) is defined as an opt-in collection of multiple pages, providing comprehensive features for managing cross-page updates and data synchronization. You can access the workspace via the `page.workspace` getter, or you can also create a workspace manually:

```ts
import { Workspace, Schema } from '@blocksuite/store';

const schema = new Schema();

// You can register a batch of block schemas to the workspace
schema.register(AffineSchemas);

const workspace = new Workspace({ schema });
```

Then multiple `page`s can be created under the workspace:

```ts
const workspace = new Workspace({ schema });

// This is an empty page at this moment
const page = workspace.createPage();
```

As an example, the `createEmptyPage` is a simple helper implemented exactly in this way ([source](https://github.com/toeverything/blocksuite/blob/master/packages/presets/src/helpers/index.ts)):

```ts
import { AffineSchemas } from '@blocksuite/blocks/models';
import { Schema, Workspace } from '@blocksuite/store';

export function createEmptyPage() {
const schema = new Schema().register(AffineSchemas);
const workspace = new Workspace({ schema });
const page = workspace.createPage();

return {
page,
async init() {
await page.load(() => {
const pageBlockId = page.addBlock('affine:page', {});
page.addBlock('affine:surface', {}, pageBlockId);
const noteId = page.addBlock('affine:note', {}, pageBlockId);
page.addBlock('affine:paragraph', {}, noteId);
});
return page;
},
};
}
```
Loading

2 comments on commit 62232b4

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Size Report

Bundles

Entry Size Gzip Brotli
examples/basic 12.5 MB (+3.79 kB) 2.61 MB (+1.25 kB) 1.64 MB (+420 B)

Packages

Name Size Gzip Brotli
blocks 2.09 MB (+437 B) 481 kB (+92 B) 362 kB (+299 B)
editor 84 B 89 B 63 B
store 62 kB 17.7 kB 15.8 kB
inline 32.5 kB 8.94 kB 8.01 kB

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Size Report

Bundles

Entry Size Gzip Brotli
examples/basic 12.5 MB 2.61 MB 1.64 MB

Packages

Name Size Gzip Brotli
blocks 2.09 MB 481 kB 362 kB
editor 84 B 89 B 63 B
store 62 kB 17.7 kB 15.8 kB
inline 32.5 kB 8.94 kB 8.01 kB

Please sign in to comment.