Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docs for Pulse Delivery Guarantees #6015

Merged
merged 51 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
d2c961c
start moving terminology from subscribe to stream
nikolasburk May 18, 2024
be9c880
continue moving terminology from subscribe to stream
nikolasburk May 18, 2024
b0d85ad
add page about delivery semantics
nikolasburk May 18, 2024
a83059d
add event docs
nikolasburk May 20, 2024
e458cad
api reference finalized
nikolasburk May 20, 2024
d4e2072
continue docs work
nikolasburk May 20, 2024
7bf92e8
add page about cost
nikolasburk May 20, 2024
7611846
cleanip
nikolasburk May 20, 2024
17c5c64
fix typo
nikolasburk May 20, 2024
71e0f50
cleanup
nikolasburk May 20, 2024
6799368
cleanup
nikolasburk May 20, 2024
e7d49b0
fix links
nikolasburk May 20, 2024
4499525
fix links
nikolasburk May 20, 2024
5db3efd
add redirect
nikolasburk May 20, 2024
cb39e52
fix cspell
nikolasburk May 20, 2024
8912fc1
fix broken links
nikolasburk May 20, 2024
826237d
fix broken links
nikolasburk May 20, 2024
e589c00
minor changes and pricing faq
nikolasburk May 21, 2024
a149659
details about event persistence and pricing
nikolasburk May 21, 2024
7a18209
details about event persistence and pricing
nikolasburk May 21, 2024
e927863
add note about tsconfig
nikolasburk May 23, 2024
58a5538
Update content/400-pulse/100-what-is-pulse.mdx
nikolasburk May 24, 2024
da078c0
Update content/400-pulse/100-what-is-pulse.mdx
nikolasburk May 24, 2024
15d31b4
Update content/400-pulse/200-getting-started.mdx
nikolasburk May 24, 2024
79ea6f4
Update content/400-pulse/200-getting-started.mdx
nikolasburk May 24, 2024
3d62d9e
Update content/400-pulse/200-getting-started.mdx
nikolasburk May 24, 2024
1a035e2
Update content/400-pulse/300-database-events.mdx
nikolasburk May 24, 2024
91ac573
Update content/400-pulse/300-database-events.mdx
nikolasburk May 24, 2024
8be9aec
Update content/400-pulse/300-database-events.mdx
nikolasburk May 24, 2024
0692f56
Update content/400-pulse/300-database-events.mdx
nikolasburk May 24, 2024
040ba02
Update content/400-pulse/200-getting-started.mdx
nikolasburk May 24, 2024
8619191
add examples
nikolasburk May 24, 2024
e3e1f58
add examples
nikolasburk May 24, 2024
1071701
Update content/400-pulse/300-database-events.mdx
nikolasburk May 24, 2024
f49c6a5
Update content/400-pulse/400-api-reference.mdx
nikolasburk May 24, 2024
bbf50d8
minor changes
nikolasburk May 24, 2024
b975d80
Merge branch 'feat/pulse-dgs' of github.com:prisma/docs into feat/pul…
nikolasburk May 24, 2024
d30f7f6
minor changes
nikolasburk May 24, 2024
eab3b14
minor changes
nikolasburk May 24, 2024
2f6e371
move static redirects into proper section
nikolasburk May 24, 2024
536608b
fix typo
nikolasburk May 24, 2024
d3e6976
Update content/400-pulse/400-api-reference.mdx
nikolasburk May 24, 2024
83723a6
add note about resuming streams
nikolasburk May 24, 2024
8778a2f
Merge branch 'feat/pulse-dgs' of github.com:prisma/docs into feat/pul…
nikolasburk May 24, 2024
ca2e66e
wording updates
nikolasburk May 24, 2024
0f5518c
add note about size limit
nikolasburk May 27, 2024
0fc3b59
fix typo
nikolasburk May 27, 2024
94f8ac1
 add warning about limit to delivery guarantees
nikolasburk May 27, 2024
8b23d62
minor changes
nikolasburk May 28, 2024
3bba85b
fix typo
nikolasburk May 28, 2024
371065b
Update content/400-pulse/100-what-is-pulse.mdx
nikolasburk May 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cSpell.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@
"Letterpad",
"Hitori",
"Zenstack",
"Streamdal"
"Streamdal",
"leaderboard"
],
"ignoreWords": [
"Ania",
Expand Down
19 changes: 10 additions & 9 deletions content/400-pulse/100-what-is-pulse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ Prisma Pulse integrates with [Prisma ORM](/orm) and lets you subscribe to databa

Here's the brief overview of how Pulse works:

### 1. Subscribe to database changes with Prisma Client
### 1. Stream database change events with Prisma Client

In this example, we subscribe to _all_ changes that are happening on the `User` table in the database:
In your application, use Prisma Client to stream _all_ changes that are happening on the `User` table from the database:

```ts
// subscribe to all changes on the `User` table
const subscription = await prisma.user.subscribe()
// stream all changes from the `User` table
const stream = await prisma.user.stream()

// wait for new events to arrive
for await (let event of subscription) {
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -49,7 +49,7 @@ A change is made to the table we've previously subscribed to. In this case, a ne

![](/img/pulse/user-insert.png)

This change can happen from anywhere, the same app that uses Prisma Client to subscribe to changes, a different application or microserivce, a database GUI or any other SQL client (like `psql`).
This change can happen from anywhere, the same app that uses Prisma Client to subscribe to changes, a different application or microserivce, a database GUI, Prisma Studio, or any other SQL client (like `psql`).
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved

### 3. The database event is propagated to all subscribers

Expand Down Expand Up @@ -100,7 +100,8 @@ Prisma Pulse can power real-time functionality like chat, notifications, data br

Here are a few example projects using Prisma Pulse:

| Project | Description |
| :--------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| [pulse-starter](https://github.com/prisma/pulse-starter) | General Pulse starter project. |
| [pulse-resend-demo](https://github.com/prisma/pulse-resend-demo) | Optimize your onboarding with Pulse by triggering welcome emails via Resend for new user sign-ups. |
| Project | Description |
| :------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| [pulse-starter](https://github.com/prisma/prisma-examples) | General Pulse starter project. |
| [pulse-resend-demo](https://github.com/prisma/prisma-examples) | Optimize your onboarding with Pulse by triggering welcome emails via Resend for new user sign-ups. |
| [pulse-leaderboard](https://github.com/prisma/prisma-examples) | TBD. |
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved
43 changes: 22 additions & 21 deletions content/400-pulse/200-getting-started.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Prisma Pulse currently supports PostgreSQL. We'd love to hear [which databases](
To get started with Pulse, you will need the following:

- A [Prisma Data Platform](https://console.prisma.io) account
- The **connection string** of a Pulse-ready database (if you don't have one yet, you can configure your database the instructions [here](/pulse/database-setup) or [use a Railway template](https://railway.app/template/pulse-pg?referralCode=VQ09uv))
- The **connection string** of a Pulse-ready database (if you don't have one yet, you can [configure your database](/pulse/database-setup) or [use a Railway template](https://railway.app/template/pulse-pg?referralCode=VQ09uv))
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved

## 1. Enable Pulse in the Platform Console

Expand All @@ -35,9 +35,10 @@ In the project environment of your choice, click the **Enable Pulse** button.

The Pulse **Setup** screen requires you to:

- provide your **Database connection string**
- select a **Region** where Pulse should be hosted
- decide whether you want to use the **Automatic setup** for [**Database replication**](/pulse/concepts#logical-replication) (only available on paid plans)
1. provide your **Database connection string**
1. select a **Region** where Pulse should be hosted
1. decide whether you want to use the **Automatic setup** for **Database replication** (only available on paid plans)
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved
1. enable **Event persistence** to use delivery guarantees with [`.stream()`](/pulse/api-reference#stream) (or disable it if you want to use [`.subscribe()`](/pulse/api-reference#subscribe) for fully ephemeral events without delivery guarantees)
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved

When you're done with that, click the **Enable Pulse** button at the bottom of the screen. This will test the connectivity to your database.

Expand Down Expand Up @@ -65,11 +66,11 @@ With Pulse enabled, proceed with these steps to integrate Pulse into your applic

<Admonition>

Pulse requires [Prisma Client](/orm/prisma-client) version `4.16.1` or higher and [`@prisma/extension-pulse`](https://www.npmjs.com/package/@prisma/extension-pulse) version `1.0.1` or higher
Pulse requires [Prisma Client](/orm/prisma-client) version `4.16.1` or higher and [`@prisma/extension-pulse`](https://www.npmjs.com/package/@prisma/extension-pulse) version `1.0.1` or higher.

</Admonition>
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved

Install the Pulse Client extension:
Install the Pulse extension:

```terminal
npm install @prisma/extension-pulse@latest
Expand All @@ -88,14 +89,14 @@ const prisma = new PrismaClient().$extends(
)
```

### 2.3. Create your first Pulse subscription
### 2.3. Create your first Pulse stream

With the Pulse extension applied, you can use Pulse's `subscribe()` method on any model defined in your Prisma Schema to subscribe to data change events.
With the Pulse extension applied, you can use Pulse's `.stream()` method on any model defined in your Prisma Schema to stream data change events.

In the below example, it is assumed that your Prisma schema has a `User` model. A subscription is made to that `User` model that listens for _any_ change event on that table:
In the below example, it is assumed that your Prisma schema has a `User` model. A stream is created for that `User` model that listens for _any_ change event on that table:
nikolasburk marked this conversation as resolved.
Show resolved Hide resolved

<TabbedContent code>
<TabItem value="Subscription">
<TabItem value="Stream">

```ts highlight=9-13;add
import { PrismaClient } from '@prisma/client'
Expand All @@ -106,9 +107,9 @@ const prisma = new PrismaClient().$extends(
)

async function main() {
const subscription = await prisma.user.subscribe()
const stream = await prisma.user.stream()

for await (const event of subscription) {
for await (const event of stream) {
console.log('just received an event:', event)
}
}
Expand All @@ -132,39 +133,39 @@ model User {

</TabbedContent>

## 3. Test your subscription
## 3. Test your stream

After running the code snippet above, you can test the subscription by _creating_, _updating_ or _deleting_ a `User` record in your database.
After running the code snippet above, you can test the stream by _creating_, _updating_ or _deleting_ a `User` record in your database.

You can do that using Prisma Studio (by running `npx prisma studio`) or by using any other database client of your choice (like [Postico](https://eggerapps.at/postico2/) or `psql`).

If everything worked, you should see the event being logged to the terminal where you can the code snippet from above. 🎉

## Next steps

You can try out more filters on your Pulse subscription, for example:
You can try out more filters on your Pulse stream, for example:

**Subscribe only to `create` events**:
**Stream only `create` events**:

```ts
const subscription = await prisma.user.subscribe({
const stream = await prisma.user.stream({
create: { },
})
```

**Subscribe only to `update` events**:
**Stream only `update` events**:

```ts
const subscription = await prisma.user.subscribe({
const stream = await prisma.user.stream({
update: { },
})
```


**Subscribe only to `delete` events**:
**Stream only `delete` events**:

```ts
const subscription = await prisma.user.subscribe({
const stream = await prisma.user.stream({
delete: { },
})
```
Expand Down
Original file line number Diff line number Diff line change
@@ -1,75 +1,77 @@
---
title: 'General database instructions'
metaTitle: 'Prisma Pulse: General database instructions'
title: 'General instructions'
metaTitle: 'Prisma Pulse: General instructions'
metaDescription: 'Instructions to use your database with Prisma Pulse'
tocDepth: 3
toc_max_heading_level: 6
toc: true
---

<TopBlock>
<Admonition>

Prepare your database to work with Pulse.
Prisma Pulse requires a publicly accessible PostgreSQL (**version 12+**) database with logical replication enabled. To configure specific database providers for Prisma Pulse, visit [here](/pulse/database-setup#provider-specific-instructions).

> Prisma Pulse requires a publicly accessible PostgreSQL (**version 12+**) database with logical replication enabled. To configure specific database providers for Prisma Pulse, visit [here](/pulse/database-setup#provider-specific-instructions).

</TopBlock>

## Database Replication
</Admonition>

Database replication is the process of creating copies of a database and storing them across various on-premises or cloud destinations. Prisma Pulse uses logical replication to monitor your database for changes.

### Enable logical replication
## Enable logical replication

##### [`wal_level`](https://www.postgresql.org/docs/current/runtime-config-wal.html)
### Required settings

Some providers may not allow direct access to this setting. If you are unable to change this setting, please refer to the provider-specific guides for further assistance.
You can enable logical replication by setting [`wal_level`](https://www.postgresql.org/docs/current/runtime-config-wal.html) to `logical` in your database.

```sql
ALTER SYSTEM SET wal_level = logical;
```

You will need to restart the database after changing this setting.

##### Optional settings
> **Note**: Some providers may not allow direct access to this setting. If you are unable to change this setting via the SQL command above, please refer to the provider-specific guides for further assistance.


###### [`wal_keep_size`](https://www.postgresql.org/docs/current/runtime-config-replication.html)
### Optional settings

#### [`wal_keep_size`](https://www.postgresql.org/docs/current/runtime-config-replication.html)

Setting `wal_keep_size` increases the memory usage of the [write-ahead log](https://www.postgresql.org/docs/current/wal-intro.html) on your PostgreSQL database.

We recommend setting a value for `wal_keep_size` tailored to your database's storage capacity. This ensures smooth operation of both your database and Prisma Pulse.

> We suggest setting these values initially and adjusting them if necessary.

```sql
ALTER SYSTEM SET wal_keep_size = 2048;
```

###### [`max_replication_slots`](https://www.postgresql.org/docs/current/runtime-config-replication.html)
#### [`max_replication_slots`](https://www.postgresql.org/docs/current/runtime-config-replication.html)

Prisma Pulse only needs one replication slot available. You can set the `max_replication_slots` if you have other replications in use.

> We suggest setting these values initially and adjusting them if necessary.

```sql
ALTER SYSTEM SET max_replication_slots = 20;
```

###### [`REPLICA IDENTITY`](https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY)
#### [`REPLICA IDENTITY`](https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-REPLICA-IDENTITY)

To get the **before** values of **all fields** in the record for some events, you must set `REPLICA IDENTITY` to `FULL` on the table(s) you want to get field values for. If this is not configured, defining a filter for those events will only be possible on the primary key.
For update and delete events, you have the option to include in the event the values of the record _before_ the operation was performed:

- For `update`: Include the values of the updated record before the update was performed in a field called `before`
- For `delete`: Include the values of the deleted record.

If you enable this, you will also be able to filter on these fields.

To include this data, you must set `REPLICA IDENTITY` to `FULL` on the table(s) you want to get field values for.

For example, running the following SQL command will set the `REPLICA IDENTITY` to `FULL` on a table named `User`:

```sql
ALTER TABLE public."User" REPLICA IDENTITY FULL;
```

### Manage your own publication slot
## Manage your own publication slot

If you want to enable replication for *specific models* or use a database provider that restricts superuser access for Prisma Pulse our advanced setup allows you to configure your own publication slot and use it to enable Pulse.
If you want to enable replication for _specific models_ or use a database provider that restricts superuser access for Prisma Pulse you can configure your own publication slot and use it to enable Pulse.

#### Creating a publication slot
### Creating a publication slot

You can create publications in the following ways below depending on the version of your PostgreSQL database.

Expand All @@ -78,7 +80,7 @@ You can create publications in the following ways below depending on the version
<summary>Publication for all models.</summary>
<br/>
```sql
CREATE PUBLICATION $PUBLICATION_NAME FOR ALL TABLES;
CREATE PUBLICATION all_models FOR ALL TABLES;
```

</details>
Expand Down Expand Up @@ -138,7 +140,7 @@ You can learn more about the PostgreSQL's `CREATE PUBLICATION`, supported versio

</details>

#### Submit your publication slot
### Submit your publication slot

You can submit the publication name in the [Console](https://console.prisma.io/), before enabling Prisma Pulse:

Expand All @@ -152,14 +154,14 @@ You can submit the publication name in the [Console](https://console.prisma.io/)

![](/img/pulse/replication-slot-submission.png)

#### Removing publications
### Removing publications

If you are managing your replications independently and choose to disable Prisma Pulse for a particular environment, you can refer to the following SQL queries to remove your publications.

1. To delete a publication:

```sql
DROP PUBLICATION IF EXISTS "$PUBLICATION_SLOT_NAME";
DROP PUBLICATION IF EXISTS "my_publication";
```

2. View your [publications](https://www.postgresql.org/docs/current/view-pg-publication-tables.html):
Expand All @@ -168,4 +170,11 @@ If you are managing your replications independently and choose to disable Prisma
SELECT * FROM pg_publication_tables;
```

> 💡 To configure specific database providers for Prisma Pulse, visit [here](/pulse/database-setup#provider-specific-instructions).
## Provider-specific instructions

If you want to see the specific instructions for your database, select your database provider:

- [AWS RDS](/pulse/database-setup/aws-rds)
- [Railway](/pulse/database-setup/railway)
- [Neon](/pulse/database-setup/neon)
- [Supabase](/pulse/database-setup/supabase)
4 changes: 0 additions & 4 deletions content/400-pulse/250-database-setup/200-aws-rds.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,8 @@ tocDepth: 3
toc: true
---

<TopBlock>

You have to enable logical replication on AWS RDS to make it compatible with Prisma Pulse.

</TopBlock>

## Enable logical replication on AWS RDS

The following instructions show how to create a parameter group, enable logical replication, and add the parameter group to your AWS RDS PostgreSQL database.
Expand Down
4 changes: 0 additions & 4 deletions content/400-pulse/250-database-setup/300-railway.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,11 @@ tocDepth: 3
toc: true
---

<TopBlock>

Railway provides a useful feature known as [templates](https://railway.app/templates) to facilitate quick project setup. These templates allow users to package a service or a group of services into a format that can be easily reused and shared. We have two templates for Prisma Pulse that you can use:

- [Prisma Pulse DB Only](https://railway.app/template/pulse-pg): Provides a fresh, pre-configured PostgreSQL database which you can use with Prisma Pulse.
- [Prisma Pulse DB & App](https://railway.app/template/pulse-starter): Provides a pre-configured PostgreSQL database and a [Prisma Pulse starter app](https://github.com/prisma/pulse-starter).

</TopBlock>

## Setup with template: "Prisma Pulse DB Only"

### Overview
Expand Down
6 changes: 1 addition & 5 deletions content/400-pulse/250-database-setup/500-supabase.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,13 @@ tocDepth: 3
toc: true
---

<TopBlock>

You have to use a direct database connection from Supabase to use it with Prisma Pulse.

</TopBlock>

## Using a direct connection in Supabase

To use a direct database connection string in Supabase:

1. Navigate to your Project settings (`Settings` > `Configurations` > `Database`).
1. Navigate to your Project settings (**Settings** > **Configurations** > **Database**).
2. Disable the **Use connection pooling** option.
![Supabase direct database connection](/img/pulse/supabase-direct-database-connection.png)
3. Copy the direct database connection string.
Expand Down
Loading
Loading