Skip to content

Commit

Permalink
docs: add integrations section to docs
Browse files Browse the repository at this point in the history
  • Loading branch information
EgeAytin committed Sep 26, 2024
1 parent 83b7ea5 commit 6b11c4b
Show file tree
Hide file tree
Showing 6 changed files with 457 additions and 0 deletions.
150 changes: 150 additions & 0 deletions docs/integrations/data-sync.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
title: Data Synchronization
---

<Note>
This is a <strong>Permify Cloud</strong> feature. To get more information, [schedule a call with an expert](https://calendly.com/d/cj79-kyf-b4z).
</Note>

Permify can connect to your databases to capture changes in your application’s database and sync authorization-related data in real time.

By using **Debezium**, **Kafka**, and the **Permify Sync Service**, your system can efficiently handle updates to permissions and roles without manual intervention.

## Permify Sync Architecture Overview

![data-sync](https://github.com/user-attachments/assets/a08ee2f2-7a3d-40df-9751-e8de9b9195d7)

### 1. Your Databases

The **databases of your applications** serve as the primary source, holding various data like user information, roles, permissions, and entity relationships. These databases contain the **authorization-related data** that will be captured and synchronized.

### 2. Debezium

**Debezium** is responsible for **Change Data Capture (CDC)**, monitoring your application's database for any changes (e.g., in tables like `repository` and `organization`). Debezium detects these changes in real-time and **publishes them to Kafka**, ensuring that any modifications are captured without impacting database performance.

### 3. Kafka

**Kafka** acts as the **message broker**, receiving change events from Debezium. It queues these events and passes them on for further processing.

### 4. Permify Sync Service

The **Permify Sync Service** is responsible for filtering the incoming data from Kafka. It **identifies authorization-related changes** (such as role assignments or permission updates) and **synchronizes** this data with Permify. Only relevant data is extracted and passed on.

- **Role**: Filters and processes the events from Kafka to focus on authorization data.
- **Integration**: Ensures that only the necessary authorization changes are synced.

### 5. Permify

**Permify** is an open-source authorization service. It applies authorization policies and ensures real-time enforcement of roles, permissions, and relationships based on the data synced from your databases.

- **Role**: Provides a centralized service to manage and enforce authorization policies.
- **Functionality**: Maintains up-to-date access controls, permissions, and entity relationships within your system, based on predefined policies.

## Permify Sync Configuration

You can use the configuration file, which maps changes in database tables to Permify entities and their relationships.

By syncing the database changes with Permify, you ensure that any updates to the authorization data are handled in real time.

To give an example of how the configuration works, let's examine the following schema.

### Example Permify Schema

The **Permify Schema** defines the entities, relationships, and permissions that are relevant to your application. In this example:

- **Entities** like `user`, `organization`, and `repository` are defined.
- Relationships between entities, such as `admin` or `member` roles in an organization, are established.
- **Permissions** are specified based on relationships or attributes. For example, a repository can be viewed if it is public (`is_public`), and only admins or owners can edit it.

```yaml
entity user {}

entity organization {
relation admin @user
relation member @user
}

entity repository {
relation parent @organization
relation owner @user

attribute is_public boolean

permission view = is_public
permission edit = parent.admin or owner
permission delete = owner
}

```

### Postgres Connector Example Config

In this example, the **Postgres Connector** is configured to track changes in the `repository` and `organization` tables:

- **Entities**: Each table is mapped to a Permify entity (e.g., `repository`, `organization`).
- **Relationships**: Columns like `owner_id` or `organization_id` in the database are mapped to relationships like `owner` or `parent` in Permify.
- **Attributes**: Columns such as `is_public` are mapped to entity attributes.
- **Logging and Retry Policy**: This configuration includes logging at the `error` level and a retry policy to handle connection issues with an exponential backoff strategy.
- **Recovery**: Backups are scheduled every 12 hours and stored in an S3 bucket.

```yaml
# my_postgres_instance.yaml

connector: postgres
metadata:
name: my_postgres_instance
tables:
- name: repository
permify_entity: repository
primary_key: id
relationships:
- column: owner_id
column_type: uint64
permify_reference: owner
- column: organization_id
column_type: uint64
permify_reference: parent
attributes:
- column: is_public
column_type: boolean
permify_reference: is_public
- name: organization
permify_entity: organization
primary_key: id
relationships:
- column: user_id
column_type: uint64
join_table: organization_admins
join_column: organization_id
join_column_type: uint64
permify_reference: admin
- column: user_id
column_type: uint64
join_table: organization_members
join_column: organization_id
join_column_type: uint64
permify_reference: member
logger:
enabled: true
level: error
retry_policy:
max_retries: 5
backoff_strategy: exponential
backoff_interval: 2s
recovery:
backup:
enabled: true
schedule: "0 */12 * * *"
location: "s3://backup-bucket/my-postgres-instance-backups"

```

This setup ensures a seamless sync between your PostgreSQL database and Permify for managing authorization logic efficiently.

The **Permify Sync Service** is applied using a resource configuration, like the one above, through a CLI tool. You can apply this service using the following command:

```bash
permfiy-sync apply -f ./my_postgres_instance.yaml
```

This command ensures that the sync service is properly configured with your database.
73 changes: 73 additions & 0 deletions docs/integrations/metrics/jaeger.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: Jaeger
---

Here’s a step-by-step guide to set up [Jaeger](https://www.jaegertracing.io/) for export Permify traces.

### Step 1: Install Jaeger in Your Environment

You can either run Jaeger via Docker or install it locally on your machine. Here's how you can do it with Docker:

```bash
docker run -d --name jaeger \
-e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
-p 5775:5775/udp \
-p 6831:6831/udp \
-p 6832:6832/udp \
-p 5778:5778 \
-p 16686:16686 \
-p 14268:14268 \
-p 14250:14250 \
-p 9411:9411 \
jaegertracing/all-in-one:1.32
```

This command runs Jaeger’s all-in-one image with all the services (agent, collector, query service, and UI) on default ports. You can access the Jaeger UI at `http://localhost:16686`.

### Step 2: Configure Permify for Jaeger Tracing

In your configuration file, you will set up Jaeger as the exporter and provide the endpoint for exporting the traces.

Here’s an example configuration:

```yaml
tracer:
exporter: jaeger
endpoint: http://localhost:14268/api/traces
enabled: true
insecure: true
```
Explanation:
- `exporter: jaeger`: Specifies that Jaeger will be used as the exporter.
- `endpoint: http://localhost:14268/api/traces`: This is the default Jaeger collector endpoint.
- `enabled: true`: Turns on the tracing feature.
- `insecure: true`: Enables HTTP instead of HTTPS for exporting traces.

<Info>
Refer to the [Configuration](/setting-up/configuration) section for more details on how to use the configuration file. You can also use environment variables (ENVs) for this.
</Info>

### Step 3: Start Permify with Jaeger Configuration

Make sure your configuration file is loaded when starting Permify, and ensure the Jaeger instance is running.

Permify will now start sending tracing data to the configured Jaeger instance.

### Step 4: View Traces in the Jaeger UI

Once you’ve set everything up and your application starts sending requests, you can access Jaeger’s UI at `http://localhost:16686` and observe the traces related to your authorization system.

### Optional: Use Secure Connection

If you want to use HTTPS for secure communication with Jaeger, you need to set up the `insecure` option to `false` and provide the appropriate endpoint:

```yaml
tracer:
exporter: jaeger
endpoint: https://your-jaeger-endpoint
enabled: true
insecure: false
```

This sets up Jaeger with the correct security protocols. See more configurations on [Configuration](/setting-up/configuration) section
91 changes: 91 additions & 0 deletions docs/integrations/metrics/otlp.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: OpenTelemetry
---

Here’s the step-by-step guide to set up [OpenTelemetry](https://opentelemetry.io/) for export Permify traces, metrics and logs.

### Step 1: Install OpenTelemetry in Your Environment

To set up OpenTelemetry, you’ll need to install the OpenTelemetry Collector. You can run it via Docker using the following command:

```bash
docker run --rm -it -p 4317:4317 -p 4318:4318 otel/opentelemetry-collector:latest
```

This command runs the OpenTelemetry Collector, exposing the necessary ports for receiving telemetry data.

### Step 2: Configure Permify for OpenTelemetry Tracing

In your configuration file, set up OpenTelemetry as the exporter and provide the endpoint for exporting traces.

Here’s an example configuration:

```yaml
tracer:
exporter: otlp
endpoint: http://localhost:4317
enabled: true
insecure: true
```
Explanation:
- `exporter: otlp`: Specifies that OpenTelemetry (OTLP) will be used as the exporter.
- `endpoint: http://localhost:4317`: This is the default OpenTelemetry Collector endpoint for receiving traces.
- `enabled: true`: Enables the tracing feature.
- `insecure: true`: Uses HTTP instead of HTTPS for exporting traces.

<Info>
Refer to the [Configuration](/setting-up/configuration) section for more details on how to use the configuration file. You can also use environment variables (ENVs) for this.
</Info>

### Step 3: Start Permify with OpenTelemetry Configuration

Ensure that your configuration file is loaded when starting Permify, and confirm that the OpenTelemetry Collector is running. Permify will now start sending trace data to the OpenTelemetry Collector.

### Step 4: View Traces in Your Chosen UI

Once the setup is complete and your application begins sending traces, you can view the trace data in a backend such as Jaeger, Zipkin, or any system supported by the OpenTelemetry Collector.

### Meter Configuration

We also have a meter configuration section in the Permify configuration file, which allows you to use the OpenTelemetry Collector to export metric data to other sources, such as ClickHouse.

**Definition:**

Configuration for observing metrics like check count, cache check count as well as system details such as Permify version, hostname, OS, and architecture.

**Structure:**

```yaml
meter:
exporter: otlp
endpoint: http://localhost:4317
enabled: true
insecure: true
urlpath: /v1/metrics
```

| Required | Argument | Default | Description |
|----------|----------|---------|--------------------------------------------------------------|
| [x] | exporter | otlp | [otlp](https://opentelemetry.io/docs/collector/) is the default. |
| [x] | endpoint | - | The export URI for metric observation. |
| [ ] | enabled | true | Switch option for meter tracing. |
| [ ] | urlpath | /v1/metrics | Allows you to override the default URL path for metrics export. |
| [ ] | insecure | false | Whether to use HTTP instead of HTTPS for exporting metrics. |

### Example: Exporting Metrics to OpenTelemetry Collector

To export metrics to the **OpenTelemetry Collector**, you can configure your `meter` section like this:

```yaml
meter:
enabled: true
exporter: otlp
endpoint: collector:4317
insecure: true
urlpath: /v1/metrics
```

In this setup:
- `endpoint: http://collector:4317`: This is the URL of your OpenTelemetry Collector instance for receiving metrics data.
- `urlpath: /v1/metrics`: The URL path used for sending the metrics (you can override it as needed).
65 changes: 65 additions & 0 deletions docs/integrations/metrics/signoz.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: SigNoz
---

Here’s a step-by-step guide to set up [SigNoz](https://signoz.io/) for export Permify traces.

### Step 1: Install SigNoz in Your Environment

You can install SigNoz via Docker. Run the following command to set up SigNoz:

```bash
git clone https://github.com/SigNoz/signoz.git
cd signoz/deploy/
./install.sh
```

Once the installation is complete, you can access the SigNoz dashboard at `http://localhost:3301`.

### Step 2: Configure Permify for SigNoz Tracing

In your configuration file, set up SigNoz as the exporter and provide the appropriate endpoint for exporting the traces.

Here’s an example configuration:

```yaml
tracer:
exporter: signoz
endpoint: http://localhost:4317
enabled: true
insecure: true
```
Explanation:
- `exporter: signoz`: Specifies that SigNoz will be used as the exporter.
- `endpoint: http://localhost:4317`: This is the default SigNoz OpenTelemetry Collector endpoint.
- `enabled: true`: Turns on the tracing feature.
- `insecure: true`: Enables HTTP instead of HTTPS for exporting traces.

<Info>
Refer to the [Configuration](/setting-up/configuration) section for more details on how to use the configuration file. You can also use environment variables (ENVs) for this.
</Info>

### Step 3: Start Permify with SigNoz Configuration

Ensure your configuration file is correctly loaded when starting Permify, and verify that the SigNoz instance is running. Permify will now start sending tracing data to SigNoz.

### Step 4: View Traces in the SigNoz Dashboard

Once the setup is complete and your application starts sending requests, you can access the SigNoz dashboard at `http://localhost:3301` to observe the traces related to your authorization system.

### Optional: Use Secure Connection

If you need to use HTTPS for secure communication with SigNoz, adjust the configuration as follows:

```yaml
tracer:
exporter: signoz
endpoint: https://your-signoz-endpoint
enabled: true
insecure: false
```

This sets up a secure connection for exporting traces to SigNoz.

See more configurations on [Configuration](/setting-up/configuration) section.
Loading

0 comments on commit 6b11c4b

Please sign in to comment.