diff --git a/contents/docs/self-host/configure/environment-variables.md b/contents/docs/self-host/configure/environment-variables.md index c40476a130fe..1679b8d144e2 100644 --- a/contents/docs/self-host/configure/environment-variables.md +++ b/contents/docs/self-host/configure/environment-variables.md @@ -33,11 +33,14 @@ Some variables here are default Django variables. This [Django Docs page](https: | `DATABASE_URL`| [Database URL](https://github.com/jacobian/dj-database-url#url-schema) pointing to your PostgreSQL instance. | `postgres://localhost:5432/posthog` if PostHog is running in DEBUG or TEST mode, must be specified otherwise. | `DEBUG_QUERIES`| Whether debugging queries (ClickHouse) is enabled in the Command Palette.| `False` | `DEBUG` | Determines if PostHog should run in [DEBUG mode](https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-DEBUG). You can set this to a truthy value when developing, but disable this in production! | `False` | +| `CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS` | If set, disables using ProtoBuf schemas for kafka communication. Needs to be set when using an external ClickHouse service provider during initial deploy. | `False` | `DISABLE_PAID_FEATURE_SHOWCASING`| Whether any showcasing of a paid feature should be disabled. Useful if running a free open source version of PostHog and are not interested in premium functionality. | `False` | `DISABLE_SECURE_SSL_REDIRECT` | Disables automatic redirect from port 80 (HTTP) to port 443 (HTTPS). | `False` | `GITHUB_TOKEN`| GitHub personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos | `None` | `GITLAB_TOKEN`| GitLab personal access token, used to prevent rate limiting when using plugins and to allow installation of plugins from private repos | `None` | `JS_URL` | URL used by Webpack for loading external resources like images and files. | `http://localhost:8234` if PostHog is running in DEBUG mode, must be specified otherwise. +| `KAFKA_URL` | Address used by the application to contact kafka | `kafka://kafka` +| `KAFKA_URL_FOR_CLICKHOUSE` | Address used by ClickHouse to read from kafka. Falls back to `KAFKA_URL` | `None` | `MATERIALIZE_COLUMNS_ANALYSIS_PERIOD_HOURS` | Diagnostic for what columns to materialize | `168` | `MATERIALIZE_COLUMNS_BACKFILL_PERIOD_DAYS` | How far back backfill materialized columns | `90` | `MATERIALIZE_COLUMNS_MAX_AT_ONCE` | How many columns to materialize at once | `10` diff --git a/contents/docs/self-host/configure/using-altinity-cloud.md b/contents/docs/self-host/configure/using-altinity-cloud.md new file mode 100644 index 000000000000..ffe969bb348e --- /dev/null +++ b/contents/docs/self-host/configure/using-altinity-cloud.md @@ -0,0 +1,137 @@ +--- +title: Deploying ClickHouse using Altinity.Cloud +sidebar: Docs +showTitle: true +--- + +This document outlines how to deploy PostHog using Altinity Cloud ClickHouse clusters. + +## Prerequisites + +- Altinity.Cloud ClickHouse cluster: + - Minimum ClickHouse version: 21.8.13 + - Single shard and no data replication + - No dashes (`-`) in cluster name +- PostHog helm chart version >= 16.1.1 +- PostHog version >= 1.33.0 + +## Deployment instructions + +PostHog uses Kafka to send data from the app to ClickHouse. For that reason, Kafka needs to be accessible to ClickHouse during deployment. + +### Deploying using external Kafka + +```yaml +env: + - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS + value: "1" + +kafka: + enabled: false + +externalKafka: + brokers: + - "broker-1.posthog.kafka.us-east-1.amazonaws.com:9094" + - "broker-2.posthog.kafka.us-east-1.amazonaws.com:9094" + - "broker-3.posthog.kafka.us-east-1.amazonaws.com:9094" + +clickhouse: + enabled: false + +externalClickhouse: + host: "somecluster.demo.altinity.cloud" + user: "admin" + password: "password" + cluster: "clustername" + secure: true +``` + +Read more about how to configure external Kafka in the chart [in our deployment documentation](https://posthog.com/docs/self-host/deploy/configuration#kafka). + +### Using internal Kafka + +To deploy using a version of Kafka managed by the PostHog Helm chart, follow these three steps: + +1. [Deploy your Helm chart](/docs/self-host) initially with the following values.yaml: + +```yaml +kafka: + enabled: true + externalAccess: + enabled: true + service: + type: LoadBalancer + ports: + external: 9094 + autoDiscovery: + enabled: true + serviceAccount: + create: true + rbac: + create: true + + +clickhouse: + enabled: false + +redis: + enabled: false + +postgresql: + enabled: false + +pgbouncer: + enabled: false + +plugins: + enabled: false + +worker: + enabled: false + +web: + enabled: false + +events: + enabled: false + +migrate: + enabled: false +``` + +2. Get the external Kafka IP via `kubectl get svc -n posthog | grep kafka-0-external` + +3. [Deploy PostHog using helm](/docs/self-host) with new values.yaml (fill in placeholder values) + +```yaml +env: + - name: KAFKA_URL_FOR_CLICKHOUSE + value: "kafka://KAFKA_IP:9094" + - name: CLICKHOUSE_DISABLE_EXTERNAL_SCHEMAS + value: "1" + +clickhouse: + enabled: false + +externalClickhouse: + host: "somecluster.demo.altinity.cloud" + user: "admin" + password: "password" + cluster: "clustername" + secure: true + +kafka: + enabled: true + externalAccess: + enabled: true + service: + type: LoadBalancer + ports: + external: 9094 + autoDiscovery: + enabled: true + serviceAccount: + create: true + rbac: + create: true +``` diff --git a/contents/docs/self-host/deploy/configuration.md b/contents/docs/self-host/deploy/configuration.md index e294c2155098..38d3da229f3c 100644 --- a/contents/docs/self-host/deploy/configuration.md +++ b/contents/docs/self-host/deploy/configuration.md @@ -207,6 +207,11 @@ ClickHouse is the datastore system that does the bulk of heavy lifting with rega By default, ClickHouse is installed as a part of the chart, powered by [clickhouse-operator](https://github.com/Altinity/clickhouse-operator/). We are currently working to add the possibility to use an external ClickHouse service (see [issue #279](https://github.com/PostHog/charts-clickhouse/issues/279) for more info). +#### Use an external service +To use an external ClickHouse service, please set `clickhouse.enabled` to `false` and then configure the `externalClickhouse` values. + +Find out how to deploy PostHog using Altinity Cloud [in our deployment configuration docs](/docs/self-host/configure/using-altinity-cloud). + #### Custom settings It's possible to pass custom settings to ClickHouse. This might be needed to e.g. set query time limits or increase max memory usable by clickhouse. diff --git a/contents/marketplace/altinity.md b/contents/marketplace/altinity.md index ac3feba57a0a..9f1537eed431 100644 --- a/contents/marketplace/altinity.md +++ b/contents/marketplace/altinity.md @@ -8,21 +8,21 @@ hideLastUpdated: true Altinity Logo -[Altinity](https://altinity.com) helps enterprises deliver real-time analytics based on ClickHouse anywhere and for any business purpose. Our offerings cover everything needed from project inception to prodution operation. +[Altinity](https://altinity.com) helps enterprises deliver real-time analytics based on ClickHouse anywhere and for any business purpose. Altinity covers everything needed from project inception to production operation. 1. Altinity.Cloud platform offering fully supported and managed ClickHouse clusters in AWS and GCP 2. 24/7 enterprise support for ClickHouse in any environment 3. Training for analytic developers and administrators 4. Altinity Stable builds for ClickHouse -Altinity customers range from startups to Fortune 10 enterprises. +Altinity customers range from startups to Fortune 10 enterprises. ## Services offered ### Support -- Coming soon (likely from PostHog 1.33.0): Altinity.Cloud ClickHouse backends for PostHog users who prefer to use ClickHouse as a hands-off service - get in touch now if you are interested - Enterprise support for users who prefer to operate ClickHouse in self-managed environments +- Altinity.Cloud ClickHouse backends for PostHog users who prefer to use ClickHouse as a hands-off service. Find out more about using Altinity.Cloud to manage your self-hosted instance in [our deployment configuration docs.](/docs/self-host/configure/using-altinity-cloud). ## Contact -[Speak to Altinity](mailto:marketplace+altinity@posthog.com) \ No newline at end of file +[Speak to Altinity](mailto:marketplace+altinity@posthog.com) diff --git a/src/sidebars/sidebars.json b/src/sidebars/sidebars.json index 17b1e59c66b7..0a0d18180721 100644 --- a/src/sidebars/sidebars.json +++ b/src/sidebars/sidebars.json @@ -536,6 +536,10 @@ { "name": "Helm chart configuration", "url": "/docs/self-host/deploy/configuration" + }, + { + "name": "Deploying ClickHouse using Altinity.Cloud", + "url": "/docs/self-host/configure/using-altinity-cloud" } ] },