Skip to content

Commit b16d887

Browse files
katmaybkatmayb
authored
Add schema_locked storage parameter for changefeed performance improvement (#18763)
1 parent 82e2e1f commit b16d887

File tree

8 files changed

+76
-0
lines changed

8 files changed

+76
-0
lines changed

‎src/current/_includes/v23.1/misc/table-storage-parameters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
| Parameter name | Description | Data type | Default value |
22
|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------|
33
| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` |
4+
| <span class="version-tag">New in v23.1.15:</span> `schema_locked` | Disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` |
45
| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` |
56
| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 |
67
| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 |

‎src/current/_includes/v23.2/misc/table-storage-parameters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
| Parameter name | Description | Data type | Default value |
22
|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------|
33
| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` |
4+
| <span class="version-tag">New in v23.2.1:</span> `schema_locked` | Disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` |
45
| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` |
56
| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 |
67
| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 |

‎src/current/_includes/v24.1/misc/table-storage-parameters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
| Parameter name | Description | Data type | Default value |
22
|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------|
33
| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` |
4+
| `schema_locked` | Disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` |
45
| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` |
56
| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 |
67
| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 |

‎src/current/_includes/v24.2/misc/table-storage-parameters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
| Parameter name | Description | Data type | Default value |
22
|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------|
33
| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` |
4+
| `schema_locked` | Disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` |
45
| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` |
56
| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 |
67
| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 |

‎src/current/v23.1/create-changefeed.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -335,6 +335,24 @@ CREATE CHANGEFEED FOR TABLE table_name INTO 'external://kafka_sink'
335335
WITH resolved;
336336
~~~
337337

338+
### Disallow schema changes on tables to improve changefeed performance
339+
340+
{% include_cached new-in.html version="v23.1.15" %} Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). You can lock the table before creating a changefeed or while a changefeed is running, which will enable the performance improvement for changefeeds watching the particular table.
341+
342+
Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}) statement:
343+
344+
{% include_cached copy-clipboard.html %}
345+
~~~ sql
346+
ALTER TABLE watched_table SET (schema_locked = true);
347+
~~~
348+
349+
While `schema_locked` is enabled on a table, attempted schema changes on the table will be rejected and an error returned. If you need to run a schema change on the locked table, unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked = false`, but it will not benefit from the performance optimization.
350+
351+
{% include_cached copy-clipboard.html %}
352+
~~~ sql
353+
ALTER TABLE watched_table SET (schema_locked = false);
354+
~~~
355+
338356
### Manage a changefeed
339357

340358
For {{ site.data.products.enterprise }} changefeeds, use [`SHOW CHANGEFEED JOBS`]({% link {{ page.version.version }}/show-jobs.md %}) to check the status of your changefeed jobs:

‎src/current/v23.2/create-changefeed.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -341,6 +341,24 @@ CREATE CHANGEFEED FOR TABLE table_name INTO 'external://kafka_sink'
341341
WITH resolved;
342342
~~~
343343

344+
### Disallow schema changes on tables to improve changefeed performance
345+
346+
{% include_cached new-in.html version="v23.2.1" %} Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). You can lock the table before creating a changefeed or while a changefeed is running, which will enable the performance improvement for changefeeds watching the particular table.
347+
348+
Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}) statement:
349+
350+
{% include_cached copy-clipboard.html %}
351+
~~~ sql
352+
ALTER TABLE watched_table SET (schema_locked = true);
353+
~~~
354+
355+
While `schema_locked` is enabled on a table, attempted schema changes on the table will be rejected and an error returned. If you need to run a schema change on the locked table, unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked = false`, but it will not benefit from the performance optimization.
356+
357+
{% include_cached copy-clipboard.html %}
358+
~~~ sql
359+
ALTER TABLE watched_table SET (schema_locked = false);
360+
~~~
361+
344362
### Manage a changefeed
345363

346364
For {{ site.data.products.enterprise }} changefeeds, use [`SHOW CHANGEFEED JOBS`]({% link {{ page.version.version }}/show-jobs.md %}) to check the status of your changefeed jobs:

‎src/current/v24.1/create-changefeed.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -365,6 +365,24 @@ CREATE CHANGEFEED FOR TABLE table_name INTO 'external://kafka_sink'
365365

366366
For guidance on how to filter changefeed messages to emit [row-level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) deletes only, refer to [Change Data Capture Queries]({% link {{ page.version.version }}/cdc-queries.md %}#reference-ttl-in-a-cdc-query).
367367

368+
### Disallow schema changes on tables to improve changefeed performance
369+
370+
Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). You can lock the table before creating a changefeed or while a changefeed is running, which will enable the performance improvement for changefeeds watching the particular table.
371+
372+
Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}) statement:
373+
374+
{% include_cached copy-clipboard.html %}
375+
~~~ sql
376+
ALTER TABLE watched_table SET (schema_locked = true);
377+
~~~
378+
379+
While `schema_locked` is enabled on a table, attempted schema changes on the table will be rejected and an error returned. If you need to run a schema change on the locked table, unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked = false`, but it will not benefit from the performance optimization.
380+
381+
{% include_cached copy-clipboard.html %}
382+
~~~ sql
383+
ALTER TABLE watched_table SET (schema_locked = false);
384+
~~~
385+
368386
### Manage a changefeed
369387

370388
For {{ site.data.products.enterprise }} changefeeds, use [`SHOW CHANGEFEED JOBS`]({% link {{ page.version.version }}/show-jobs.md %}) to check the status of your changefeed jobs:

‎src/current/v24.2/create-changefeed.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,24 @@ CREATE CHANGEFEED FOR TABLE table_name INTO 'external://kafka_sink'
366366

367367
For guidance on how to filter changefeed messages to emit [row-level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) deletes only, refer to [Change Data Capture Queries]({% link {{ page.version.version }}/cdc-queries.md %}#reference-ttl-in-a-cdc-query).
368368

369+
### Disallow schema changes on tables to improve changefeed performance
370+
371+
Use the `schema_locked` [storage parameter]({% link {{ page.version.version }}/with-storage-parameter.md %}) to disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on a watched table, which helps to decrease the latency between a write committing to a table and it emitting to the [changefeed's sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). You can lock the table before creating a changefeed or while a changefeed is running, which will enable the performance improvement for changefeeds watching the particular table.
372+
373+
Enable `schema_locked` on the watched table with the [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}) statement:
374+
375+
{% include_cached copy-clipboard.html %}
376+
~~~ sql
377+
ALTER TABLE watched_table SET (schema_locked = true);
378+
~~~
379+
380+
While `schema_locked` is enabled on a table, attempted schema changes on the table will be rejected and an error returned. If you need to run a schema change on the locked table, unlock the table with `schema_locked = false`, complete the schema change, and then lock the table again with `schema_locked = true`. The changefeed will run as normal while `schema_locked = false`, but it will not benefit from the performance optimization.
381+
382+
{% include_cached copy-clipboard.html %}
383+
~~~ sql
384+
ALTER TABLE watched_table SET (schema_locked = false);
385+
~~~
386+
369387
### Manage a changefeed
370388

371389
For {{ site.data.products.enterprise }} changefeeds, use [`SHOW CHANGEFEED JOBS`]({% link {{ page.version.version }}/show-jobs.md %}) to check the status of your changefeed jobs:

0 commit comments

Comments
 (0)