Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move MOLT docs into molt directory; add redirects #18628

Merged
merged 4 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions src/current/_data/redirects.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,6 @@
- destination: advisories/index.md
sources: ['advisories/advisories.md']

# Pages undergoing maintenance

# Renamed pages

- destination: admin-ui-overview.md
Expand Down Expand Up @@ -248,6 +246,12 @@
- 'learn-about-request-units.md'
versions: ['cockroachcloud']

- destination: molt/live-migration-service.md
sources:
- v23.1/live-migration-service.md
- v23.2/live-migration-service.md
- v24.1/live-migration-service.md

- destination: logging-overview.md
sources: ['debug-and-error-logs.md']
versions: ['v21.1']
Expand All @@ -269,6 +273,17 @@
- destination: migration-overview.md
sources: ['import-data.md']
versions: ['v2.1', 'v19.1', 'v19.2', 'v20.1', 'v20.2', 'v21.1']

- destination: molt/molt-fetch.md
taroface marked this conversation as resolved.
Show resolved Hide resolved
sources:
- v23.2/molt-fetch.md
- v24.1/molt-fetch.md

- destination: molt/molt-verify.md
sources:
- v23.1/molt-verify.md
- v23.2/molt-verify.md
- v24.1/molt-verify.md

- destination: node-shutdown.md
sources: ['cockroach-quit.md']
Expand Down
4 changes: 2 additions & 2 deletions src/current/_includes/v23.1/sidebar-data/migrate.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@
{
"title": "Verify",
"urls": [
"/${VERSION}/molt-verify.html"
"/molt/molt-verify.html"
]
},
{
"title": "Live Migration Service",
"urls": [
"/${VERSION}/live-migration-service.html"
"/molt/live-migration-service.html"
]
}
]
Expand Down
6 changes: 3 additions & 3 deletions src/current/_includes/v23.2/sidebar-data/migrate.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,19 @@
{
"title": "Fetch",
"urls": [
"/${VERSION}/molt-fetch.html"
"/molt/molt-fetch.html"
]
},
{
"title": "Verify",
"urls": [
"/${VERSION}/molt-verify.html"
"/molt/molt-verify.html"
]
},
{
"title": "Live Migration Service",
"urls": [
"/${VERSION}/live-migration-service.html"
"/molt/live-migration-service.html"
]
}
]
Expand Down
6 changes: 3 additions & 3 deletions src/current/_includes/v24.1/sidebar-data/migrate.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,19 @@
{
"title": "Fetch",
"urls": [
"/${VERSION}/molt-fetch.html"
"/molt/molt-fetch.html"
]
},
{
"title": "Verify",
"urls": [
"/${VERSION}/molt-verify.html"
"/molt/molt-verify.html"
]
},
{
"title": "Live Migration Service",
"urls": [
"/${VERSION}/live-migration-service.html"
"/molt/live-migration-service.html"
]
}
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ docs_area: migrate
{% include feature-phases/preview.md %}
{{site.data.alerts.end}}

MOLT LMS (Live Migration Service) is used during a [live migration]({% link {{ page.version.version }}/migration-overview.md %}#minimal-downtime) to CockroachDB.
MOLT Live Migration Service (LMS) is used during a [live migration]({% link {{site.current_cloud_version}}/migration-overview.md %}#minimal-downtime) to CockroachDB.

The LMS is a self-hosted, horizontally scalable proxy that routes traffic between an application, a source database, and a target CockroachDB database. You use the LMS to control which database, as the "source of truth", is serving reads and writes to an application. You can optionally configure the LMS to [shadow production traffic](#shadowing-modes) from the source database and validate the query results on CockroachDB. When you have sufficiently tested your application and are confident with its consistency and performance on CockroachDB, you use the LMS to [perform the cutover](#perform-a-cutover) to CockroachDB.

Expand All @@ -18,9 +18,9 @@ MOLT LMS is self-hosted on [Kubernetes](https://kubernetes.io/) and [configured
- A number of proxy [instances](#lms-instances) (running in separate Kubernetes pods) across which application traffic is distributed and routed to the source and target databases.
- An "orchestrator" service (running in a single Kubernetes pod) that coordinates the proxy instances and sends the cutover commands.

This page describes how to [install](#installation), [configure](#configuration), [secure](#security), and [use the LMS](#molt-lms-cli) to perform a live migration. {% comment %}For more information, see [Migration Strategy: Live Migration]({% link {{ page.version.version }}/migration-strategy-live-migration.md %}).{% endcomment %}
This page describes how to [install](#installation), [configure](#configuration), [secure](#security), and [use the LMS](#molt-lms-cli) to perform a live migration. {% comment %}For more information, see [Migration Strategy: Live Migration]({% link {{site.current_cloud_version}}/migration-strategy-live-migration.md %}).{% endcomment %}

For a demo of the Live Migration Service in action, watch the following video:
Watch a demo of the Live Migration Service in action:

{% include_cached youtube.html video_id="HA8ec9e_a-s" %}

Expand All @@ -37,8 +37,8 @@ For a demo of the Live Migration Service in action, watch the following video:

#### Supported database technologies

- [PostgreSQL]({% link {{ page.version.version }}/migrate-from-postgres.md %}) (source)
- [MySQL]({% link {{ page.version.version }}/migrate-from-mysql.md %}) (source)
- [PostgreSQL]({% link {{site.current_cloud_version}}/migrate-from-postgres.md %}) (source)
- [MySQL]({% link {{site.current_cloud_version}}/migrate-from-mysql.md %}) (source)
- CockroachDB (source and target)

## Installation
Expand Down Expand Up @@ -261,7 +261,7 @@ spec:
The connection strings are specified with the following keys inside `config.json`:

- `INIT_SOURCE`: External connection string for the source database, including the paths to your client certificate and keys.
- `INIT_TARGET`: External [connection string for the CockroachDB database]({% link {{ page.version.version }}/connection-parameters.md %}#connect-using-a-url), including the paths to your client certificate and keys.
- `INIT_TARGET`: External [connection string for the CockroachDB database]({% link {{site.current_cloud_version}}/connection-parameters.md %}#connect-using-a-url), including the paths to your client certificate and keys.

The remote secret `lms-secret` will contain the full connection strings and paths, such that the `config.json` keys resolve to:

Expand Down Expand Up @@ -326,7 +326,7 @@ The connection strings are specified with the following keys inside `config.json
If you named the release `lms` during [installation](#installation), exclude `{releasename}-` from the LMS connection string.
{{site.data.alerts.end}}

- `CRDB_URL`: External [connection string for the CockroachDB database]({% link {{ page.version.version }}/connection-parameters.md %}#connect-using-a-url), including the paths to your client certificate and keys.
- `CRDB_URL`: External [connection string for the CockroachDB database]({% link {{site.current_cloud_version}}/connection-parameters.md %}#connect-using-a-url), including the paths to your client certificate and keys.

The remote secret `orch-secret` will contain the full connection strings, such that the `config.json` keys resolve to:

Expand Down Expand Up @@ -644,16 +644,16 @@ You can use this mode to perform an [immediate cutover](#immediate-cutover).

### Consistent cutover

A consistent cutover maintains data consistency with [minimal downtime]({% link {{ page.version.version }}/migration-overview.md %}#minimal-downtime). The goal of consistent cutover is to stop application traffic long enough for replication to catch up and ensure that the cutover achieves consistency across the two databases.
A consistent cutover maintains data consistency with [minimal downtime]({% link {{site.current_cloud_version}}/migration-overview.md %}#minimal-downtime). The goal of consistent cutover is to stop application traffic long enough for replication to catch up and ensure that the cutover achieves consistency across the two databases.

When using the LMS, consistent cutover is handled using the [`molt-lms-cli`](#molt-lms-cli) commands `cutover consistent begin` and `cutover consistent commit`, during which application requests are queued and will be responded to after cutover. This delay in response time is related to the maximum duration of any transactions and queries that need to complete, and the time it takes for replication to catch up from the source to the target database.

{% comment %}
For more information about the consistent cutover approach, see [Migration Strategy: Live Migration]({% link {{ page.version.version }}/migration-strategy-live-migration.md %}).
For more information about the consistent cutover approach, see [Migration Strategy: Live Migration]({% link {{site.current_cloud_version}}/migration-strategy-live-migration.md %}).
{% endcomment %}

{{site.data.alerts.callout_info}}
These steps assume you have already followed the overall steps to [prepare for migration]({% link {{ page.version.version }}/migration-overview.md %}#prepare-for-migration). In particular, [update your schema and application queries]({% link {{ page.version.version }}/migration-overview.md %}#update-the-schema-and-queries) to work with CockroachDB.
These steps assume you have already followed the overall steps to [prepare for migration]({% link {{site.current_cloud_version}}/migration-overview.md %}#prepare-for-migration). In particular, [update your schema and application queries]({% link {{site.current_cloud_version}}/migration-overview.md %}#update-the-schema-and-queries) to work with CockroachDB.
{{site.data.alerts.end}}

To perform a consistent cutover with the LMS:
Expand All @@ -672,7 +672,7 @@ To perform a consistent cutover with the LMS:

1. Send application requests to the LMS, which routes the traffic to the source database. The source database is designated the source of truth.

1. Use [MOLT Verify]({% link {{ page.version.version }}/molt-verify.md %}) to validate that the replicated data on CockroachDB is consistent with the source of truth.
1. Use [MOLT Verify]({% link molt/molt-verify.md %}) to validate that the replicated data on CockroachDB is consistent with the source of truth.

1. Begin the consistent cutover. **Requests are now queued in the LMS**, including queries from existing connections and new connection requests to the LMS:

Expand Down Expand Up @@ -720,7 +720,7 @@ To perform a consistent cutover with the LMS:

To verify that CockroachDB is now the source of truth, you can run `molt-lms-cli status`.

1. Again, use [MOLT Verify]({% link {{ page.version.version }}/molt-verify.md %}) to validate that the data on the source database and CockroachDB are consistent.
1. Again, use [MOLT Verify]({% link molt/molt-verify.md %}) to validate that the data on the source database and CockroachDB are consistent.

If any problems arise during a consistent cutover:

Expand All @@ -740,14 +740,14 @@ If any problems arise during a consistent cutover:
{% comment %}
### Immediate cutover

An immediate cutover can potentially [reduce downtime to zero]({% link {{ page.version.version }}/migration-overview.md %}#minimal-downtime), at the likely risk of introducing data inconsistencies between the source and target databases. The LMS is configured to dual write to the source and target databases, while the [`molt-lms-cli`](#molt-lms-cli) command `cutover immediate` initiates cutover.
An immediate cutover can potentially [reduce downtime to zero]({% link {{site.current_cloud_version}}/migration-overview.md %}#minimal-downtime), at the likely risk of introducing data inconsistencies between the source and target databases. The LMS is configured to dual write to the source and target databases, while the [`molt-lms-cli`](#molt-lms-cli) command `cutover immediate` initiates cutover.

For more information about the immediate cutover approach, see [Migration Strategy: Live Migration]({% link {{ page.version.version }}/migration-strategy-live-migration.md %}).
For more information about the immediate cutover approach, see [Migration Strategy: Live Migration]({% link {{site.current_cloud_version}}/migration-strategy-live-migration.md %}).

To perform an immediate cutover with the LMS:

{{site.data.alerts.callout_info}}
These steps assume you have already followed the overall steps to [prepare for migration]({% link {{ page.version.version }}/migration-overview.md %}#prepare-for-migration). In particular, [update your schema and application queries]({% link {{ page.version.version }}/migration-overview.md %}#update-the-schema-and-queries) to work with CockroachDB.
These steps assume you have already followed the overall steps to [prepare for migration]({% link {{site.current_cloud_version}}/migration-overview.md %}#prepare-for-migration). In particular, [update your schema and application queries]({% link {{site.current_cloud_version}}/migration-overview.md %}#update-the-schema-and-queries) to work with CockroachDB.
{{site.data.alerts.end}}

1. [Configure the LMS](#configuration) with your deployment details, and follow our [security recommendations](#security).
Expand All @@ -756,9 +756,9 @@ These steps assume you have already followed the overall steps to [prepare for m

1. Send application requests to the LMS, which routes the traffic to the source database and to CockroachDB. The source database is designated the source of truth.

1. Use [MOLT Verify]({% link {{ page.version.version }}/molt-verify.md %}) to validate that the replicated data on CockroachDB is consistent with the source of truth.
1. Use [MOLT Verify]({% link molt/molt-verify.md %}) to validate that the replicated data on CockroachDB is consistent with the source of truth.

To ensure data integrity, shadowing must be enabled for a sufficient duration with a low error rate. All LMS instances should have been continuously shadowing your workload for the past **seven days** at minimum, with only transient inconsistencies caused by events such as [transaction retry errors]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}). The longer shadowing has been enabled, the better this allows you to evaluate consistency.
To ensure data integrity, shadowing must be enabled for a sufficient duration with a low error rate. All LMS instances should have been continuously shadowing your workload for the past **seven days** at minimum, with only transient inconsistencies caused by events such as [transaction retry errors]({% link {{site.current_cloud_version}}/transaction-retry-error-reference.md %}). The longer shadowing has been enabled, the better this allows you to evaluate consistency.

1. Once nearly all data from the source database is replicated to CockroachDB (for example, with a <1 second delay or <1000 rows), initiate the cutover:

Expand All @@ -769,7 +769,7 @@ These steps assume you have already followed the overall steps to [prepare for m

This command tells the LMS to switch the source of truth to CockroachDB. Application traffic is immediately directed to CockroachDB.

1. Any writes that were made during the cutover will have been missed on CockroachDB. Use [MOLT Verify]({% link {{ page.version.version }}/molt-verify.md %}) to identify the inconsistencies. These will need to be manually reconciled.
1. Any writes that were made during the cutover will have been missed on CockroachDB. Use [MOLT Verify]({% link molt/molt-verify.md %}) to identify the inconsistencies. These will need to be manually reconciled.
{% endcomment %}

### Monitor cutover
Expand Down Expand Up @@ -884,9 +884,9 @@ Individual LMS Proxy Status.

## See also

- [Migration Overview]({% link {{ page.version.version }}/migration-overview.md %})
{% comment %}- [Migration Strategy: Live Migration]({% link {{ page.version.version }}/migration-strategy-live-migration.md %}){% endcomment %}
- [Migration Overview]({% link {{site.current_cloud_version}}/migration-overview.md %})
{% comment %}- [Migration Strategy: Live Migration]({% link {{site.current_cloud_version}}/migration-strategy-live-migration.md %}){% endcomment %}
- [Use the Schema Conversion Tool](https://www.cockroachlabs.com/docs/cockroachcloud/migrations-page)
- [MOLT Verify]({% link {{ page.version.version }}/molt-verify.md %})
- [Migrate from PostgreSQL]({% link {{ page.version.version }}/migrate-from-postgres.md %})
- [Migrate from MySQL]({% link {{ page.version.version }}/migrate-from-mysql.md %})
- [MOLT Verify]({% link molt/molt-verify.md %})
- [Migrate from PostgreSQL]({% link {{site.current_cloud_version}}/migrate-from-postgres.md %})
- [Migrate from MySQL]({% link {{site.current_cloud_version}}/migrate-from-mysql.md %})
Loading
Loading