From b7e424641c53ba596d91a6bf3468969c95f4954d Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Fri, 16 Dec 2022 15:04:52 +0530 Subject: [PATCH 1/8] feat: fix pdfs in the presentations page to be full width Signed-off-by: Manan Gupta --- content/en/docs/resources/presentations.md | 26 +++++++++++----------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/content/en/docs/resources/presentations.md b/content/en/docs/resources/presentations.md index 60002795b..7e5e3ce62 100644 --- a/content/en/docs/resources/presentations.md +++ b/content/en/docs/resources/presentations.md @@ -130,7 +130,7 @@ Lizz van Dijk demonstrates how to migrate from a regular MySQL release to Vitess Lizz van Dijk presents an introduction to Vitess for MySQL users. - + ## QCon 2019 @@ -155,19 +155,19 @@ Vitess was also featured during the [CNCF project updates keynote](https://www.y Sugu Sougoumarane presents an overview of Vitess at Highload in Moscow. -{{< pdf src="/ViewerJS/#../files/2019-sugu-highload.pdf" >}} + ## Utah Kubernetes Meetup 2019 Jiten Vaidya shows how you can extend Vitess to create jurisdiction-aware database clusters. -{{< pdf src="/ViewerJS/#../files/2019-jiten-utah.pdf" >}} + ## CNCF Meetup Paris 2019 Sugu Sougoumarane and Morgan Tocker present a three hour Vitess workshop on Kubernetes. -{{< pdf src="/ViewerJS/#../files/2019-paris-cncf.pdf" >}} + ## Percona Live Europe 2019 @@ -175,14 +175,14 @@ Sugu Sougoumarane and Morgan Tocker present a three hour Vitess workshop on Kube Morgan Tocker talks about his adventures in Vitess, after having come from a MySQL background. -{{< pdf src="/ViewerJS/#../files/2019-morgan-percona-eu.pdf" >}} + **Sharded MySQL on Kubernetes** Sugu Sougoumarane presents an overview of running sharded MySQL on Kubernetes. -{{< pdf src="/ViewerJS/#../files/2019-sugu-percona-eu.pdf" >}} + ## Vitess Meetup 2019 @ Slack HQ @@ -190,14 +190,14 @@ Sugu Sougoumarane presents an overview of running sharded MySQL on Kubernetes. Deepthi Sigireddi shares new features recently introduced in Vitess, and what's on the roadmap moving forward. -{{< pdf src="/ViewerJS/#../files/2019-deepthi-vitess-meetup.pdf" >}} + **Deploying multi-cell Vitess** Rafael Chacon Vivas describes how Vitess is used in Slack. -{{< pdf src="/ViewerJS/#../files/2019-rafael-vitess-meetup.pdf" >}} + **Vitess at Pinterest** @@ -296,7 +296,7 @@ Slack is embarking on a major migration of the MySQL infrastructure at the core Michael Demmer shows us how, at [Percona Live Europe 2017](https://www.percona.com/live/e17/sessions/migrating-to-vitess-at-slack-scale). -{{< pdf src="/ViewerJS/#../files/2017-demmer-percona.pdf" >}} + ## Vitess Deep Dive sessions @@ -308,7 +308,7 @@ Start with session 1 and work your way through the playlist. This series focuses [Sugu](https://github.com/sougou) and [Anthony](https://github.com/enisoc) showed what it looks like to use Vitess now that Keyspace IDs can be completely hidden from the application. They gave a live demo of resharding the Guestbook sample app, which now knows nothing about shards, and explained how new features in VTGate make all of this possible. -{{< pdf src="/ViewerJS/#../files/percona-2016.pdf" >}} + ## CoreOS Meetup, January 2016 @@ -320,20 +320,20 @@ of the Vitess on Kubernetes deployment templates. Please note the Vitess on Kubernetes deployment templates were removed as of February 27, 2020. -{{< pdf src="/ViewerJS/#../files/coreos-meetup-2016-01-27.pdf" >}} + ## Oracle OpenWorld 2015 Vitess team member [Anthony Yeh](https://github.com/enisoc)'s talk at Oracle OpenWorld 2015 focused on what the [Cloud Native Computing](http://cncf.io) paradigm means when applied to MySQL in the cloud. The talk also included a deep dive into [transparent, live resharding](../../reference/features/sharding/), one of the key features of Vitess that makes it well-adapted for a Cloud Native environment. -{{< pdf src="/ViewerJS/#../files/openworld-2015-vitess.pdf" >}} + ## Percona Live 2015 Vitess team member [Anthony Yeh](https://github.com/enisoc)'s talk at Percona Live 2015 provided an overview of Vitess as well as an explanation of how Vitess has evolved to live in a containerized world with Kubernetes and Docker. -{{< pdf src="/ViewerJS/#../files/percona-2015-vitess-and-kubernetes.pdf" >}} + ## Google I/O 2014 - Scaling with Go: YouTube's Vitess From c1ed1a5c9a37a1efe22b354ff9469671a58a1fa2 Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Fri, 16 Dec 2022 15:57:42 +0530 Subject: [PATCH 2/8] feat: fix sharding doc Signed-off-by: Manan Gupta --- content/en/docs/15.0/reference/features/sharding.md | 4 ++-- content/en/docs/16.0/reference/features/sharding.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/15.0/reference/features/sharding.md b/content/en/docs/15.0/reference/features/sharding.md index 6f30e8d6f..0501484be 100644 --- a/content/en/docs/15.0/reference/features/sharding.md +++ b/content/en/docs/15.0/reference/features/sharding.md @@ -11,7 +11,7 @@ Sharding is a method of horizontally partitioning a database to store data acros A keyspace in Vitess can be sharded or unsharded. An unsharded keyspace maps directly to a MySQL database. If sharded, the rows of the keyspace are partitioned into different databases of identical schema. -For example, if an application's "user" keyspace is split into two shards, each shard contains records for approximately half of the application's users. Similarly, each user's information is stored in only one shard. +For example, if an application's "user" keyspace is split into two shards, each shard contains records for approximately half of the application's users. Each single user's information however, is stored in only one shard. Note that sharding is orthogonal to (MySQL) replication. A Vitess shard typically contains one MySQL primary and many MySQL replicas. The primary handles write operations, while replicas handle read-only traffic, batch processing operations, and other tasks. Each MySQL instance within the shard should have the same data, excepting some replication lag. @@ -43,7 +43,7 @@ When building the serving graph for a sharded keyspace, Vitess ensures that each * An empty start value represents the lowest value, and all values are greater than it. * An empty end value represents a value larger than the highest possible value, and all values are strictly lower than it. -Vitess always converts sharding keys to a left-justified binary string for computing a shard. This left-justification makes the right-most zeroes insignificant and optional. Therefore, the value `0x80` is always the middle value for sharding keys. So, in a keyspace with two shards, sharding keys that have a binary value lower than 0x80 are assigned to one shard. Keys with a binary value equal to or higher than 0x80 are assigned to the other shard. +Vitess always converts sharding keys to a left-justified binary string for computing a shard. This left-justification makes the right-most zeroes insignificant and optional. Therefore, the value `0x80` is the middle value for sharding keys. So, in a keyspace with two shards, sharding keys that have a binary value lower than 0x80 are assigned to the first shard. Keys with a binary value equal to or higher than 0x80 are assigned to the other shard. Several sample key ranges are shown below: diff --git a/content/en/docs/16.0/reference/features/sharding.md b/content/en/docs/16.0/reference/features/sharding.md index 6f30e8d6f..0501484be 100644 --- a/content/en/docs/16.0/reference/features/sharding.md +++ b/content/en/docs/16.0/reference/features/sharding.md @@ -11,7 +11,7 @@ Sharding is a method of horizontally partitioning a database to store data acros A keyspace in Vitess can be sharded or unsharded. An unsharded keyspace maps directly to a MySQL database. If sharded, the rows of the keyspace are partitioned into different databases of identical schema. -For example, if an application's "user" keyspace is split into two shards, each shard contains records for approximately half of the application's users. Similarly, each user's information is stored in only one shard. +For example, if an application's "user" keyspace is split into two shards, each shard contains records for approximately half of the application's users. Each single user's information however, is stored in only one shard. Note that sharding is orthogonal to (MySQL) replication. A Vitess shard typically contains one MySQL primary and many MySQL replicas. The primary handles write operations, while replicas handle read-only traffic, batch processing operations, and other tasks. Each MySQL instance within the shard should have the same data, excepting some replication lag. @@ -43,7 +43,7 @@ When building the serving graph for a sharded keyspace, Vitess ensures that each * An empty start value represents the lowest value, and all values are greater than it. * An empty end value represents a value larger than the highest possible value, and all values are strictly lower than it. -Vitess always converts sharding keys to a left-justified binary string for computing a shard. This left-justification makes the right-most zeroes insignificant and optional. Therefore, the value `0x80` is always the middle value for sharding keys. So, in a keyspace with two shards, sharding keys that have a binary value lower than 0x80 are assigned to one shard. Keys with a binary value equal to or higher than 0x80 are assigned to the other shard. +Vitess always converts sharding keys to a left-justified binary string for computing a shard. This left-justification makes the right-most zeroes insignificant and optional. Therefore, the value `0x80` is the middle value for sharding keys. So, in a keyspace with two shards, sharding keys that have a binary value lower than 0x80 are assigned to the first shard. Keys with a binary value equal to or higher than 0x80 are assigned to the other shard. Several sample key ranges are shown below: From a857875c2ae272a3c42c7fc275dc8e9e50a22880 Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Sun, 18 Dec 2022 18:26:02 +0530 Subject: [PATCH 3/8] feat: vitess-sequences review Signed-off-by: Manan Gupta --- content/en/docs/15.0/reference/features/vitess-sequences.md | 2 +- content/en/docs/16.0/reference/features/vitess-sequences.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/15.0/reference/features/vitess-sequences.md b/content/en/docs/15.0/reference/features/vitess-sequences.md index 936e259f9..7803cd8be 100644 --- a/content/en/docs/15.0/reference/features/vitess-sequences.md +++ b/content/en/docs/15.0/reference/features/vitess-sequences.md @@ -51,7 +51,7 @@ exposed to the clients only when safe. Alternative to auto-incrementing IDs are: * Using a 64 bit random generator number. Try to insert a new row with that - ID. If taken (because the statement returns an integrity error), try another + ID. If already taken (the statement returns an integrity error), try another ID. * Using a UUID scheme, and generate truly unique IDs. diff --git a/content/en/docs/16.0/reference/features/vitess-sequences.md b/content/en/docs/16.0/reference/features/vitess-sequences.md index 936e259f9..7803cd8be 100644 --- a/content/en/docs/16.0/reference/features/vitess-sequences.md +++ b/content/en/docs/16.0/reference/features/vitess-sequences.md @@ -51,7 +51,7 @@ exposed to the clients only when safe. Alternative to auto-incrementing IDs are: * Using a 64 bit random generator number. Try to insert a new row with that - ID. If taken (because the statement returns an integrity error), try another + ID. If already taken (the statement returns an integrity error), try another ID. * Using a UUID scheme, and generate truly unique IDs. From b263f27491ef8662d4c04da1606aa5bbe59b3a7f Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Mon, 19 Dec 2022 11:25:24 +0530 Subject: [PATCH 4/8] feat: mysql-replication page Signed-off-by: Manan Gupta --- .../15.0/reference/features/mysql-replication.md | 12 +++++++----- .../16.0/reference/features/mysql-replication.md | 12 +++++++----- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/content/en/docs/15.0/reference/features/mysql-replication.md b/content/en/docs/15.0/reference/features/mysql-replication.md index 29d4b09c4..7d4caea11 100644 --- a/content/en/docs/15.0/reference/features/mysql-replication.md +++ b/content/en/docs/15.0/reference/features/mysql-replication.md @@ -12,23 +12,25 @@ Vitess makes use of MySQL Replication for both high availability and to receive ## Semi-Sync -Vitess strongly recommends the use of Semi-synchronous replication for High Availability. When enabled in Vitess, _semi-sync_ has the following characteristics: +Vitess strongly recommends the use of Semi-synchronous replication for High Availability. The characteristics of semi-sync are replication governed by the [Durability Policy](../../../user-guides/configuration-basic/durability_policy) configured for the keyspace. +Some characteristics are shared by all the policies - -* The primary will only accept writes if it has at least one replica connected, and configured correctly to send semi-sync ACKs. Vitess configures the semi-sync timeout to essentially an unlimited number so that it will never fallback to asyncronous replication. This is important to prevent split brain (or alternate futures) in case of a network partition. If we can verify all replicas have stopped replicating, we know the old primary is not accepting writes, even if we are unable to contact the old primary itself. +* Vitess configures the semi-sync timeout to essentially an unlimited number so that it will never fallback to asyncronous replication. This is important to prevent split brain (or alternate futures) in case of a network partition. If we can verify all replicas have stopped replicating, we know the old primary is not accepting writes, even if we are unable to contact the old primary itself. -* Tablets of type rdonly will not send semi-sync ACKs. This is intentional because rdonly tablets are not eligible to be promoted to primary, so Vitess avoids the case where a rdonly tablet is the single best candidate for election at the time of primary failure. +* All pre-configured durability policies do not allow tablets of type rdonly to send semi-sync ACKs. This is intentional because rdonly tablets are not eligible to be promoted to primary, so Vitess avoids the case where a rdonly tablet is the single best candidate for election at the time of primary failure. -These behaviors combine to give you the property that, in case of primary failure, there is at least one other replica that has every transaction that was ever reported to clients as having completed. You can then ([manually](../../programs/vtctl/#emergencyreparentshard), or [using Orchestrator](../../../user-guides/configuration-advanced/integration-with-orchestrator/) to pick the replica that is farthest ahead in GTID position and promote that to be the new primary. +Having semi-sync enabled, gives you the property that, in case of primary failure, there is at least one other replica that has every transaction that was ever reported to clients as having completed. You can then wait for [VTOrc](../../../user-guides/configuration-basic/vtorc) to repair it, or ([manually](../../programs/vtctl/#emergencyreparentshard) pick the replica that is farthest ahead in GTID position and promote that to be the new primary. Thus, you can survive sudden primary failure without losing any transactions that were reported to clients as completed. In MySQL 5.7+, this guarantee is strengthened slightly to preventing loss of any transactions that were ever **committed** on the original primary, eliminating so-called [phantom reads](http://bugs.mysql.com/bug.php?id=62174). On the other hand these behaviors also give a requirement that each shard must have at least 2 tablets with type *replica* (with addition of the primary that can be demoted to type *replica* this gives a minimum of 3 tablets with initial type *replica*). This will allow for the primary to have a semi-sync acker when one of the replica tablets is down for any reason (for a version update, machine reboot, schema swap or anything else). +These requirements will changed based on the durability policy. With regard to replication lag, note that this does **not** guarantee there is always at least one replica from which queries will always return up-to-date results. Semi-sync guarantees that at least one replica has the transaction in its relay log, but it has not necessarily been applied yet. The only way Vitess guarantees a fully up-to-date read is to send the request to the primary. ## Database Schema Considerations -* Row-based replication requires that replicas have the same schema as the primary, and corruption will likely occur if the column order does not match. Earlier versions of Vitess which used Statement-Based replication recommended applying schema changes on replicas first, and then swapping their role to primary. This method is no longer recommended and a tool such as [`gh-ost`](https://github.com/github/gh-ost) or [`pt-online-schema-change`](https://www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html) should be used instead. +* Row-based replication requires that replicas have the same schema as the primary, and corruption will likely occur if the column order does not match. Earlier versions of Vitess which used Statement-Based replication recommended applying schema changes on replicas first, and then swapping their role to primary. This method is no longer recommended in favour of the use of Online-DDL. More information can be found [here](../../../user-guides/schema-changes). * Using a column of type `FLOAT` or `DOUBLE` as part of a Primary Key is not supported. This limitation is because Vitess may try to execute a query for a value (for example 2.2) which MySQL will return zero results, even when the approximate value is present. diff --git a/content/en/docs/16.0/reference/features/mysql-replication.md b/content/en/docs/16.0/reference/features/mysql-replication.md index 29d4b09c4..f4f9c61cc 100644 --- a/content/en/docs/16.0/reference/features/mysql-replication.md +++ b/content/en/docs/16.0/reference/features/mysql-replication.md @@ -12,23 +12,25 @@ Vitess makes use of MySQL Replication for both high availability and to receive ## Semi-Sync -Vitess strongly recommends the use of Semi-synchronous replication for High Availability. When enabled in Vitess, _semi-sync_ has the following characteristics: +Vitess strongly recommends the use of Semi-synchronous replication for High Availability. The characteristics of semi-sync are replication governed by the [Durability Policy](../../../user-guides/configuration-basic/durability_policy) configured for the keyspace. +Some characteristics are shared by all the policies - -* The primary will only accept writes if it has at least one replica connected, and configured correctly to send semi-sync ACKs. Vitess configures the semi-sync timeout to essentially an unlimited number so that it will never fallback to asyncronous replication. This is important to prevent split brain (or alternate futures) in case of a network partition. If we can verify all replicas have stopped replicating, we know the old primary is not accepting writes, even if we are unable to contact the old primary itself. +* Vitess configures the semi-sync timeout to essentially an unlimited number so that it will never fallback to asyncronous replication. This is important to prevent split brain (or alternate futures) in case of a network partition. If we can verify all replicas have stopped replicating, we know the old primary is not accepting writes, even if we are unable to contact the old primary itself. -* Tablets of type rdonly will not send semi-sync ACKs. This is intentional because rdonly tablets are not eligible to be promoted to primary, so Vitess avoids the case where a rdonly tablet is the single best candidate for election at the time of primary failure. +* All pre-configured durability policies do not allow tablets of type rdonly to send semi-sync ACKs. This is intentional because rdonly tablets are not eligible to be promoted to primary, so Vitess avoids the case where a rdonly tablet is the single best candidate for election at the time of primary failure. -These behaviors combine to give you the property that, in case of primary failure, there is at least one other replica that has every transaction that was ever reported to clients as having completed. You can then ([manually](../../programs/vtctl/#emergencyreparentshard), or [using Orchestrator](../../../user-guides/configuration-advanced/integration-with-orchestrator/) to pick the replica that is farthest ahead in GTID position and promote that to be the new primary. +Having semi-sync enabled, gives you the property that, in case of primary failure, there is at least one other replica that has every transaction that was ever reported to clients as having completed. You can then wait for [VTOrc](../../../user-guides/configuration-basic/vtorc) to repair it, or ([manually](../../programs/vtctl/#emergencyreparentshard) pick the replica that is farthest ahead in GTID position and promote that to be the new primary. Thus, you can survive sudden primary failure without losing any transactions that were reported to clients as completed. In MySQL 5.7+, this guarantee is strengthened slightly to preventing loss of any transactions that were ever **committed** on the original primary, eliminating so-called [phantom reads](http://bugs.mysql.com/bug.php?id=62174). On the other hand these behaviors also give a requirement that each shard must have at least 2 tablets with type *replica* (with addition of the primary that can be demoted to type *replica* this gives a minimum of 3 tablets with initial type *replica*). This will allow for the primary to have a semi-sync acker when one of the replica tablets is down for any reason (for a version update, machine reboot, schema swap or anything else). +These requirements will changed based on the durability policy. With regard to replication lag, note that this does **not** guarantee there is always at least one replica from which queries will always return up-to-date results. Semi-sync guarantees that at least one replica has the transaction in its relay log, but it has not necessarily been applied yet. The only way Vitess guarantees a fully up-to-date read is to send the request to the primary. ## Database Schema Considerations -* Row-based replication requires that replicas have the same schema as the primary, and corruption will likely occur if the column order does not match. Earlier versions of Vitess which used Statement-Based replication recommended applying schema changes on replicas first, and then swapping their role to primary. This method is no longer recommended and a tool such as [`gh-ost`](https://github.com/github/gh-ost) or [`pt-online-schema-change`](https://www.percona.com/doc/percona-toolkit/LATEST/pt-online-schema-change.html) should be used instead. +* Row-based replication requires that replicas have the same schema as the primary, and corruption will likely occur if the column order does not match. Earlier versions of Vitess which used Statement-Based replication recommended applying schema changes on replicas first, and then swapping their role to primary. This method is no longer recommended in favour of the use of Online-DDL. More information can be found [here](../../../user-guides/schema-changes). * Using a column of type `FLOAT` or `DOUBLE` as part of a Primary Key is not supported. This limitation is because Vitess may try to execute a query for a value (for example 2.2) which MySQL will return zero results, even when the approximate value is present. From 3d66ebf0ae213f87941ccd39d4c529f1139f4d43 Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Mon, 19 Dec 2022 12:25:35 +0530 Subject: [PATCH 5/8] feat: vschema page Signed-off-by: Manan Gupta --- content/en/docs/15.0/reference/features/vschema.md | 2 ++ content/en/docs/16.0/reference/features/vschema.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/content/en/docs/15.0/reference/features/vschema.md b/content/en/docs/15.0/reference/features/vschema.md index d8ae39e37..3136bed8b 100644 --- a/content/en/docs/15.0/reference/features/vschema.md +++ b/content/en/docs/15.0/reference/features/vschema.md @@ -299,6 +299,8 @@ If a query goes across multiple shards and ordering is needed on the `name` colu If `column_list_authoritative` is false or not specified, then VTGate will treat the list of columns as partial and will not automatically expand open-ended constructs like `select *`. +Vtgates also have the capability to track the schema changes and populate the columns list on its own. To know more about this feature, read [here](../schema-tracking). + ### Advanced usage The examples/demo also shows more tricks you can perform: diff --git a/content/en/docs/16.0/reference/features/vschema.md b/content/en/docs/16.0/reference/features/vschema.md index d8ae39e37..3136bed8b 100644 --- a/content/en/docs/16.0/reference/features/vschema.md +++ b/content/en/docs/16.0/reference/features/vschema.md @@ -299,6 +299,8 @@ If a query goes across multiple shards and ordering is needed on the `name` colu If `column_list_authoritative` is false or not specified, then VTGate will treat the list of columns as partial and will not automatically expand open-ended constructs like `select *`. +Vtgates also have the capability to track the schema changes and populate the columns list on its own. To know more about this feature, read [here](../schema-tracking). + ### Advanced usage The examples/demo also shows more tricks you can perform: From c577fc10d95aa7002a7843d63c7d5411fecfcff6 Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Mon, 19 Dec 2022 14:39:25 +0530 Subject: [PATCH 6/8] feat: vindexes page Signed-off-by: Manan Gupta --- content/en/docs/15.0/reference/features/vindexes.md | 4 ++-- content/en/docs/16.0/reference/features/vindexes.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/15.0/reference/features/vindexes.md b/content/en/docs/15.0/reference/features/vindexes.md index 7d1845180..f38ed24e2 100644 --- a/content/en/docs/15.0/reference/features/vindexes.md +++ b/content/en/docs/15.0/reference/features/vindexes.md @@ -64,7 +64,7 @@ The lookup table that implements a Lookup Vindex can be sharded or unsharded. N Vitess allows for the transparent population of these lookup table rows by assigning an owner table, which is the main table that requires this lookup. When a row is inserted into this owner table, the lookup row for it is created in the lookup table. The lookup row is also deleted upon a delete of the corresponding row in the owner table. These essentially result in distributed transactions, which traditionally require 2PC to guarantee atomicity. -Consistent lookup vindexes use an alternate approach that makes use of careful locking and transaction sequences to guarantee consistency without using 2PC. This gives the best of both worlds, with the benefit of a consistent cross-shard vindex without paying the price of 2PC. To read more about what makes a consistent lookup vindex different from a standard lookup vindex read our [consistent lookup vindexes design documentation](../../../design-docs/query-serving/clookup-vindex/). +Consistent lookup vindexes use an alternate approach that makes use of careful locking and transaction sequences to guarantee consistency without using 2PC. This gives the best of both worlds, with the benefit of a consistent cross-shard vindex without paying the price of 2PC. To read more about what makes a consistent lookup vindex different from a standard lookup vindex read our [consistent lookup vindexes design documentation](../../../../design-docs/query-serving/clookup-vindex). There are currently two vindex types in Vitess for consistent lookup: @@ -170,7 +170,7 @@ Vindex Type | Cost In the case of a simple select, Vitess scans the WHERE clause to match references to Vindex columns and chooses the best one to use. If there is no match and the query is simple without complex constructs like aggregates, etc., it is sent to all shards. -Vitess can handle more complex queries. For now, refer to the [design doc](https://github.com/vitessio/vitess/blob/main/doc/V3HighLevelDesign.md) for background information on how it handles them. +Vitess can handle more complex queries with the new Gen4 planner. #### Insert diff --git a/content/en/docs/16.0/reference/features/vindexes.md b/content/en/docs/16.0/reference/features/vindexes.md index 57ac78191..a885ebe30 100644 --- a/content/en/docs/16.0/reference/features/vindexes.md +++ b/content/en/docs/16.0/reference/features/vindexes.md @@ -64,7 +64,7 @@ The lookup table that implements a Lookup Vindex can be sharded or unsharded. N Vitess allows for the transparent population of these lookup table rows by assigning an owner table, which is the main table that requires this lookup. When a row is inserted into this owner table, the lookup row for it is created in the lookup table. The lookup row is also deleted upon a delete of the corresponding row in the owner table. These essentially result in distributed transactions, which traditionally require 2PC to guarantee atomicity. -Consistent lookup vindexes use an alternate approach that makes use of careful locking and transaction sequences to guarantee consistency without using 2PC. This gives the best of both worlds, with the benefit of a consistent cross-shard vindex without paying the price of 2PC. To read more about what makes a consistent lookup vindex different from a standard lookup vindex read our [consistent lookup vindexes design documentation](../../../design-docs/query-serving/clookup-vindex/). +Consistent lookup vindexes use an alternate approach that makes use of careful locking and transaction sequences to guarantee consistency without using 2PC. This gives the best of both worlds, with the benefit of a consistent cross-shard vindex without paying the price of 2PC. To read more about what makes a consistent lookup vindex different from a standard lookup vindex read our [consistent lookup vindexes design documentation](../../../../design-docs/query-serving/clookup-vindex). There are currently two vindex types in Vitess for consistent lookup: @@ -170,7 +170,7 @@ Vindex Type | Cost In the case of a simple select, Vitess scans the WHERE clause to match references to Vindex columns and chooses the best one to use. If there is no match and the query is simple without complex constructs like aggregates, etc., it is sent to all shards. -Vitess can handle more complex queries. For now, refer to the [design doc](https://github.com/vitessio/vitess/blob/main/doc/V3HighLevelDesign.md) for background information on how it handles them. +Vitess can handle more complex queries with the new Gen4 planner. #### Insert From 63672fadf86f7a45ca418833f0c6e1cabb4ffc7f Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Mon, 19 Dec 2022 15:02:23 +0530 Subject: [PATCH 7/8] feat: pitr docs Signed-off-by: Manan Gupta --- .../docs/15.0/reference/features/recovery.md | 15 ++++++--------- .../docs/16.0/reference/features/recovery.md | 19 +++++++------------ 2 files changed, 13 insertions(+), 21 deletions(-) diff --git a/content/en/docs/15.0/reference/features/recovery.md b/content/en/docs/15.0/reference/features/recovery.md index e6de67e71..64b53d35e 100644 --- a/content/en/docs/15.0/reference/features/recovery.md +++ b/content/en/docs/15.0/reference/features/recovery.md @@ -9,9 +9,6 @@ aliases: ['/docs/recovery/pitr','/docs/reference/pitr/'] ### Supported Databases - MySQL 5.7 - -### Believed to work, but untested - - MySQL 8.0 ### Introduction @@ -42,7 +39,7 @@ To use this feature, you need a usable backup of Vitess data and continuous binl Here is how you can create a backup. ```sh -$ vtctlclient --server : Backup zone1-101 +$ vtctldclient --server : Backup zone1-101 ``` Here `zone1-101` is the tablet alias of a replica tablet in the shard that you @@ -74,18 +71,18 @@ Once the above is done, you can proceed with doing a recovery. #### Recovery Procedure -First, you need to create a `SNAPSHOT` keyspace with a `base_keyspace` +First, you need to create a `SNAPSHOT` keyspace with a `base-keyspace` pointing to the original keyspace you are recovering the backup of. This can be done by using following: ```sh -$ vtctlclient --server : CreateKeyspace -- --keyspace_type=SNAPSHOT --base_keyspace=originalks --snapshot_time=2020-07-17T18:25:20Z restoreks +$ vtctldclient --server : CreateKeyspace --type=SNAPSHOT --base-keyspace=originalks --snapshot-time=2020-07-17T18:25:20Z restoreks ``` Here: - `originalks` is the base keyspace, i.e. the keyspace we took a backup of, and are trying to recover. - - `snapshot_time` is the timestamp of the point in time to we want to recover + - `snapshot-time` is the timestamp of the point in time to we want to recover to. Note the use of the `Z` in the timestamp, indicating it is expressed in UTC. - `restoreks` is the name of recovery keyspace, i.e. the keyspace to which @@ -148,7 +145,7 @@ You will also probably want to use other flags for backup and restore like: You need to be consistent in your use of these flags for backup and restore. -Once the restore of the last backup earlier than the `snapshot_time` timestamp +Once the restore of the last backup earlier than the `snapshot-time` timestamp is completed, the vttablet proceeds to use the `binlog_*` parameters to connect to the binlog server and then apply all binlog events from the time of the backup until the timestamp provided. @@ -157,7 +154,7 @@ Since the last backup for each shard making up the keyspace could be taken at different points in time, the amount of time that it takes to apply these events may differ between restores of different shards in the keyspace. -Note that to restore to the specified `snapshot_time` timestamp, vttablet needs +Note that to restore to the specified `snapshot-time` timestamp, vttablet needs to find the GTID corresponding to the last event before this timestamp from the binlog server. This is an expensive operation and may take some time. By default the timeout for this operation is one minute (1m). This can be changed diff --git a/content/en/docs/16.0/reference/features/recovery.md b/content/en/docs/16.0/reference/features/recovery.md index e6de67e71..7d9786d50 100644 --- a/content/en/docs/16.0/reference/features/recovery.md +++ b/content/en/docs/16.0/reference/features/recovery.md @@ -7,11 +7,6 @@ aliases: ['/docs/recovery/pitr','/docs/reference/pitr/'] ## Point in Time Recovery ### Supported Databases - -- MySQL 5.7 - -### Believed to work, but untested - - MySQL 8.0 ### Introduction @@ -42,11 +37,11 @@ To use this feature, you need a usable backup of Vitess data and continuous binl Here is how you can create a backup. ```sh -$ vtctlclient --server : Backup zone1-101 +$ vtctldclient --server : Backup zone1-101 ``` Here `zone1-101` is the tablet alias of a replica tablet in the shard that you -want to back up. Note that you can also use `vtctlclient BackupShard` to just +want to back up. Note that you can also use `vtctldclient BackupShard` to just specify a keyspace and shard, and have Vitess choose the tablet to run the backup for you, instead of having to specify the tablet alias explicitly. @@ -74,18 +69,18 @@ Once the above is done, you can proceed with doing a recovery. #### Recovery Procedure -First, you need to create a `SNAPSHOT` keyspace with a `base_keyspace` +First, you need to create a `SNAPSHOT` keyspace with a `base-keyspace` pointing to the original keyspace you are recovering the backup of. This can be done by using following: ```sh -$ vtctlclient --server : CreateKeyspace -- --keyspace_type=SNAPSHOT --base_keyspace=originalks --snapshot_time=2020-07-17T18:25:20Z restoreks +$ vtctldclient --server : CreateKeyspace --type=SNAPSHOT --base-keyspace=originalks --snapshot-time=2020-07-17T18:25:20Z restoreks ``` Here: - `originalks` is the base keyspace, i.e. the keyspace we took a backup of, and are trying to recover. - - `snapshot_time` is the timestamp of the point in time to we want to recover + - `snapshot-time` is the timestamp of the point in time to we want to recover to. Note the use of the `Z` in the timestamp, indicating it is expressed in UTC. - `restoreks` is the name of recovery keyspace, i.e. the keyspace to which @@ -148,7 +143,7 @@ You will also probably want to use other flags for backup and restore like: You need to be consistent in your use of these flags for backup and restore. -Once the restore of the last backup earlier than the `snapshot_time` timestamp +Once the restore of the last backup earlier than the `snapshot-time` timestamp is completed, the vttablet proceeds to use the `binlog_*` parameters to connect to the binlog server and then apply all binlog events from the time of the backup until the timestamp provided. @@ -157,7 +152,7 @@ Since the last backup for each shard making up the keyspace could be taken at different points in time, the amount of time that it takes to apply these events may differ between restores of different shards in the keyspace. -Note that to restore to the specified `snapshot_time` timestamp, vttablet needs +Note that to restore to the specified `snapshot-time` timestamp, vttablet needs to find the GTID corresponding to the last event before this timestamp from the binlog server. This is an expensive operation and may take some time. By default the timeout for this operation is one minute (1m). This can be changed From 0d5d2fce4fe21ce6420250107e5f8f7f0085a3a0 Mon Sep 17 00:00:00 2001 From: Manan Gupta Date: Mon, 19 Dec 2022 15:30:00 +0530 Subject: [PATCH 8/8] feat: connection pools page Signed-off-by: Manan Gupta --- content/en/docs/15.0/reference/features/connection-pools.md | 2 +- content/en/docs/16.0/reference/features/connection-pools.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/15.0/reference/features/connection-pools.md b/content/en/docs/15.0/reference/features/connection-pools.md index 33da4351a..d30eb0eb7 100644 --- a/content/en/docs/15.0/reference/features/connection-pools.md +++ b/content/en/docs/15.0/reference/features/connection-pools.md @@ -122,7 +122,7 @@ Flag: `--transaction_limit_per_user` ### vtgate system settings Flag: `--enable_system_settings` -This vtgate flag allows clients to modify a [subset of system settings](https://github.com/vitessio/vitess/blob/main/go/vt/sysvars/sysvars.go#L166-L211) on the MySQL. This is done using the mechanism of [reserved connection](../../query-serving/reserved-conn/#enabling-reserved-connections). +This vtgate flag allows clients to modify a [subset of system settings](https://github.com/vitessio/vitess/blob/main/go/vt/sysvars/sysvars.go#L174-L217) on the MySQL. This is done using the mechanism of [reserved connection](../../query-serving/reserved-conn/#enabling-reserved-connections). Once a reserved connection is created, it lives for the life of the vtgate client session. These connections live outside of the regular connection pool and as a result, the number of MySQL server connections used by vttablet may become significantly higher than what you might expect based on the pool settings. diff --git a/content/en/docs/16.0/reference/features/connection-pools.md b/content/en/docs/16.0/reference/features/connection-pools.md index 33da4351a..d30eb0eb7 100644 --- a/content/en/docs/16.0/reference/features/connection-pools.md +++ b/content/en/docs/16.0/reference/features/connection-pools.md @@ -122,7 +122,7 @@ Flag: `--transaction_limit_per_user` ### vtgate system settings Flag: `--enable_system_settings` -This vtgate flag allows clients to modify a [subset of system settings](https://github.com/vitessio/vitess/blob/main/go/vt/sysvars/sysvars.go#L166-L211) on the MySQL. This is done using the mechanism of [reserved connection](../../query-serving/reserved-conn/#enabling-reserved-connections). +This vtgate flag allows clients to modify a [subset of system settings](https://github.com/vitessio/vitess/blob/main/go/vt/sysvars/sysvars.go#L174-L217) on the MySQL. This is done using the mechanism of [reserved connection](../../query-serving/reserved-conn/#enabling-reserved-connections). Once a reserved connection is created, it lives for the life of the vtgate client session. These connections live outside of the regular connection pool and as a result, the number of MySQL server connections used by vttablet may become significantly higher than what you might expect based on the pool settings.