Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new doc about TSO #14630

Merged
merged 31 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
66c15ba
Add more details about TSO
dveeden Aug 28, 2023
6bb2307
Fix manual line breaks reported by CI
dveeden Aug 28, 2023
9d98828
Add TIDB_PARSE_TSO_LOGICAL
dveeden Sep 8, 2023
657b9f7
Merge remote-tracking branch 'upstream/master' into tso
dveeden Sep 8, 2023
0cb373b
Merge remote-tracking branch 'upstream/master' into tso
dveeden Sep 28, 2023
8601410
Update tso.md
dveeden Oct 19, 2023
03a0888
Merge remote-tracking branch 'upstream/master' into tso
dveeden Oct 19, 2023
a5a7ffa
Revert "Update tso.md"
dveeden Oct 19, 2023
23173fa
Update based on review
dveeden Oct 19, 2023
8deecf7
Merge remote-tracking branch 'upstream/master' into pr/14630
hfxsd Nov 1, 2023
cfaa6de
polish wording
hfxsd Nov 1, 2023
04442d3
fix link of TIDB_PARSE_TSO_LOGICAL()
hfxsd Nov 1, 2023
a541cde
fix abbr
hfxsd Nov 1, 2023
2064bbd
Apply suggestions from code review
hfxsd Nov 1, 2023
28281f3
Update tso.md
hfxsd Nov 1, 2023
aa09d7f
Update tso.md
hfxsd Nov 1, 2023
43774eb
Update tso.md
hfxsd Nov 1, 2023
f5430f0
Merge branch 'tso' of https://github.com/dveeden/docs into pr/14630
hfxsd Nov 1, 2023
817d0a6
Update tso.md
hfxsd Nov 1, 2023
a0181a1
Update tso.md
hfxsd Nov 1, 2023
93f825a
Update tso.md
hfxsd Nov 1, 2023
e6122f8
Update tso.md
hfxsd Nov 1, 2023
25c99ce
Update tso.md
hfxsd Nov 1, 2023
c52faba
Update tso.md
hfxsd Nov 1, 2023
a9573be
Update tso.md
hfxsd Nov 1, 2023
300d8a4
Update tso.md
hfxsd Nov 1, 2023
3d429e2
Update tso.md
hfxsd Nov 1, 2023
1919de3
Apply suggestions from code review
hfxsd Nov 3, 2023
578d169
Update TOC.md
hfxsd Nov 3, 2023
bbed7a9
Merge branch 'tso' of https://github.com/dveeden/docs into pr/14630
hfxsd Nov 3, 2023
451831a
Update TOC-tidb-cloud.md
hfxsd Nov 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,7 @@
- Reference
- TiDB Cluster Architecture
- [Overview](/tidb-architecture.md)
- [TSO](/tso.md)
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
- [Storage](/tidb-storage.md)
- [Computing](/tidb-computing.md)
- [Scheduling](/tidb-scheduling.md)
Expand Down
1 change: 1 addition & 0 deletions TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,7 @@
- Reference
- Cluster Architecture
- [Overview](/tidb-architecture.md)
- [TSO](/tso.md)
- [Storage](/tidb-storage.md)
- [Computing](/tidb-computing.md)
- [Scheduling](/tidb-scheduling.md)
Expand Down
87 changes: 87 additions & 0 deletions tso.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
title: TimeStamp Oracle
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
summary: Learn about TSO timestamps in TiDB.
---

hfxsd marked this conversation as resolved.
Show resolved Hide resolved
One of the tasks of the Placement Driver (PD) is to hand out timestamps to other components of the cluster. Transactions and data get timestamps assigned and this allows the [Percolator](https://research.google.com/pubs/pub36726.html) model in TiDB to work, which is used for MVCC and [transactions](/transaction-overview.md).
mjonss marked this conversation as resolved.
Show resolved Hide resolved

Let's get a TSO timestamp:

```
mysql> BEGIN; SET @ts := @@tidb_current_ts; ROLLBACK;
Query OK, 0 rows affected (0.0007 sec)
Query OK, 0 rows affected (0.0002 sec)
Query OK, 0 rows affected (0.0001 sec)

sql> SELECT @ts;
+--------------------+
| @ts |
+--------------------+
| 443852055297916932 |
+--------------------+
1 row in set (0.00 sec)
```

Note that this is done in a transction with (`BEGIN; ...; ROLLBACK`) as TSO timestamps are assigned to transactions.

There are two SQL function which helps us to inspect the number that we got back: [`TIDB_PARSE_TSO()`](/functions-and-operators/tidb-functions.md#tidb_parse_tso) and [`TIDB_PARSE_TSO_LOGICAL()`]((/functions-and-operators/tidb-functions.md#tidb_parse_tso_logical).

```
mysql> SELECT TIDB_PARSE_TSO(443852055297916932);
+------------------------------------+
| TIDB_PARSE_TSO(443852055297916932) |
+------------------------------------+
| 2023-08-27 20:33:41.687000 |
+------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT TIDB_PARSE_TSO_LOGICAL(443852055297916932);
+--------------------------------------------+
| TIDB_PARSE_TSO_LOGICAL(443852055297916932) |
+--------------------------------------------+
| 4 |
+--------------------------------------------+
1 row in set (0.00 sec)
```

Let's now dive a bit deeper into what a TSO timestamp looks like:

```
0000011000101000111000010001011110111000110111000000000000000100 ← This is 443852055297916932, but in binary
hfxsd marked this conversation as resolved.
Show resolved Hide resolved
0000011000101000111000010001011110111000110111 ← The first 46 bits are the physical timestamp
000000000000000100 ← The last 18 bits are the logical timestamp
```

There are two parts to the TSO timestamp:

- The *Physical timestamp*: This is a UNIX timestamp in milliseconds since 1 January 1970.
- The *Logical timestamp*: This is an increasing counter. This is used when there are multiple timestamps needed within the same millisecond or if there is a change that makes the clock go backwards, in that case the physical timestamp is kept the same while the logical timestamp increases. This is done as the TSO timestamp is guaranteed to never go back.

With this knowledge we can inspect the TSO timestamp a bit more in SQL:

```
mysql> SELECT @ts, UNIX_TIMESTAMP(NOW(6)), (@ts >> 18)/1000, FROM_UNIXTIME((@ts >> 18)/1000), NOW(6), @ts-((@ts >> 18) << 18)\G
dveeden marked this conversation as resolved.
Show resolved Hide resolved
*************************** 1. row ***************************
@ts: 443852055297916932
UNIX_TIMESTAMP(NOW(6)): 1693161835.502954
(@ts >> 18)/1000: 1693161221.6870
FROM_UNIXTIME((@ts >> 18)/1000): 2023-08-27 20:33:41.6870
NOW(6): 2023-08-27 20:43:55.502954
@ts-((@ts >> 18) << 18): 4
1 row in set (0.00 sec)
```

The `>> 18` is to [shift right](/functions-and-operators/bit-functions-and-operators.md) by 18 bits, which is used to filter out the physical timestamp. As the physical timestamp is in milliseconds and not in seconds as is more usual for UNIX timestamps we need to divide by 1000 to get it in the format that [`FROM_UNIXTIME()`](/functions-and-operators/date-and-time-functions.md) understands. This is basically the same as what `TIDB_PARSE_TSO()` does for us.

Then we also filter out the logical timestamp: `000000000000000100` in binary, which is 4 in decimals.

And let's do the same via the CLI tools:

```
$ tiup ctl:v7.3.0 pd tso 443852055297916932
dveeden marked this conversation as resolved.
Show resolved Hide resolved
Starting component `ctl`: /home/dvaneeden/.tiup/components/ctl/v7.3.0/ctl pd tso 443852055297916932
dveeden marked this conversation as resolved.
Show resolved Hide resolved
system: 2023-08-27 20:33:41.687 +0200 CEST
logic: 4
```

Here you can see the same physical timestamp in the line that starts with `system:` and the logical timestamp in the line that starts with `logic:`.
hfxsd marked this conversation as resolved.
Show resolved Hide resolved