Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removes references to Index Capacity #47

Merged
merged 1 commit into from
Aug 26, 2024
Merged

Removes references to Index Capacity #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 1 addition & 41 deletions concept/indexes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,31 +61,6 @@ Table Index
| 6 | B | 1 |
```

### Index capacity

When a symbol column is indexed, an additional **index capacity** can be defined
to specify how many row IDs to store in a single storage block on disk:

- Server-wide setting: `cairo.index.value.block.size` with a default of `256`
- Column-wide setting: The
[`index` option](/docs/reference/sql/create-table/#column-indexes) for
`CREATE TABLE`
- Column-wide setting:
[ALTER TABLE COLUMN ADD INDEX](/docs/reference/sql/alter-table-alter-column-add-index/)

Fewer blocks used to store row IDs achieves better performance. At the same time
over-sizing the setting will result in higher than necessary disk space usage.

:::note

- The **index capacity** and
[**symbol capacity**](/docs/concept/symbol/#usage-of-symbols) are different
settings.
- The index capacity value should not be changed, unless an user is aware of all
the implications.

:::

## Advantages

Index allows you to greatly reduce the complexity of queries that span a subset
Expand Down Expand Up @@ -117,24 +92,9 @@ Consider the following query applied to the above table

### Table with index

An example of `CREATE TABLE` command creating a table with an index capacity of
128:
An example of `CREATE TABLE` command:

```questdb-sql
CREATE TABLE my_table(symb SYMBOL, price DOUBLE, ts TIMESTAMP),
INDEX (symb CAPACITY 128) timestamp(ts);
-- equivalent to
CREATE TABLE my_table(symb SYMBOL INDEX CAPACITY 128, price DOUBLE, ts TIMESTAMP),
timestamp(ts);
```

### Index capacity

Consider an example table with 200 unique stock symbols and 1,000,000,000
records over time. The index will have to store 1,000,000,000 / 200 row IDs for
each symbol, i.e. 5,000,000 per symbol.

- If the index capacity is set to 1,048,576 in this case, QuestDB will use 5
blocks to store the row IDs.
- If the index capacity is set to 1,024 in this case, the block count will be
4,883.
2 changes: 1 addition & 1 deletion operations/design-for-performance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ has multiple options passed for performance optimization.

```questdb-sql
CREATE TABLE my_table(
symb SYMBOL CAPACITY 1048576 NOCACHE INDEX CAPACITY 512,
symb SYMBOL CAPACITY 1048576 NOCACHE,
vch VARCHAR,
ts TIMESTAMP
) timestamp(ts) PARTITION BY DAY;
Expand Down
52 changes: 38 additions & 14 deletions reference/sql/alter-table-change-column-type.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,16 @@ description: ALTER TABLE COLUMN TYPE SQL keyword reference documentation.

Changes the data type of an existing column in a table.

The data type of the column is altered without affecting the data already stored in the table. However, it's important to note that altering the column type can result in data loss or errors if the new type cannot accommodate the existing data. Therefore, it's recommended to review the data and backup the table before altering the column type.
The data type of the column is altered without affecting the data already stored
in the table. However, it's important to note that altering the column type can
result in data loss or errors if the new type cannot accommodate the existing
data. Therefore, it's recommended to review the data and backup the table before
altering the column type.

:::caution

- Changing the column type may lead to data loss or errors if the new type cannot accommodate the existing data.
- Changing the column type may lead to data loss or errors if the new type
cannot accommodate the existing data.

- The new data type must be compatible with the existing data in the column.

Expand All @@ -22,7 +27,8 @@ The data type of the column is altered without affecting the data already stored

## Supported Data Types

The `ALTER TABLE COLUMN TYPE` command supports changing the column type to any compatible data type.
The `ALTER TABLE COLUMN TYPE` command supports changing the column type to any
compatible data type.

## Examples

Expand All @@ -32,23 +38,32 @@ Change the data type of the column `age` in the table `employees` to `INTEGER`:
ALTER TABLE employees ALTER COLUMN age TYPE INTEGER;
```

When changing the column type, ensure that the new type is compatible with the existing data. For instance, changing a column type from STRING to DOUBLE might result in data loss or conversion errors if the existing data contains non-numeric values.
When changing the column type, ensure that the new type is compatible with the
existing data. For instance, changing a column type from STRING to DOUBLE might
result in data loss or conversion errors if the existing data contains
non-numeric values.

```questdb-sql
ALTER TABLE tbl ALTER COLUMN col_name TYPE DOUBLE;
```

It is possible to specify all the additional column type parameters, like `CAPACITY`, `INDEX`
It is possible to specify all the additional column type parameters, like
`CAPACITY` & `CACHE`:

```questdb-sql
ALTER TABLE tbl ALTER COLUMN department TYPE SYMBOL CAPACITY 10000 CACHE INDEX CAPACITY 512;
ALTER TABLE tbl ALTER COLUMN department TYPE SYMBOL CAPACITY 10000 CACHE;
```

## Available Conversions

QuestDB supports a wide range of conversions. However, certain type conversions may lead to data precision loss (e.g., converting a `FLOAT` type to an `INT`) or range overflow (e.g., converting a `LONG` type to an `INT`). The matrices below depict fully compatible conversions marked with `X` and conversions that may result in data loss marked with `L`.
QuestDB supports a wide range of conversions. However, certain type conversions
may lead to data precision loss (e.g., converting a `FLOAT` type to an `INT`) or
range overflow (e.g., converting a `LONG` type to an `INT`). The matrices below
depict fully compatible conversions marked with `X` and conversions that may
result in data loss marked with `L`.

Numeric types support a wide range of conversions, but many of them can result in the data / precision loss.
Numeric types support a wide range of conversions, but many of them can result
in the data / precision loss.

| From \ To | boolean | byte | short | int | float | long | double | date | timestamp |
| --------- | ------- | ---- | ----- | --- | ----- | ---- | ------ | ---- | --------- |
Expand All @@ -60,16 +75,22 @@ Numeric types support a wide range of conversions, but many of them can result i
| long | L | L | L | L | L | | L | X | X |
| double | L | L | L | L | X | L | | L | L |

Conversions between `TIMESTAMP` and `DATE` types and numeric types are fully supported. Timestamp values are represented in microseconds since the EPOCH, while Date values are represented in milliseconds since the EPOCH. The EPOCH is defined as `1970-01-01T00:00:00.000000Z`.
Conversions between `TIMESTAMP` and `DATE` types and numeric types are fully
supported. Timestamp values are represented in microseconds since the EPOCH,
while Date values are represented in milliseconds since the EPOCH. The EPOCH is
defined as `1970-01-01T00:00:00.000000Z`.

Additionally, when converting from `BOOLEAN` values to numerics, `false` is represented as `0`, and `true` is represented as `1`. On the way back `0` and `NULL` are converted to `false` and all other values converted to `true`.
Additionally, when converting from `BOOLEAN` values to numerics, `false` is
represented as `0`, and `true` is represented as `1`. On the way back `0` and
`NULL` are converted to `false` and all other values converted to `true`.

| From \ To | boolean | byte | short | int | float | long | double | date | timestamp |
| --------- | ------- | ---- | ----- | --- | ----- | ---- | ------ | ---- | --------- |
| date | L | L | L | L | L | X | X | | X |
| timestamp | L | L | L | L | L | X | X | L | |

Conversions to `SYMBOL`, `STRING` and `VARCHAR` are supported from most of the data types.
Conversions to `SYMBOL`, `STRING` and `VARCHAR` are supported from most of the
data types.

| From \ To | symbol | string | varchar |
| --------- | ------ | ------ | ------- |
Expand All @@ -89,21 +110,24 @@ Conversions to `SYMBOL`, `STRING` and `VARCHAR` are supported from most of the d
| string | X | | X |
| varchar | X | X | |

However conversion from `SYMBOL`, `STRING` and `VARCHAR` to other types can result in `NULL` values for inconvertable string values.
However conversion from `SYMBOL`, `STRING` and `VARCHAR` to other types can
result in `NULL` values for inconvertable string values.

| From \ To | boolean | byte | short | char | int | float | long | date | timestamp | double | uuid |
| --------- | ------- | ---- | ----- | ---- | --- | ----- | ---- | ---- | --------- | ------ | ---- |
| string | L | L | L | L | L | L | L | L | L | L | L |
| varchar | L | L | L | L | L | L | L | L | L | L | L |
| symbol | L | L | L | L | L | L | L | L | L | L | L |

When column type change results into range overflow or precision loss, the same rules as explicit [CAST](/docs/reference/sql/cast/) apply.
When column type change results into range overflow or precision loss, the same
rules as explicit [CAST](/docs/reference/sql/cast/) apply.

## Unsupported Conversions

Converting from the type to itself is not supported.

If the column `department` is of type `SYMBOL`, then the following query will result in error, even if the capacity parameter changes:
If the column `department` is of type `SYMBOL`, then the following query will
result in error, even if the capacity parameter changes:

```questdb-sql
ALTER TABLE employees ALTER COLUMN department TYPE SYMBOL CAPACITY 4096;
Expand Down
60 changes: 13 additions & 47 deletions reference/sql/create-table.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Finally, we add additional parameters for our SYMBOL type:
```questdb-sql title="Adding parameters for symbol type"
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL CAPACITY 256 NOCACHE INDEX CAPACITY 1048576,
symbol SYMBOL CAPACITY 256 NOCACHE,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp)
Expand Down Expand Up @@ -256,18 +256,6 @@ CREATE TABLE trades(
PARTITION BY DAY;
```

The symbol capacity is not to be confused with **index capacity** described in
[column indexes](#column-indexes) below.

```questdb-sql
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL CAPACITY 50 INDEX capacity 256,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp);
```

#### Symbol caching

`CACHE | NOCACHE` is used to specify whether a symbol should be cached. The
Expand Down Expand Up @@ -312,30 +300,8 @@ CREATE TABLE trades(
), INDEX(symbol) TIMESTAMP(timestamp);
```

An index capacity may be provided for the index by defining the index storage
parameter, `valueBlockSize`:

```questdb-sql

CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
), INDEX (symbol CAPACITY 128) TIMESTAMP(timestamp);

-- equivalent to

CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp);
```

See [Index](/docs/concept/indexes/#how-indexes-work) for more information about
index capacity.
See the [Index concept](/docs/concept/indexes/#how-indexes-work) for more
information about indexes.

## OWNED BY

Expand All @@ -349,7 +315,7 @@ go to the user, group, or service account named in that clause.
CREATE GROUP analysts;
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp) PARTITION BY DAY
Expand All @@ -369,8 +335,8 @@ CREATE TABLE new_trades AS(
) TIMESTAMP(timestamp);
```

We can use keywords such as `IF NOT EXISTS`, `PARTITION BY`..., as needed for the new
table. The data type of a column can be changed:
We can use keywords such as `IF NOT EXISTS`, `PARTITION BY`..., as needed for
the new table. The data type of a column can be changed:

```questdb-sql title="Clone an existing wide table and change type of cherry-picked columns"
CREATE TABLE new_trades AS(
Expand Down Expand Up @@ -429,7 +395,7 @@ The size of the batches can be configured:
- locally, by using the `BATCH` keyword in the `CREATE TABLE` statement.

```questdb-sql title="Create batched table as select"
CREATE BATCH 4096 TABLE new_trades AS(
CREATE BATCH 4096 TABLE new_trades AS(
SELECT *
FROM
trades
Expand Down Expand Up @@ -464,8 +430,8 @@ PARTITION BY MONTH;
## CREATE TABLE LIKE

The `LIKE` keyword clones the table schema of an existing table without copying
the data. Table settings and parameters such as designated timestamp, symbol
column indexes, and index capacity will be cloned, too.
the data. Table settings and parameters such as designated timestamp and symbol
column indexes will be cloned, too.

```questdb-sql title="Create table like"
CREATE TABLE new_table (LIKE my_table);
Expand All @@ -489,7 +455,7 @@ The global setting for the same parameter is `cairo.max.uncommitted.rows`.
```questdb-sql title="Setting out-of-order table parameters via SQL"
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp)
Expand Down Expand Up @@ -523,7 +489,7 @@ The use of the comma (`,`) depends on the existence of the `WITH` clause:
```questdb-sql
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp)
Expand All @@ -538,7 +504,7 @@ The use of the comma (`,`) depends on the existence of the `WITH` clause:
```questdb-sql
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp)
Expand All @@ -553,7 +519,7 @@ The use of quotation marks (`'`) depends on the volume alias:
```questdb-sql
CREATE TABLE trades(
timestamp TIMESTAMP,
symbol SYMBOL INDEX CAPACITY 128,
symbol SYMBOL,
price DOUBLE,
amount DOUBLE
) TIMESTAMP(timestamp)
Expand Down
Loading