diff --git a/Gemfile.lock b/Gemfile.lock index ce879b30e69..1a6f40c5259 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -10,7 +10,7 @@ GIT GIT remote: https://github.com/duckdb/rouge.git - revision: d487f892d0caa08e0ea6053509aa83b765c60cd1 + revision: 0ca2c48ea4da4419f7089988c8663a9bf8c75064 branch: duckdb specs: rouge (3.3823.1) diff --git a/docs/extensions/mysql.md b/docs/extensions/mysql.md index 923f89b75aa..1061c4d487e 100644 --- a/docs/extensions/mysql.md +++ b/docs/extensions/mysql.md @@ -35,21 +35,95 @@ The connection string determines the parameters for how to connect to MySQL as a
-| Setting | Default | Environment variable | -|----------|----------------|----------------------| -| database | NULL | MYSQL_DATABASE | -| host | localhost | MYSQL_HOST | -| password | | MYSQL_PWD | -| port | 0 | MYSQL_TCP_PORT | -| socket | NULL | MYSQL_UNIX_PORT | -| user | ⟨current user⟩ | MYSQL_USER | +| Setting | Default | Environment variable | +|-------------|----------------|----------------------| +| database | NULL | MYSQL_DATABASE | +| host | localhost | MYSQL_HOST | +| password | | MYSQL_PWD | +| port | 0 | MYSQL_TCP_PORT | +| socket | NULL | MYSQL_UNIX_PORT | +| user | ⟨current user⟩ | MYSQL_USER | +| ssl_mode | preferred | | +| ssl_ca | | | +| ssl_capath | | | +| ssl_cert | | | +| ssl_cipher | | | +| ssl_crl | | | +| ssl_crlpath | | | +| ssl_key | | | + + +### Configuring via Secrets + +MySQL connection information can also be specified with [secrets](/docs/configuration/secrets_manager). The following syntax can be used to create a secret. + +```sql +CREATE SECRET ( + TYPE MYSQL, + HOST '127.0.0.1', + PORT 0, + DATABASE mysql, + USER 'mysql', + PASSWORD '' +); +``` + +The information from the secret will be used when `ATTACH` is called. We can leave the connection string empty to use all of the information stored in the secret. + +```sql +ATTACH '' AS mysql_db (TYPE MYSQL); +``` + +We can use the connection string to override individual options. For example, to connect to a different database while still using the same credentials, we can override only the database name in the following manner. + +```sql +ATTACH 'database=my_other_db' AS mysql_db (TYPE MYSQL); +``` + +By default, created secrets are temporary. Secrets can be persisted using the [`CREATE PERSISTENT SECRET` command]({% link docs/configuration/secrets_manager.md %}#persistent-secrets). Persistent secrets can be used across sessions. + +#### Managing Multiple Secrets + +Named secrets can be used to manage connections to multiple MySQL database instances. Secrets can be given a name upon creation. + +```sql +CREATE SECRET mysql_secret_one ( + TYPE MYSQL, + HOST '127.0.0.1', + PORT 0, + DATABASE mysql, + USER 'mysql', + PASSWORD '' +); +``` + +The secret can then be explicitly referenced using the `SECRET` parameter in the `ATTACH`. + +```sql +ATTACH '' AS mysql_db_one (TYPE MYSQL, SECRET mysql_secret_one); +``` + +### SSL Connections + +The [`ssl` connection parameters](https://dev.mysql.com/doc/refman/8.4/en/using-encrypted-connections.html) can be used to make SSL connections. Below is a description of the supported parameters. + +| Setting | Description | +|-------------|--------------------------------------------------------------------------------------------------------------------------------------------------| +| ssl_mode | The security state to use for the connection to the server: `disabled, required, verify_ca, verify_identity or preferred` (default: `preferred`) | +| ssl_ca | The path name of the Certificate Authority (CA) certificate file. | +| ssl_capath | The path name of the directory that contains trusted SSL CA certificate files. | +| ssl_cert | The path name of the client public key certificate file. | +| ssl_cipher | The list of permissible ciphers for SSL encryption. | +| ssl_crl | The path name of the file containing certificate revocation lists. | +| ssl_crlpath | The path name of the directory that contains files containing certificate revocation lists. | +| ssl_key | The path name of the client private key file. | ### Reading MySQL Tables The tables in the MySQL database can be read as if they were normal DuckDB tables, but the underlying data is read directly from MySQL at query time. ```sql -SHOW TABLES; +SHOW ALL TABLES; ```
diff --git a/docs/extensions/postgres.md b/docs/extensions/postgres.md index 3d16ebc0c3d..70b27b318c2 100644 --- a/docs/extensions/postgres.md +++ b/docs/extensions/postgres.md @@ -35,12 +35,19 @@ To connect to the PostgreSQL instance with the given parameters in read-only mod ATTACH 'dbname=postgres user=postgres host=127.0.0.1' AS db (TYPE POSTGRES, READ_ONLY); ``` +By default, all schemas are attached. When working with large instances, it can be useful to only attach a specific schema. This can be accomplished using the `SCHEMA` command. + +```sql +ATTACH 'dbname=postgres user=postgres host=127.0.0.1' AS db (TYPE POSTGRES, SCHEMA 'public'); +``` + + ### Configuration The `ATTACH` command takes as input either a [`libpq` connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING) or a [PostgreSQL URI](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS). -Below are some example connection strings and commonly used parameters. A full list of available parameters can be found [in the PostgreSQL documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS). +Below are some example connection strings and commonly used parameters. A full list of available parameters can be found in the [PostgreSQL documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS). ```text dbname=postgresscanner @@ -59,6 +66,56 @@ host=localhost port=5432 dbname=mydb connect_timeout=10 An example URI is `postgresql://username@hostname/dbname`. +### Configuring via Secrets + +PostgreSQL connection information can also be specified with [secrets](/docs/configuration/secrets_manager). The following syntax can be used to create a secret. + +```sql +CREATE SECRET ( + TYPE POSTGRES, + HOST '127.0.0.1', + PORT 5432, + DATABASE postgres, + USER 'postgres', + PASSWORD '' +); +``` + +The information from the secret will be used when `ATTACH` is called. We can leave the Postgres connection string empty to use all of the information stored in the secret. + +```sql +ATTACH '' AS postgres_db (TYPE POSTGRES); +``` + +We can use the Postgres connection string to override individual options. For example, to connect to a different database while still using the same credentials, we can override only the database name in the following manner. + +```sql +ATTACH 'dbname=my_other_db' AS postgres_db (TYPE POSTGRES); +``` + +By default, created secrets are temporary. Secrets can be persisted using the [`CREATE PERSISTENT SECRET` command]({% link docs/configuration/secrets_manager.md %}#persistent-secrets). Persistent secrets can be used across sessions. + +#### Managing Multiple Secrets + +Named secrets can be used to manage connections to multiple Postgres database instances. Secrets can be given a name upon creation. + +```sql +CREATE SECRET postgres_secret_one ( + TYPE POSTGRES, + HOST '127.0.0.1', + PORT 5432, + DATABASE postgres, + USER 'postgres', + PASSWORD '' +); +``` + +The secret can then be explicitly referenced using the `SECRET` parameter in the `ATTACH`. + +```sql +ATTACH '' AS postgres_db_one (TYPE POSTGRES, SECRET postgres_secret_one); +``` + ### Configuring via Environment Variables PostgreSQL connection information can also be specified with [environment variables](https://www.postgresql.org/docs/current/libpq-envars.html). @@ -330,6 +387,7 @@ The extension exposes the following configuration parameters. | `pg_experimental_filter_pushdown` | Whether or not to use filter pushdown (currently experimental) | `false` | | `pg_pages_per_task` | The amount of pages per task | `1000` | | `pg_use_binary_copy` | Whether or not to use BINARY copy to read data | `true` | +| `pg_null_byte_replacement` | When writing NULL bytes to Postgres, replace them with the given character | `NULL` | | `pg_use_ctid_scan` | Whether or not to parallelize scanning using table ctids | `true` | ## Schema Cache