Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore bit and varbit support with tests #83

Merged
merged 16 commits into from
Nov 16, 2023
Merged
28 changes: 23 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Features
- `mod()` is pushdowned. In PostgreSQL gives [argument-dependend data type](https://www.postgresql.org/docs/current/functions-math.html), but result from SQLite always [have `real` affinity](https://www.sqlite.org/lang_mathfunc.html#mod).
- `upper`, `lower` and other character case functions are **not** pushed down because they does not work with UNICODE character in SQLite.
- `WITH TIES` option is **not** pushed down.
- Bit string `#` (XOR) operator is **not** pushed down becasuse there is no equal SQLite operator.

### Notes about pushdowning

Expand Down Expand Up @@ -117,7 +118,7 @@ Usage
### Datatypes
**WARNING! The table above represents roadmap**, work still in progress. Untill it will be ended please refer real behaviour in non-obvious cases, where there is no ✔ or ∅ mark.

This table represents `sqlite_fdw` behaviour if in PostgreSQL foreign table column some [affinity](https://www.sqlite.org/datatype3.html) of SQLite data is detected.
This table represents `sqlite_fdw` behaviour if in PostgreSQL foreign table column some [affinity](https://www.sqlite.org/datatype3.html) of SQLite data is detected. Some details about data values support see in [limitations](#limitations).

* **∅** - no support (runtime error)
* **V** - transparent transformation
Expand All @@ -133,7 +134,7 @@ SQLite `NULL` affinity always can be transparent converted for a nullable column
| PostgreSQL | SQLite <br> INT | SQLite <br> REAL | SQLite <br> BLOB | SQLite <br> TEXT | SQLite <br> TEXT but <br>empty|SQLite<br>nearest<br>affinity|
|-------------:|:------------:|:------------:|:------------:|:------------:|:------------:|-------------:|
| bool | V | ? | T | - | ∅ | INT |
| bit(n) | V | ∅ | V | ? | ∅ | INT |
| bit(n) | V n<=64 | ∅ | V | ? | ∅ | INT |
| bytea | b | b | ✔ | - | ? | BLOB |
| date | V | V | T | V+ | `NULL` | ? |
| float4 | V+ | ✔ | T | - | `NULL` | REAL |
Expand All @@ -150,8 +151,7 @@ SQLite `NULL` affinity always can be transparent converted for a nullable column
|timestamp + tz| V | V | T | V+ | `NULL` | ? |
| uuid | ∅ | ∅ |V+<br>(only<br>16 bytes)| V+ | ∅ | TEXT, BLOB |
| varchar | ? | ? | T | ✔ | V | TEXT |


| varbit(n) | V n<=64 | ∅ | V | ? | ∅ | INT |

### CREATE SERVER options

Expand Down Expand Up @@ -523,6 +523,10 @@ Limitations
- Expected affinity of UUID value in SQLite table determined by `column_type` option of the column
for `INSERT` and `UPDATE` commands.

### bit and varbit support
- `sqlite_fdw` PostgreSQL `bit`/`varbit` values support based on `int` SQLite data affinity, because there is no per bit operations for SQLite `blob` affinity data. Maximum SQLite `int` affinity value is 8 bytes length, hence maximum `bit`/`varbit` values length is 64 bits.
- `sqlite_fdw` doesn't pushdown `#` (XOR) operator becasuse there is no equal SQLite operator.

Tests
-----
Test directory have structure as following:
Expand Down Expand Up @@ -566,10 +570,24 @@ Contributing

Opening issues and pull requests on GitHub are welcome.
For pull request, please make sure these items below for testing:
- Create test cases (if needed) for the latest version of PostgreSQL supported by sqlite_fdw.
- Create test cases (if needed) for the latest version of PostgreSQL supported by `sqlite_fdw`. All error testcases should have a comment about test purpose.
- Execute test cases and update expectations for the latest version of PostgreSQL
- Test creation and execution for other PostgreSQL versions are welcome but not required.

Preferred code style see in PostgreSQL source codes. For example

```C
type
funct_name (type arg ...)
{
for (;;)
{
}
if ()
{
}
}
```
Useful links
------------

Expand Down
38 changes: 24 additions & 14 deletions deparse.c
Original file line number Diff line number Diff line change
Expand Up @@ -675,17 +675,17 @@ sqlite_foreign_expr_walker(Node *node,
ReleaseSysCache(tuple);

/*
* Factorial (!) and Bitwise XOR (^) cannot be pushed down to
* SQLite
* Factorial (!) and Bitwise XOR (^), (#)
* cannot be pushed down to SQLite
* Full list see in https://www.postgresql.org/docs/current/functions-bitstring.html
* ILIKE cannot be pushed down to SQLite
* Full list see in https://www.postgresql.org/docs/current/functions-matching.html
*/
if (strcmp(cur_opname, "!") == 0
|| strcmp(cur_opname, "^") == 0)
{
return false;
}

/* ILIKE cannot be pushed down to SQLite */
if (strcmp(cur_opname, "~~*") == 0 || strcmp(cur_opname, "!~~*") == 0)
|| strcmp(cur_opname, "^") == 0
|| strcmp(cur_opname, "#") == 0
|| strcmp(cur_opname, "~~*") == 0
|| strcmp(cur_opname, "!~~*") == 0)
{
return false;
}
Expand Down Expand Up @@ -2621,8 +2621,16 @@ sqlite_deparse_const(Const *node, deparse_expr_cxt *context, int showtype)
break;
case BITOID:
case VARBITOID:
extval = OidOutputFunctionCall(typoutput, node->constvalue);
appendStringInfo(buf, "B\'%s\'", extval);
{
extval = OidOutputFunctionCall(typoutput, node->constvalue);
if (strlen(extval) > SQLITE_FDW_BIT_DATATYPE_BUF_SIZE - 1 )
{
ereport(ERROR, (errcode(ERRCODE_FDW_INVALID_DATA_TYPE),
errmsg("SQLite FDW dosens't support very long bit/varbit data"),
errhint("bit length %ld, maxmum %ld", strlen(extval), SQLITE_FDW_BIT_DATATYPE_BUF_SIZE - 1)));
}
appendStringInfo(buf, "%lld", binstr2int64(extval));
}
break;
case BOOLOID:
{
Expand Down Expand Up @@ -2880,14 +2888,16 @@ sqlite_deparse_operator_name(StringInfo buf, Form_pg_operator opform)
}
else if (strcmp(cur_opname, "~~*") == 0 ||
strcmp(cur_opname, "!~~*") == 0 ||
strcmp(cur_opname, "~") == 0 ||
/* ~ operator is both one of text RegEx operators and bit string NOT */
(strcmp(cur_opname, "~") == 0 && opform->oprresult != VARBITOID && opform->oprresult != BITOID) ||
strcmp(cur_opname, "!~") == 0 ||
strcmp(cur_opname, "~*") == 0 ||
strcmp(cur_opname, "!~*") == 0)
{
elog(ERROR, "OPERATOR is not supported");
ereport(ERROR, (errcode(ERRCODE_FDW_ERROR),
errmsg("SQL operator is not supported"),
errhint("operator name: %s", cur_opname)));
}

else
{
appendStringInfoString(buf, cur_opname);
Expand Down
Loading