Skip to content

Commit

Permalink
Carlo feedback. Add Wasm shell link. Script to convert to Wasm link.
Browse files Browse the repository at this point in the history
  • Loading branch information
Alex-Monahan committed Sep 26, 2024
1 parent 90ec895 commit 654c024
Show file tree
Hide file tree
Showing 2 changed files with 79 additions and 2 deletions.
11 changes: 9 additions & 2 deletions _posts/2024-09-27-sql-only-extensions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ excerpt: "Easily create sharable extensions using only SQL macros that can apply
SQL is not a new language.
As a result, it has historically been missing some of the modern luxuries we take for granted.
With version 1.1, DuckDB has launched community extensions, bringing the incredible power of a package manager to the SQL language.
One goal for these extensions is to enable C++ libraries to be accessible through SQL across all of the languages with a DuckDB library.
A bold goal of ours is for DuckDB to become a convenient way to wrap any C++ library, much the way that Python does today, but across any language with a DuckDB client.

For extension builders, compilation and distribution are much easier.
For the user community, installation is as simple as two commands:

Expand All @@ -21,7 +22,9 @@ INSTALL pivot_table FROM community;
LOAD pivot_table;
```

However, not all of us are C++ developers!
The extension can then be used in any query through SQL functions.

However, **not all of us are C++ developers**!
Can we, as a SQL community, build up a set of SQL helper functions?
What would it take to build these extensions with *just SQL*?

Expand All @@ -43,6 +46,8 @@ They are very powerful when used in combination with other friendly SQL features

Traditionally, there has been no central repository for SQL functions across databases, let alone across companies!
DuckDB's community extensions can be that knowledge base.
DuckDB extensions can be used across all languages with a DuckDB client, including Python, NodeJS, Java, Rust, Go, and even Webassembly (Wasm)!

If you are a DuckDB fan and a SQL user, you can share your expertise back to the community with an extension.
This post will show you how!
No C++ knowledge is needed - just a little bit of copy/paste and GitHub Actions handles all the compilation.
Expand Down Expand Up @@ -253,6 +258,8 @@ Any set of tables (or views!) will first be vertically stacked and then pivoted.

## Example Using `pivot_table`

[Check out a live example using the extension in the DuckDB Wasm shell here](https://shell.duckdb.org/#queries=v0,CREATE-OR-REPLACE-TABLE-business_metrics-(-----product_line-VARCHAR%2C-product-VARCHAR%2C-year-INTEGER%2C-quarter-VARCHAR%2C-revenue-integer%2C-cost-integer-)~,INSERT-INTO-business_metrics-VALUES-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2022%2C-'Q1'%2C-100%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2022%2C-'Q2'%2C-200%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2022%2C-'Q3'%2C-300%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2022%2C-'Q4'%2C-400%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2023%2C-'Q1'%2C-500%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2023%2C-'Q2'%2C-600%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2023%2C-'Q3'%2C-700%2C-100)%2C-----('Waterfowl-watercraft'%2C-'Duck-boats'%2C-2023%2C-'Q4'%2C-800%2C-100)%2C------('Duck-Duds'%2C-'Duck-suits'%2C-2022%2C-'Q1'%2C-10%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2022%2C-'Q2'%2C-20%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2022%2C-'Q3'%2C-30%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2022%2C-'Q4'%2C-40%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2023%2C-'Q1'%2C-50%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2023%2C-'Q2'%2C-60%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2023%2C-'Q3'%2C-70%2C-10)%2C-----('Duck-Duds'%2C-'Duck-suits'%2C-2023%2C-'Q4'%2C-80%2C-10)%2C------('Duck-Duds'%2C-'Duck-neckties'%2C-2022%2C-'Q1'%2C-1%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2022%2C-'Q2'%2C-2%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2022%2C-'Q3'%2C-3%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2022%2C-'Q4'%2C-4%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2023%2C-'Q1'%2C-5%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2023%2C-'Q2'%2C-6%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2023%2C-'Q3'%2C-7%2C-1)%2C-----('Duck-Duds'%2C-'Duck-neckties'%2C-2023%2C-'Q4'%2C-8%2C-1)%2C~,FROM-business_metrics~,INSTALL-pivot_table-from-community~,LOAD-'https%3A%2F%2Fcommunity extensions.duckdb.org%2Fv1.1.1%2Fwasm_eh%2Fpivot_table.duckdb_extension.wasm'~,DROP-TYPE-IF-EXISTS-columns_parameter_enum~,CREATE-TYPE-columns_parameter_enum-AS-ENUM-(FROM-build_my_enum(['business_metrics']%2C-['year'%2C-'quarter']%2C-[]))~,FROM-pivot_table(['business_metrics']%2C['sum(revenue)'%2C-'sum(cost)']%2C-['product_line'%2C-'product']%2C-['year'%2C-'quarter']%2C-[]%2C-subtotals-%3A%3D-1%2C-grand_totals-%3A%3D-1%2C-values_axis-%3A%3D-'rows')~)!

<details markdown='1'>
<summary markdown='span'>
First we will create an example data table. We are a duck product distributor, and we are tracking our fowl finances.
Expand Down
70 changes: 70 additions & 0 deletions scripts/sql_to_wasm_shell_link.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@


# Note, this may not handle all special characters
shell_link_stub = "https://shell.duckdb.org/#queries=v0,"
# sql = """
# install tpch;
# load tpch;
# call dbgen(sf=0.1);
# pragma tpch(7);
# """

sql = """
CREATE OR REPLACE TABLE business_metrics (
product_line VARCHAR, product VARCHAR, year INTEGER, quarter VARCHAR, revenue integer, cost integer
);
INSERT INTO business_metrics VALUES
('Waterfowl watercraft', 'Duck boats', 2022, 'Q1', 100, 100),
('Waterfowl watercraft', 'Duck boats', 2022, 'Q2', 200, 100),
('Waterfowl watercraft', 'Duck boats', 2022, 'Q3', 300, 100),
('Waterfowl watercraft', 'Duck boats', 2022, 'Q4', 400, 100),
('Waterfowl watercraft', 'Duck boats', 2023, 'Q1', 500, 100),
('Waterfowl watercraft', 'Duck boats', 2023, 'Q2', 600, 100),
('Waterfowl watercraft', 'Duck boats', 2023, 'Q3', 700, 100),
('Waterfowl watercraft', 'Duck boats', 2023, 'Q4', 800, 100),
('Duck Duds', 'Duck suits', 2022, 'Q1', 10, 10),
('Duck Duds', 'Duck suits', 2022, 'Q2', 20, 10),
('Duck Duds', 'Duck suits', 2022, 'Q3', 30, 10),
('Duck Duds', 'Duck suits', 2022, 'Q4', 40, 10),
('Duck Duds', 'Duck suits', 2023, 'Q1', 50, 10),
('Duck Duds', 'Duck suits', 2023, 'Q2', 60, 10),
('Duck Duds', 'Duck suits', 2023, 'Q3', 70, 10),
('Duck Duds', 'Duck suits', 2023, 'Q4', 80, 10),
('Duck Duds', 'Duck neckties', 2022, 'Q1', 1, 1),
('Duck Duds', 'Duck neckties', 2022, 'Q2', 2, 1),
('Duck Duds', 'Duck neckties', 2022, 'Q3', 3, 1),
('Duck Duds', 'Duck neckties', 2022, 'Q4', 4, 1),
('Duck Duds', 'Duck neckties', 2023, 'Q1', 5, 1),
('Duck Duds', 'Duck neckties', 2023, 'Q2', 6, 1),
('Duck Duds', 'Duck neckties', 2023, 'Q3', 7, 1),
('Duck Duds', 'Duck neckties', 2023, 'Q4', 8, 1),
;
FROM business_metrics;
INSTALL pivot_table from community;
LOAD 'https://community-extensions.duckdb.org/v1.1.1/wasm_eh/pivot_table.duckdb_extension.wasm';
DROP TYPE IF EXISTS columns_parameter_enum;
CREATE TYPE columns_parameter_enum AS ENUM (FROM build_my_enum(['business_metrics'], ['year', 'quarter'], []));
FROM pivot_table(['business_metrics'],['sum(revenue)', 'sum(cost)'], ['product_line', 'product'], ['year', 'quarter'], [], subtotals := 1, grand_totals := 1, values_axis := 'rows');
"""

statements = sql.strip().split(sep=";")

encoded_statements = []

for statement in statements:
trimmed = statement.strip()
no_hyphens = trimmed.replace('-','%2D')
no_spaces = no_hyphens.replace('\n',' ').replace(' ', '-')
encoded = no_spaces.replace(',','%2C').replace('=','%3D').replace(':','%3A').replace(r'/','%2F').replace('%2D',' ')
encoded_statements.append(encoded)

combined = shell_link_stub + '~,'.join(encoded_statements)

print(combined)

0 comments on commit 654c024

Please sign in to comment.