0.4.0

0.4.0 brings lots of new features including switch, select ![] and numbers with underscores. We have initial (unpublished) bindings to Elixir. And there's the usual improvements to fixes & documentation (only a minority are listed below in this release).

0.4.0 also has some breaking changes: table is let, dialect is renamed to target, and the compiler's API has changed. Full details below.

Features:

• Defining a temporary table is now expressed as let rather than table (@aljazerzen, #1315). See the tables docs for details.

• Experimental: The switch function sets a variable to a value based on one of several expressions (@aljazerzen, #1278).

  derive var = switch [
    score <= 10 -> "low",
    score <= 30 -> "medium",
    score <= 70 -> "high",
    true -> "very high",
  ]
  

  ...compiles to:

  SELECT
    *,
    CASE
      WHEN score <= 10 THEN 'low'
      WHEN score <= 30 THEN 'medium'
      WHEN score <= 70 THEN 'high'
      ELSE 'very high'
    END AS var
  FROM
    bar

  Check out the switch docs for more details.

• Experimental: Columns can be excluded by name with select (@aljazerzen, #1329)

  from albums
  select ![title, composer]
  

• Experimental: append transform, equivalent to UNION ALL in SQL. (@aljazerzen, #894)

  from employees
  append managers
  

  Check out the append docs for more details.

• Numbers can contain underscores, which can make reading long numbers easier (@max-sixty, #1467):

  from numbers
  select [
      small = 1.000_000_1,
      big = 5_000_000,
  ]
  

• The SQL output contains a comment with the PRQL compiler version (@aljazerzen, #1322)

• dialect is renamed to target, and its values are prefixed with sql. (@max-sixty, #1388); for example:

  prql target:sql.bigquery  # previously was `dialect:bigquery`
  
  from employees
  

  This gives us the flexibility to target other languages than SQL in the long term.

• Tables definitions can contain a bare s-string (@max-sixty, #1422), which enables us to include a full CTE of SQL, for example:

  let grouping = s"""
    SELECT SUM(a)
    FROM tbl
    GROUP BY
      GROUPING SETS
      ((b, c, d), (d), (b, d))
  """
  

• Ranges supplied to in can be half-open (@aljazerzen, #1330).

• The crate's external API has changed to allow for compiling to intermediate representation. This also affects bindings. See prql_compiler docs for more details.

Fixes:

[This release, the changelog only contains a subset of fixes]

• Allow interpolations in table s-strings (@aljazerzen, #1337)

Documentation:

[This release, the changelog only contains a subset of documentation improvements]

• Add docs on aliases in
  Select
• Add JS template literal and multiline example (@BCsabaEngine, #1432)
• JS template literal and multiline example (@BCsabaEngine, #1432)
• Improve prql-compiler docs & examples (@aljazerzen, #1515)
• Fix string highlighting in book (@max-sixty, #1264)

Web:

• The playground allows querying some sample data. As before, the result updates
  on every keystroke. (@aljazerzen, #1305)

Integrations:

[This release, the changelog only contains a subset of integration improvements]

• Added Elixir integration exposing PRQL functions as NIFs (#1500, @kasvith)
• Exposed Elixir flavor with exceptions (#1513, @kasvith)
• Rename prql-compiler binary to prqlc (@aljazerzen #1515)

Internal changes:

[This release, the changelog only contains a subset of internal changes]

• Add parsing for negative select (@max-sixty, #1317)
• Allow for additional builtin functions (@aljazerzen, #1325)
• Add an automated check for typos (@max-sixty, #1421)
• Add tasks for running playground & book (@max-sixty, #1265)
• Add tasks for running tests on every file change (@max-sixty, #1380)

New contributors:

• @EArazli, with #1359
• @boramalper, with #1362
• @allurefx, with #1377
• @bcho, with #1375
• @JettChenT, with #1385
• @BlurrechDev, with #1411
• @BCsabaEngine, with #1432
• @kasvith, with #1500

0.3.1

0.3.1 brings a couple of small improvements and fixes.

Features:

• Support for using s-strings for from (#1197, @aljazerzen)

  from s"SELECT * FROM employees WHERE foo > 5"

• Helpful error message when referencing a table in an s-string (#1203, @aljazerzen)

Fixes:

• Multiple columns with same name created (#1211, @aljazerzen)
• Renaming via select breaks preceding sorting (#1204, @aljazerzen)
• Same column gets selected multiple times (#1186, @mklopets)

Internal:

• Update Github Actions and Workflows to current version numbers (and avoid using Node 12) (#1201)

0.3.0

🎉 0.3.0 is the biggest ever change in PRQL's compiler 🎉. It rewrites much of the internals: the compiler now has a semantic understanding of expressions, including resolving names & building a DAG of column lineage.

While the immediate changes to the language are modest — some long-running bugs are fixed — this unlocks beginning development of many of the broad features we've had ambitions for, such as type-checking & auto-complete. And it simplifies building our next language features, such as match-case expressions, unions & table expressions.

@aljazerzen has (mostly single-handedly) done this work over the past few months. The project owes him immense appreciation.

Breaking changes:

We've had to make some modest breaking changes for 0.3:

• Pipelines must start with from. For example, a pipeline with only derive foo = 5, with no from transform, is no longer valid. Depending on demand for this feature, it would be possible to add this back.

• Shared column names now require == in a join. For example:

  from employees
  -join positions [id]
  +join positions [==id]

  The existing approach is ambiguous to the compiler — id could be a boolean column.

• Table references containing periods must be surrounded by backticks. For example, when referencing a schema name:

  -from public.sometable
  +from `public.sometable`

Features:

• Change self equality op to == (#1176, @aljazerzen)
• Add logging (@aljazerzen)
• Add clickhouse dialect (#1090, @max-sixty)
• Allow namespaces & tables to contain . (#1079, @aljazerzen)

Fixes:

• Deduplicate column appearing in SELECT multiple times (#1186, @aljazerzen)
• Fix uppercase table names (#1184, @aljazerzen)
• Omit table name when only one ident in SELECT (#1094, @aljazerzen)

Documentation:

• Add chapter on semantics' internals (@aljazerzen, #1028)
• Add note about nesting variables in s-strings (@max-sixty, #1163)

Internal changes:

• Flatten group and window (#1120, @aljazerzen)
• Split ast into expr and stmt (@aljazerzen)
• Refactor associativity (#1156, @aljazerzen)
• Rename Ident constructor to from_name (#1084, @aljazerzen)
• Refactor rq folding (#1177, @aljazerzen)
• Add tests for reported bugs fixes in semantic (#1174, @aljazerzen)
• Bump duckdb from 0.5.0 to 0.6.0 (#1132)
• Bump once_cell from 1.15.0 to 1.16.0 (#1101)
• Bump pest from 2.4.0 to 2.5.0 (#1161)
• Bump pest_derive from 2.4.0 to 2.5.0 (#1179)
• Bump sqlparser from 0.25.0 to 0.27.0 (#1131)
• Bump trash from 2.1.5 to 3.0.0 (#1178)

0.2.11

0.2.11 contains a few helpful fixes.

Work continues on our semantic refactor — look out for 0.3.0 soon! Many thanks to @aljazerzen for his continued contributions to this.

Note: 0.2.10 was skipped due to this maintainer's inability to read his own docs on bumping versions...

Features:

• Detect when compiler version is behind query version (@MarinPostma, #1058)
• Add __version__ to prql-python package (@max-sixty, #1034)

Fixes:

• Fix nesting of expressions with equal binding strength and left associativity,
  such as a - (b - c) (@max-sixty, #1136)
• Retain floats without significant digits as floats (@max-sixty, #1141)

Documentation:

• Add documentation of prqlr bindings (@eitsupi, #1091)
• Add a 'Why PRQL' section to the website (@max-sixty, #1098)
• Add @snth to core-devs (@max-sixty, #1050)

Internal changes:

• Use workspace versioning (@max-sixty, #1065)

0.2.9

0.2.9 is a small release containing a bug fix for empty strings.

Fixes:

• Fix parsing of empty strings (@aljazerzen, #1024)

0.2.8

0.2.8 is another modest release with some fixes, doc improvements, bindings improvements, and lots of internal changes. Note that one of the fixes causes the behavior of round and cast to change slightly — though it's handled as a fix rather than a breaking change in semantic versioning.

Fixes:

• Change order of the round & cast function parameters to have the column last; for example round 2 foo_col / cast int foo. This is consistent with other functions, and makes piping possible:

  derive [
    gross_salary = (salary + payroll_tax | as int),
    gross_salary_rounded = (gross_salary | round 0),
  ]

Documentation:

• Split DEVELOPMENT.md from CONTRIBUTING.md (@richb-hanover, #1010)
• Make s-strings more prominent in website intro (@max-sixty, #982)

Web:

• Add GitHub star count to website (@max-sixty, #990)

Integrations:

• Expose a shortened error message, in particular for the VSCode extension (@aljazerzen, #1005)

Internal changes:

• Specify 1.60.0 as minimum rust version (@max-sixty, #1011)
• Remove old wee-alloc code (@max-sixty, #1013)
• Upgrade clap to version 4 (@aj-bagwell, #1004)
• Improve book-building script in Taskfile (@max-sixty, #989)
• Publish website using an artifact rather than a long-lived branch (@max-sixty, #1009)

0.2.7

0.2.7 is a fairly modest release, six weeks after 0.2.6. We have some more significant features, including a union operator and an overhaul of our type system, as open PRs which will follow in future releases.

We also have new features in the VSCode extension, courtesy of @jiripospisil, including a live output panel.

Fixes:

• range_of_ranges checks the Range end is smaller than its start (@Shuozeli, #946)

Documentation:

• Improve various docs (@max-sixty, #974, #971, #972, #970, #925)
• Add reference to EdgeDB's blog post in our FAQ (@max-sixty, #922)
• Fix typos (@kianmeng, #943)

Integrations:

• Add prql-lib, enabling language bindings with go (@sigxcpu76, #923)
• Fix line numbers in JS exceptions (@charlie-sanders, #929)

Internal changes:

• Lock the version of the rust-toolchain, with auto-updates (@max-sixty, #926, #927)

0.2.6

0.2.6 is a modest release following 0.2.5 a week ago. It includes lots of small fixes and some docs:

Fixes

• Adjust fmt to only escape names when needed (@aljazerzen, #907)
• Fix quoting on upper case table names (@max-sixty, #893)
• Fix scoping of identical column names from multiple tables (@max-sixty, #908)
• Fix parse error on newlines in a table (@sebastiantoh 🆕, #902)
• Fix quoting of upper case table names (@max-sixty, #893)

Documentation

• Add docs on Architecture (@aljazerzen, #904)
• Add Changelog (@max-sixty, #890 #891)

Internal changes

• Start trial using Conventional Commits (@max-sixty, #889)
• Add crates.io release workflow, docs (@max-sixty, #887)

---

Thanks to @sebastiantoh for his first PR into PRQL!

0.2.5

0.2.5 is a very small release following 0.2.4 yesterday. It includes:

• Add the ability to represent single brackets in an s-string, with two brackets (#752, @max-sixty )
• Fix the "Copy to Clipboard" command in the Playground, for Firefox (#880, @mklopets )

0.2.4

0.2.4 is a small release following 0.2.3 a few days ago. The 0.2.4 release includes:

• Enrich our CLI, adding commands to get different stages of the compilation process (@aljazerzen , #863)
• Fix multiple take n statements in a query, leading to duplicate proxy columns in generated SQL (@charlie-sanders )
• Fix BigQuery quoting of identifiers in SELECT statements (@max-sixty )
• Some internal changes — reorganize top-level functions (@aljazerzen ), add a workflow to track our rust compilation time (@max-sixty ), simplify our simple prql-to-sql tests (@max-sixty )

Thanks to @ankane, prql-compiler is now available from homebrew core; brew install prql-compiler¹.

Reiterating our plans from the 0.2.3 release notes:

From here, we're planning to continue squashing bugs (albeit more minor than those in this release), adding some features like union, while working on bigger issues such as type-inference.

We're also going to document and modularize the compiler further. It's important that we give more people an opportunity to contribute to the guts of PRQL, especially given the number and enthusiasm of contributions to project in general — and it's not that easy to do so at the moment. While this is ongoing if anyone has something they'd like to work on in the more difficult parts of the compiler, let us know on GitHub or Discord, and we'd be happy to work together on it.

1. we still need to update docs and add a release workflow for this: #866 ↩