0.11.0

0.11.0 introduces new date, text & math modules with lots of standard functions, including a new date.to_text function. It contains a few bugs fixes, and lots of internal improvements to the compiler.

This release has 119 commits from 9 contributors. Selected changes:

Language:

• Breaking: group's by columns are now excluded from the partition. (#3490)
• Breaking: round is now in the math module and needs to be called via math.round. (#3928)
• Breaking: lower and upper are now in the text module and need to be called via text.lower and text.upper. (#3913, #3973)

Features:

• The std.in function now supports a list of values (@PrettyWood, #3883)
• Most standard mathematical functions are now supported: abs, floor, ceil, pi, exp, ln, log10, log, sqrt, degrees, radians, cos, acos, sin, asin, tan, atan, pow and round. Those functions are in the math module (@PrettyWood, #3909, #3916 & 3928)
• Most standard string functions are now supported: ltrim, rtrim, trim, length, extract, replace. Utility functions starts_with, contains and ends_with are also available. Those functions are in the text module (@PrettyWood, #3913, #3973)
• Formatting a date to a text is now available for Clickhouse, DuckDB, MySQL, MSSQL and Postgres. A new date module has been added with the to_text function (@PrettyWood, #3951, #3954 & #3955)

Fixes:

• Fix an issue with arithmetic precedence (@max-sixty, #3846)
• + and - can be used after a cast (@PrettyWood, #3923)
• The Lezer grammar had plenty of improvements and fixes. (@vanillajonathan)

Web:

• The Playground now uses Vite. (@vanillajonathan)

Internal changes:

• Bump prql-compiler's MSRV to 1.70.0 (@eitsupi, #3876)

New Contributors:

• @PrettyWood, with #3883

0.10.1

0.10.1 is a small release containing some internal fixes of the compiler.

This release has 36 commits from 7 contributors. Selected changes:

Features:

• The std.sql.read_csv function and the std.sql.read_parquet function supports the sql.glaredb target. (@eitsupi, #3749)

Fixes:

• Fix the bug of compiling to DISTINCT ON when take 1 is used with group by for the targets sql.clickhouse, sql.duckdb and sql.postgres. (@aljazerzen, #3792)

Integrations:

• Enable integration tests for GlareDB. (@eitsupi, #3749)
• trapd00r/LS_COLORS, a collection of LS_COLORS definitions colorizes .prql files. (@vanillajonathan)
• vivid, a themeable LS_COLORS generator colorizes .prql files. (@vanillajonathan)
• colorls, displays .prql files with a database icon. (@vanillajonathan)
• Emoji File Icons, a VS Code extension displays .prql files with a database emoji icon. (@vanillajonathan)
• eza, a modern ls replacement colorizes .prql files. (@vanillajonathan)
• lsd, next gen ls command displays .prql files with a database icon. (@vanillajonathan)

0.10.0

0.10.0 contains lots of small improvements, including support for new types of literal notation, support for read_* functions in more dialects, playground improvements, and a better Lezer grammar (which we're planning on using for a Jupyter extension).

This release has 155 commits from 9 contributors. Selected changes:

Language:

• Breaking: Case syntax now uses brackets [] rather than braces {}. To convert previous PRQL queries to this new syntax simply change case { ... } to case [ ... ]. (@AaronMoat, #3517)

Features:

• Breaking: The std.sql.read_csv function is now compiled to read_csv by default. Please set the target sql.duckdb to use the DuckDB's read_csv_auto function as previously. (@eitsupi, #3599)
• The std.sql.read_csv function and the std.sql.read_parquet function supports the sql.clickhouse target. (@eitsupi, #1533)
• Add std.prql_version function to return PRQL version (@hulxv, #3533)
• Add support for hex escape sequences in strings. Example "Hello \x51". (@vanillajonathan, #3568)
• Add support for long Unicode escape sequences. Example "Hello \u{01F422}". (@vanillajonathan, #3569)
• Add support for binary numerical notation. Example filter status == 0b1111000011110000. (@vanillajonathan, #3661)
• Add support for hexadecimal numerical notation. Example filter status == 0xff. (@vanillajonathan, #3654)
• Add support for octal numerical notation. Example filter status == 0o777. (@vanillajonathan, #3672)
• New compile target sql.glaredb for GlareDB and integration tests for it (However, there is a bug in the test and it is currently not running). (@universalmind303, @scsmithr, @eitsupi, #3669)

Web:

• Allow cmd-/ (Mac) or ctrl-/ (Windows) to toggle comments in the playground editor (@AaronMoat, #3522)

• Limit maximum height of the playground editor's error panel to avoid taking over whole screen (@AaronMoat, #3524)

• The playground now uses Vite (@vanillajonathan).

Integrations:

• Add a CLI command prqlc collect to collect a project's modules into a single file (@aljazerzen, #3739)
• Add a CLI command prqlc debug expand-pl to parse & and expand into PL without resolving (@aljazerzen, #3739)
• Bump prqlc's MSRV to 1.70.0 (@eitsupi, #3521)
• Pygments, a syntax highlighting library now has syntax highlighting for PRQL. (@vanillajonathan, #3564)
• chroma, a syntax highlighting library written in Go and used by the static website generator Hugo. (@vanillajonathan, #3597)
• scc, a source lines of code counter now has support for .prql files. (@vanillajonathan)
• gcloc a source lines of code counter now has support for .prql files. (@vanillajonathan)
• cloc a source lines of code counter now has support for .prql files. (@AlDanial)
• gocloc a source lines of code counter now has support for .prql files. (@vanillajonathan)
• The Quarto VS Code extension supports editing PRQL code blocks (prqlr is required to render Quarto Markdown with PRQL code blocks). (@jjallaire)

New Contributors:

• @hulxv, with #3533
• @AaronMoat, with #3522
• @jangorecki, with #3634

0.9.5

0.9.5 adds a line-wrapping character, fixes a few bugs, and improves our CI. The release has 77 commits from 8 contributors. Selected changes are below.

Look out for some conference talks coming up over the next few weeks, including QCon SF on Oct 2 and date2day on Oct 12.

Language:

• A new line-wrapping character, for lines that are long and we want to break up into multiple physical lines. This is slightly different from from many languages — it's on the subsequent line:

  from artists
  select is_europe =
  \ country == "DE"
  \ || country == "FR"
  \ || country == "ES"
  

  This allows for easily commenting out physical lines while maintaining a correct logical line; for example:

  from artists
  select is_europe =
  \ country == "DE"
  \ || country == "FR"
  \ || country == "FR"
  -\ || country == "ES"
  +#\ || country == "ES"

  (@max-sixty, #3408)

Fixes:

• Fix stack overflow on very long queries in Windows debug builds (@max-sixty, #2908)

• Fix panic when unresolved lineage appears in group or window (@davidot, #3266)

• Fix a corner-case in handling precedence, and remove unneeded parentheses in some outputs (@max-sixty, #3472)

Web:

• Compiler panics are now printed to the console (@max-sixty, #3446)

Integrations:

• Ace, the JavaScript code editor now has syntax highlighting for PRQL. (@vanillajonathan, #3493)

Internal changes:

• Simplify & speed up lexer (@max-sixty, #3426, #3418)

New Contributors:

• @davidot, with #3450

0.9.4

0.9.4 — 2023-08-24

0.9.4 is a small release with some improvements and bug fixes in the compiler and prqlc. And, the documentation and CI are continually being improved.

This release has 110 commits from 9 contributors. Selected changes:

Features:

• Strings can be delimited with any odd number of quote characters. The logic for lexing quotes is now simpler and slightly faster. Escapes in single-quote-delimited strings escape single-quotes rather than double-quotes. (@max-sixty, #3274)

Fixes:

• S-strings within double braces now parse correctly (@max-sixty, #3265)

Documentation:

• New docs for strings (@max-sixty, #3281)

Web:

• Improve syntax highlighting for numbers in the book & website (@max-sixty, #3261)
• Add ClickHouse integration to docs (@max-sixty, #3251)

Integrations:

• prqlc no longer displays a prompt when piping a query into its stdin (@max-sixty, #3248).
• Add a minimal example for use prql-lib with Zig (@vanillajonathan, #3372)

Internal changes:

• Overhaul our CI to run a cohesive set of tests depending on the specific changes in the PR, and elide all others. This cuts CI latency to less than three minutes for most changes, and enables GitHub's auto-merge to wait for all relevant tests. It also reduces the CI time on merging to main, by moving some tests to only run on specific path changes or on our nightly run.
  We now have one label we can add to PRs to run more tests — pr-nightly. (@max-sixty, #3317 & others).
• Auto-merge PRs for backports or pre-commit updates (@max-sixty, #3246)
• Add a workflow to create an issue when the scheduled nightly workflow fails (@max-sixty, #3304)

New Contributors:

• @FinnRG, with #3292
• @sitiom, with #3353

0.9.3

0.9.3 is a small release, with mostly documentation, internal, and CI changes.

This release has 85 commits from 10 contributors.

We'd like to welcome @not-my-profile as someone who has helped with lots of internal refactoring in the past couple of weeks.

New Contributors:

• @vthriller, with #3171
• @postmeback, with #3216

0.9.2

0.9.2 is a hotfix release to fix an issue in the 0.9.0 & 0.9.1 release pipelines.

Check out the 0.9.0 release notes for a useful changelog.

0.9.1

0.9.1 is a hotfix release to fix an issue in the 0.9.0 release pipeline.

Check out the 0.9.0 release notes for a useful changelog.

0.9.0

0.9.0 is probably PRQL's biggest ever release. We have dialect-specific standard-libraries, a regex operator, an initial implementation of multiple-file projects & modules, lots of bug fixes, and many many internal changes.

We've made a few backward incompatible syntax changes. Most queries will work with a simple find/replace; see below for details.

The release has 421 commits from 12 contributors.

A small selection of the changes:

Language:

• The major breaking change is a new syntax for lists, which have been renamed to tuples, and are now represented with braces {} rather than brackets [].

  To convert previous PRQL queries to this new syntax simply change [ ... ] to { ... }.

  We made the syntax change to incorporate arrays. Almost every major language uses [] for arrays. We are adopting that convention — arrays use [], tuples will use {}. (Though we recognize that {} for tuples is also rare (Hi, Erlang!), but didn't want to further load parentheses with meaning.)

  Arrays are conceptually similar to columns — their elements have a single type. Array syntax can't contain assignments.

  As part of this, we've also formalized tuples as containing both individual items (select {foo, baz}), and assignments (select {foo=bar, baz=fuz}).

• Some significant changes regarding SQL dialects:

  • Operators and functions can be defined on per-dialect basis. (@aljazerzen, #2681)
  • Breaking: The sql.duckdb target supports DuckDB 0.8 (@eitsupi, #2810).
  • Breaking: The sql.hive target is removed (@eitsupi, #2837).

• New arithmetic operators. These compile to different function or operator depending on the target.

  • Breaking: Operator / now always performs floating division (@aljazerzen, #2684). See the Division docs for details.

  • Truncated integer division operator // (@aljazerzen, #2684). See the Division docs for details.

  • Regex search operator ~= (@max-sixty, #2458). An example:

    from tracks
    filter {name ~= "Love"}
    

    ...compiles to;

    SELECT
      *
    FROM
      tracks
    WHERE
      REGEXP(name, 'Love')

    ...though the exact form differs by dialect; see the Regex docs for more details.

• New aggregation functions: every, any, average, and concat_array. Breaking: Remove avg in favor of average.

• Breaking: We've changed our function declaration syntax to match other declarations. Functions were one of the first language constructs in PRQL, and since then we've added normal declarations there's no compelling reason for functions to be different.

  let add = a b -> a + b
  

  Previously, this was:

  func add a b -> a + b
  

• Experimental modules, which allow importing declarations from other files. Docs are forthcoming.

• Relation literals create a relation (a "table") as an array of tuples. This example demonstrates the new syntax for arrays [] and tuples {}. (@aljazerzen, #2605)

  from [{a=5, b=false}, {a=6, b=true}]
  filter b == true
  select a
  

• this can be used to refer to the current pipeline, for situations where plain column name would be ambiguous:

  from x
  derive sum = my_column
  select this.sum   # does not conflict with `std.sum`
  

  Within a join transform, there is also a reference to the right relation: that.

• Breaking: functions count, rank and row_number now require an argument of the array to operate on. In most cases you can directly replace count with count this. The non_null argument of count has been removed.

Features:

• We've changed how we handle colors.

  Options::color is deprecated and has no effect. Code which consumes prql_compiler::compile should instead accept the output with colors and use a library such as anstream to handle the presentation of colors. To ensure minimal disruption, prql_compiler will currently strip color codes when a standard environment variable such as CLI_COLOR=0 is set or when it detects stderr is not a TTY.

  We now use the anstream library in prqlc & prql-compiler.

  (@max-sixty, #2773)

• prqlc can now show backtraces when the standard backtrace env var (RUST_BACKTRACE) is active. (@max-sixty, #2751)

Fixes:

• Numbers expressed with scientific notation — 1e9 — are now handled correctly by the compiler (@max-sixty, #2865).

Integrations:

• prql-python now provides type hints (@philpep, #2912)

Internal changes:

• Annotations in PRQL. These have limited support but are currently used to specify binding strengths. They're modeled after Rust's annotations, but with @ syntax, more similar to traditional decorators. (#2729)

  @{binding_strength=11}
  let mod = l r -> s"{l} % {r}"
  

• Remove BigQuery's special handling of quoted identifiers, now that our module system handles its semantics (@max-sixty, #2609).

• ClickHouse is tested in CI (@eitsupi, #2815).

New Contributors:

• @maxmcd, with #2533
• @khoa165, with #2876
• @philpep, with #2912
• @not-my-profile, with #2971

0.8.1

0.8.1 is a small release with a new list-targets command in prqlc, some documentation improvements, and some internal improvements.

This release has 41 commits from 8 contributors.

From the broader perspective of the project, we're increasing the relative prioritization of it being easy for folks to actually use PRQL — either with existing tools, or a tool we'd build. We'll be thinking about & discussing the best way to do that over the next few weeks.