Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Extend recognised EXTRACT and DATE_PART SQL part abbreviations #16767

Merged

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Jun 6, 2024

Consolidated parsing code for SQL EXTRACT and DATE_PART (also aligned the allowed INTERVAL sub-second abbreviations) ensuring consistent support across all functions (previously only DATE_PART supported part-extraction using all of the standard abbreviations, and INTERVAL allowed some non-standard shortcuts for ms/us/ns).

The following are all now supported across both part/extract functions, and were validated against a local PostgreSQL instance; interval-parsing abbreviations have also been aligned and tested with the below:

* "millennium" | "millennia"
* "century" | "centuries"
* "decade" | "decades"
* "isoyear"
* "year" | "years" | "y"
* "quarter" | "quarters"
* "month" | "months" | "mon" | "mons"
* "dayofyear" | "doy"
* "dayofweek" | "dow"
* "isoweek" | "week"
* "isodow"
* "day" | "days" | "d"
* "hour" | "hours" | "h"
* "minute" | "minutes" | "mins" | "min" | "m"
* "second" | "seconds" | "sec" | "secs" | "s"
* "millisecond" | "milliseconds" | "ms"
* "microsecond" | "microseconds" | "us"
* "nanosecond" | "nanoseconds" | "ns"
* "timezone"
* "time"
* "epoch"

Note: DATE_PART parameter order was also fixed; should be "DATE_PART(part, col)".

Example

from datetime import datetime
import polars as pl

df = pl.DataFrame({
    "dt": [
        datetime(1969, 12, 31, 1, 2, 3),
        datetime(2026, 8, 22, 4, 5, 6),
        datetime(2077, 2, 10, 7, 8, 9),
    ]
})

df.sql("""
  SELECT
    DATE_PART('years',dt) AS y1,
    EXTRACT(y FROM dt)    AS y2,
    DATE_PART('h',dt)     AS h,
    EXTRACT(min FROM dt)  AS m,
    DATE_PART('secs',dt)  AS s,
  FROM self
""")
# shape: (3, 5)
# ┌──────┬──────┬─────┬─────┬─────┐
# │ y1   ┆ y2   ┆ h   ┆ m   ┆ s   │
# │ ---  ┆ ---  ┆ --- ┆ --- ┆ --- │
# │ i32  ┆ i32  ┆ i8  ┆ i8  ┆ i8  │
# ╞══════╪══════╪═════╪═════╪═════╡
# │ 1969 ┆ 1969 ┆ 1   ┆ 2   ┆ 3   │
# │ 2026 ┆ 2026 ┆ 4   ┆ 5   ┆ 6   │
# │ 2077 ┆ 2077 ┆ 7   ┆ 8   ┆ 9   │
# └──────┴──────┴─────┴─────┴─────┘

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels Jun 6, 2024
@alexander-beedie alexander-beedie added the A-sql Area: Polars SQL functionality label Jun 6, 2024
@alexander-beedie alexander-beedie force-pushed the sql-improve-extract-parsing branch from a54e4bb to 19632f9 Compare June 6, 2024 10:43
Copy link

codecov bot commented Jun 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.37%. Comparing base (38149d6) to head (4d7bdb3).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16767      +/-   ##
==========================================
- Coverage   81.39%   81.37%   -0.03%     
==========================================
  Files        1424     1424              
  Lines      187623   187598      -25     
  Branches     2691     2691              
==========================================
- Hits       152719   152650      -69     
- Misses      34409    34453      +44     
  Partials      495      495              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alexander-beedie alexander-beedie force-pushed the sql-improve-extract-parsing branch from 19632f9 to 4d7bdb3 Compare June 8, 2024 10:04
@ritchie46 ritchie46 merged commit f548041 into pola-rs:main Jun 9, 2024
29 checks passed
@alexander-beedie alexander-beedie deleted the sql-improve-extract-parsing branch June 9, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sql Area: Polars SQL functionality enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants