v0.4.0

What's Changed

• Feature: Add support for Python 3.12, latest versions of Dask by @johncmerfeld in #127
• Feature: adds a --set flag to the cli to enable overriding values in compiled earthmover.yml by @tomreitz in #128
• Bugfix: remove excel source pyarrow dependency by @sleblanc23 in #126

Full Changelog: v0.3.8...v0.4.0

v0.3.8

What's Changed

• Hotfix: Refactor Jinja Destination Headers and Footers by @jayckaiser in #124

Full Changelog: v0.3.7...v0.3.8

v0.3.7

What's Changed

• Bugfix: union fails with duplicate columns by @jayckaiser in #122
• implementing a limit_rows operation by @tomreitz in #121
• add support for a require_rows boolean or non-negative int on any node by @tomreitz in #120
• add support for Jinja in a destination node header and footer by @tomreitz in #119

Full Changelog: v0.3.6...v0.3.7

v0.3.6

What's Changed

• feature: add json_array_agg function to group_by operation by @johncmerfeld in #112
• feature: select all columns using "*" in modify_columns operation by @bcadet2 in #110
• internal: set working directory to the location of the earthmover.yaml file by @sleblanc23 in #113
• documentation: add information on earthmover init and earthmover clean to the README by @johncmerfeld in #114
• bugfix: fix bug with earthmover clean that could have removed earthmover.yaml files by @johncmerfeld in #114

Full Changelog: v0.3.5...v0.3.6

v0.3.5

What's Changed

• feature: add earthmover init command to initialize a new sample project by @johncmerfeld in #107
• internal: expand test run to include the new debug and flatten operations, as well as a nested JSON source file by @tomreitz in #106
• internal: improve customization in write behavior in new file destinations by @jayckaiser in #109
• bugfix: Fix bug when writing null values in FileDestination by @jayckaiser in #109

Full Changelog: v0.3.4...v0.3.5

v0.3.4

What's Changed

• Hotfix: Resolve race-condition bug when writing to disk using FileDestination by @jayckaiser in #108

Full Changelog: v0.3.3...v0.3.4

v0.3.3

What's Changed

• Hotfix: Resolve incompatible dependencies by @johncmerfeld in #103
• Hotfix: Nested JSON not working when rendering destination templates by @tomreitz in #104

Full Changelog: v0.3.2...v0.3.3

v0.3.2

What's Changed

• feature: Add DebugOperation for logging data head, tail, columns, or metadata midrun
• feature: Add FlattenOperation for splitting and exploding string columns into values
• feature: Add optional 'fill_missing_columns' field to UnionOperation to fill disjunct columns with nulls, instead of raising an error (default False)
• feature: Add git_auth_timeout config when entering Git credentials during package composition
• feature: Add earthmover clean command that removes local project artifacts
• feature: only output compiled template during earthmover compile
• feature: Render full row into JSON lines when template is undefined in FileDestination
• Many bugfixes and compile improvements

Full Changelog: v0.3.1...v0.3.2

v0.3.1

What's Changed

• allow any ordering of Transformations during graph-building in compile by @jayckaiser
• only create a /packages dir when earthmover deps succeeds by @jayckaiser
• explain earthmover_compiled.yaml in README.md by @sleblanc23

Full Changelog: v0.3.0...v0.3.1

earthmover 0.3.0

What's Changed

• feature: add project composition using packages keyword in template file (see README)
• feature: add installation extras for optional libraries, and improve error logging to notify which is missing
• feature: GroupByWithRankOperation cumulatively sums record counts by group-by columns
• feature: setting log_level: DEBUG in template configs or setting debug: True for a node displays the head of the node mid-run
• feature: add optional_fields key to all Sources to add optional empty columns when missing from schema
• feature: add optional ignore_errors and exact_match boolean flags to DateFormatOperation
• internal: remove attempted directory-hashing when a source is a directory (i.e., Parquet)
• internal: Remove unused group_by_with_count and group_by_with_agg operations

Full Changelog: v0.2.1...v0.3.0

Note: This version has slightly different packaging requirements than v0.2.1. Please make sure to re-install the package if using locally.