Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Polars to v0.41.3 #917

Merged
merged 32 commits into from
Jul 24, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ae29cb6
WIP: making things run, but break some tests
philss May 27, 2024
fcd8678
fix compilation
lkarthee Jun 1, 2024
c702a2b
Merge remote-tracking branch 'upstream/main' into ps-bump-polars-to-v…
lkarthee Jun 24, 2024
9d415dc
bump polars to 0.41
lkarthee Jun 24, 2024
c0b9763
Merge remote-tracking branch 'upstream/main' into ps-bump-polars-to-v…
lkarthee Jul 14, 2024
66ee582
Working version 0.41.3
lkarthee Jul 14, 2024
e79c0f6
don't pass keys when cross-joining
billylanchantin Jul 14, 2024
6f463ff
Run "cargo update"
philss Jul 15, 2024
a7759b7
Changes that fix some tests
philss Jul 16, 2024
0edc5d8
Fix args for pivot_longer
philss Jul 17, 2024
bc234bf
Fix order of arguments for pivot_stable
philss Jul 17, 2024
c80e6dd
Changes in pivot_wider with repeated columns
philss Jul 18, 2024
048d46a
Explicitly use literal expressions with their types
philss Jul 18, 2024
bc4e2a3
Cast to float 64 series for window functions with weights
philss Jul 18, 2024
4274c88
Use aggregate function for polars query
philss Jul 18, 2024
a6d923b
Improve check of weights
philss Jul 18, 2024
2644d27
Attempt to fix "pow/2" after Polars changes
philss Jul 18, 2024
54991c0
Revert "Attempt to fix "pow/2" after Polars changes"
philss Jul 19, 2024
cb3886c
Adapt pow/2 to follow the same rules from Polars
philss Jul 19, 2024
7c2baeb
Capture errors when inspecting DFs at PolarsBackend
philss Jul 19, 2024
878b422
Cleaning comments and add missing option
philss Jul 19, 2024
0b1f39e
Rewrite CloudWriter to use BufWriter from object_store
philss Jul 20, 2024
15f8521
Update object_store to v0.10.2
philss Jul 22, 2024
be9a236
WIP: enable the "to_parquet/2" test case with new object_store
philss Jul 22, 2024
142155b
Revert "WIP: enable the "to_parquet/2" test case with new object_store"
philss Jul 22, 2024
739bc82
Ensure the CloudWriter finishes its job after writers
philss Jul 22, 2024
d64af7e
WIP: saving PoC that didn't work
philss Jul 23, 2024
c8aabfe
Revert "WIP: saving PoC that didn't work"
philss Jul 23, 2024
bb04f29
Raise for when saving lazy df as parquet to cloud using streaming
philss Jul 23, 2024
d7b6010
Add changes to the changelog
philss Jul 24, 2024
64370e2
Last round of "cargo update"
philss Jul 24, 2024
1ce99b9
Update CHANGELOG.md
philss Jul 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/explorer/data_frame.ex
Original file line number Diff line number Diff line change
Expand Up @@ -2778,7 +2778,7 @@ defmodule Explorer.DataFrame do
#Explorer.DataFrame<
Polars[3 x 2]
a string ["a", "b", "c"]
b f64 [1.0, 4.0, 9.0]
b s64 [1, 4, 9]
>

It's possible to "reuse" a variable for different computations:
Expand Down
7 changes: 6 additions & 1 deletion lib/explorer/polars_backend/data_frame.ex
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,11 @@ defmodule Explorer.PolarsBackend.DataFrame do

{columns, with_projection} = column_names_or_projection(columns)

dtypes_list =
if not Enum.empty?(dtypes) do
Map.to_list(dtypes)
end

df =
Native.df_load_csv(
contents,
Expand All @@ -207,7 +212,7 @@ defmodule Explorer.PolarsBackend.DataFrame do
delimiter,
true,
columns,
Map.to_list(dtypes),
dtypes_list,
encoding,
nil_values,
parse_dates,
Expand Down
4 changes: 2 additions & 2 deletions lib/explorer/series.ex
Original file line number Diff line number Diff line change
Expand Up @@ -3616,7 +3616,7 @@ defmodule Explorer.Series do
iex> Explorer.Series.pow(s, 3)
#Explorer.Series<
Polars[3]
f64 [8.0, 64.0, 216.0]
s64 [8, 64, 216]
>

iex> s = [2, 4, 6] |> Explorer.Series.from_list()
Expand Down Expand Up @@ -3657,7 +3657,7 @@ defmodule Explorer.Series do
defp cast_to_pow({:f, l}, {:f, r}), do: {:f, max(l, r)}
defp cast_to_pow({:f, l}, {n, _}) when K.in(n, [:u, :s]), do: {:f, l}
defp cast_to_pow({n, _}, {:f, r}) when K.in(n, [:u, :s]), do: {:f, r}
defp cast_to_pow({n, _}, {:s, _}) when K.in(n, [:u, :s]), do: {:f, 64}
defp cast_to_pow({n, _}, {:s, _}) when K.in(n, [:u, :s]), do: {:s, 64}
philss marked this conversation as resolved.
Show resolved Hide resolved
defp cast_to_pow(_, _), do: nil

@doc """
Expand Down
Loading
Loading