Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

concat_list creates a nested list #9949

Closed
2 tasks done
tim-x-y-z opened this issue Jul 18, 2023 · 1 comment
Closed
2 tasks done

concat_list creates a nested list #9949

tim-x-y-z opened this issue Jul 18, 2023 · 1 comment
Labels
bug Something isn't working invalid A bug report that is not actually a bug python Related to Python Polars

Comments

@tim-x-y-z
Copy link
Contributor

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

>>> df=pl.DataFrame({"a": [[1,2,3], [3,4,5]], "b": [0.5, 0.1], "group":[1,2]})
>>> df.explode("a").with_columns(new_col=pl.col("a") * pl.col("b")).groupby("group").agg(pl.concat_list("a"), pl.first("b"), pl.concat_list("new_col"))
shape: (2, 4)
┌───────┬─────────────────┬─────┬───────────────────────┐
│ groupabnew_col               │
│ ------------                   │
│ i64list[list[i64]] ┆ f64list[list[f64]]       │
╞═══════╪═════════════════╪═════╪═══════════════════════╡
│ 1     ┆ [[1], [2], [3]] ┆ 0.5 ┆ [[0.5], [1.0], [1.5]] │
│ 2     ┆ [[3], [4], [5]] ┆ 0.1 ┆ [[0.3], [0.4], [0.5]] │
└───────┴─────────────────┴─────┴───────────────────────┘

Issue description

In some ways, related to #9948, as my goal is to multiply a list by another column value. This stack overflow questions did the same: https://stackoverflow.com/questions/74372173/python-polars-how-to-multiply-each-element-in-a-list-with-a-value-in-a-differen
However, the api seems to have cahnge since then as list is not callable anymore, being now a namespace only seemingly.
I assumed the replacement was concat_list which doesn't work as expected. I may be wrong that this is not what concat_list is for, but from the example here it seemed it is.

Expected behavior

Just returns a list of the object that it is concat. ie in this case, just return list[f64]

Installed versions

>>> pl.show_versions()
--------Version info---------
Polars:              0.18.7
Index type:          UInt32
Platform:            macOS-13.4.1-x86_64-i386-64bit
Python:              3.11.4 (main, Jun 20 2023, 16:59:59) [Clang 14.0.3 (clang-1403.0.22.14.1)]

----Optional dependencies----
adbc_driver_sqlite:  <not installed>
connectorx:          0.3.2a7
deltalake:           <not installed>
fsspec:              <not installed>
matplotlib:          3.7.1
numpy:               1.25.1
pandas:              2.0.3
pyarrow:             12.0.1
pydantic:            1.10.11
sqlalchemy:          1.4.49
xlsx2csv:            <not installed>
xlsxwriter:          <not installed>```

</details>
@tim-x-y-z tim-x-y-z added bug Something isn't working python Related to Python Polars labels Jul 18, 2023
@stinodego
Copy link
Contributor

We changed the behavior of agg some time ago.

You can do:

import polars as pl

df = pl.DataFrame({"a": [[1, 2, 3], [3, 4, 5]], "b": [0.5, 0.1], "group": [1, 2]})

result = (
    df.explode("a")
    .with_columns(new_col=pl.col("a") * pl.col("b"))
    .groupby("group")
    .agg("new_col")
)

No need to manually convert to a list.

@stinodego stinodego added the invalid A bug report that is not actually a bug label Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working invalid A bug report that is not actually a bug python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

2 participants