Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SchemaFieldNotFoundError with rename and select combination #20407

Open
2 tasks done
c0dearm opened this issue Dec 22, 2024 · 1 comment
Open
2 tasks done

SchemaFieldNotFoundError with rename and select combination #20407

c0dearm opened this issue Dec 22, 2024 · 1 comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@c0dearm
Copy link

c0dearm commented Dec 22, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

pl.LazyFrame({"TEST": [1]}).rename({"NOT FOUND": "TEST"}, strict=False).select("TEST").collect()

Log output

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/aitor/Projects/.venv/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 2031, in collect
    return wrap_df(ldf.collect(callback))
                   ^^^^^^^^^^^^^^^^^^^^^
polars.exceptions.SchemaFieldNotFoundError: NOT FOUND

Issue description

This combination of rename + select causes the issue shown above.

Expected behavior

Polars complains about not finding the NOT FOUND field, but the select is on the TEST field and the renaming was not strict, so it shouldn't complain and should successfully return a DataFrame with the TEST column.

Installed versions

--------Version info---------
Polars:              1.17.1
Index type:          UInt32
Platform:            macOS-15.0.1-arm64-arm-64bit
Python:              3.12.6 (main, Sep  6 2024, 19:03:47) [Clang 16.0.0 (clang-1600.0.26.3)]
LTS CPU:             False

----Optional dependencies----
adbc_driver_manager  <not installed>
altair               <not installed>
boto3                1.35.81
cloudpickle          3.1.0
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               <not installed>
gevent               <not installed>
google.auth          2.37.0
great_tables         <not installed>
matplotlib           3.10.0
nest_asyncio         <not installed>
numpy                2.2.1
openpyxl             <not installed>
pandas               2.2.3
pyarrow              18.1.0
pydantic             2.10.4
pyiceberg            <not installed>
sqlalchemy           2.0.36
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>
@c0dearm c0dearm added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Dec 22, 2024
@c0dearm
Copy link
Author

c0dearm commented Dec 22, 2024

The workaround for now is to do .collect(projection_pushdown=False), but I say we should fix it for good! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

1 participant