Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Cannot import pandas2.2.3 in free-threading python3.13.0 #60224

Closed
2 of 3 tasks
shinyano opened this issue Nov 7, 2024 · 6 comments
Closed
2 of 3 tasks

BUG: Cannot import pandas2.2.3 in free-threading python3.13.0 #60224

shinyano opened this issue Nov 7, 2024 · 6 comments
Labels
Bug Build Library building on various platforms Needs Triage Issue that has not been reviewed by a pandas team member

Comments

@shinyano
Copy link

shinyano commented Nov 7, 2024

Pandas version checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of pandas.
  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

python3.13t -m venv myenv
myenv\Scripts\activate

(myenv)pip install pandas==2.2.3
(myenv)python
(myenv)>>> import pandas

Issue Description

According to release note, pandas2.2.3 can be used in free-threading python. But when importing pandas(2.2.3) in python3.13t on windows11, python would exit without output or error message:

E:\>python3.13t -m venv myenv
E:\>myenv\Scripts\activate
(myenv) E:\>pip install pandas==2.2.3
Collecting pandas==2.2.3
  Using cached pandas-2.2.3-cp313-cp313t-win_amd64.whl
Collecting numpy>=1.26.0 (from pandas==2.2.3)
  Using cached numpy-2.1.3-cp313-cp313t-win_amd64.whl.metadata (60 kB)
Collecting python-dateutil>=2.8.2 (from pandas==2.2.3)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas==2.2.3)
  Using cached pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.7 (from pandas==2.2.3)
  Using cached tzdata-2024.2-py2.py3-none-any.whl.metadata (1.4 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas==2.2.3)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Using cached numpy-2.1.3-cp313-cp313t-win_amd64.whl (12.6 MB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Using cached tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: pytz, tzdata, six, numpy, python-dateutil, pandas
Successfully installed numpy-2.1.3 pandas-2.2.3 python-dateutil-2.9.0.post0 pytz-2024.2 six-1.16.0 tzdata-2024.2

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip

(myenv) E:\>python
Python 3.13.0 experimental free-threading build (tags/v3.13.0:60403a5, Oct  7 2024, 09:53:29) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas

(myenv) E:\>

Then I built cpython with debug flag and tried again, the output is as follows:

(debugenv) E:\cpython\cpython-3.13.0\PCbuild\amd64>python
Python 3.13.0 experimental free-threading build (main, Nov  4 2024, 19:14:19) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import pandas
  File "E:\cpython\cpython-3.13.0\PCbuild\amd64\debugenv\Lib\site-packages\pandas\__init__.py", line 49, in <module>
    from pandas.core.api import (
    ...<62 lines>...
    )
  File "E:\cpython\cpython-3.13.0\PCbuild\amd64\debugenv\Lib\site-packages\pandas\core\api.py", line 1, in <module>
    from pandas._libs import (
    ...<4 lines>...
    )
  File "E:\cpython\cpython-3.13.0\PCbuild\amd64\debugenv\Lib\site-packages\pandas\_libs\__init__.py", line 16, in <module>
    import pandas._libs.pandas_parser  # isort: skip # type: ignore[reportUnusedImport]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: DLL load failed while importing pandas_parser: The specified module could not be found.

Expected Behavior

Pandas should be correctly imported.

Installed Versions

I can't use pandas in python3.13t, so it's a little hard to execute pd.show_versions().

@shinyano shinyano added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 7, 2024
@rhshadrach
Copy link
Member

cc @lithomas1

@rhshadrach rhshadrach added Build Library building on various platforms Multithreading Parallelism in pandas and removed Multithreading Parallelism in pandas labels Nov 7, 2024
@jorisvandenbossche
Copy link
Member

According to release note, pandas2.2.3 can be used in free-threading python.

That's true, although in practice this was only tested on Linux and MacOS, because at that time numpy only did have free-threading support for those platforms (at least only wheels for those platforms).

So we also didn't provide windows wheels with free-threading support.
I see in your output above:

Collecting pandas==2.2.3
  Using cached pandas-2.2.3-cp313-cp313t-win_amd64.whl

That's not a wheel that pandas provided (https://pypi.org/project/pandas/2.2.3/#files), so did you build that yourself locally?


NumPy recently started providing free-threaded wheels for Windows, and so now pandas will also be able to test Windows support. @lysnikolaou started doing that in #60146, although I don't see any code changed that were required to get it running, which should suggest that also pandas 2.2.3 already worked on Windows.

@lithomas1
Copy link
Member

lithomas1 commented Nov 7, 2024

Correct, no free-threading Windows binaries were uploaded yet.
(I think it might work if you compile from source though)

I agree with Joris that the downloaded wheel looks suspicious.

@shinyano
Copy link
Author

shinyano commented Nov 8, 2024

So we also didn't provide windows wheels with free-threading support. I see in your output above:

I deleted this wheel file and noticed pip install was building wheels from source code, which explains how I got this wheel file.

NumPy recently started providing free-threaded wheels for Windows, and so now pandas will also be able to test Windows support. @lysnikolaou started doing that in #60146, although I don't see any code changed that were required to get it running, which should suggest that also pandas 2.2.3 already worked on Windows.

Thanks for the information. That's a big help.
I'll close this for now and update if I manage to get pandas 2.2.3 working.

@shinyano shinyano closed this as completed Nov 8, 2024
@shinyano
Copy link
Author

shinyano commented Nov 8, 2024

UPDATE:
The wheel file(cp313t-win_amd64) built in #60146 works without any error. If anyone is interested, you can find it in Artifacts section which is in the bottom of Summary of the CI run Wheel Builder

@jorisvandenbossche
Copy link
Member

@shinyano Thanks for testing! That's good to hear it is working

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Build Library building on various platforms Needs Triage Issue that has not been reviewed by a pandas team member
Projects
None yet
Development

No branches or pull requests

4 participants