RuntimeError: Loop is Not Running for Parquet Send Bytes to Helper

[danyeaw]

Describe the bug
When I select a parquet file on S3, I'm getting a RuntimeError: Loop is not running when I try to send bytes to helper and then run helper.out.

jupyter-fsspec.yaml config file

sources:

  - name: "Conda Stats"
    path: "s3://anaconda-package-data/conda/hourly/"
    kwargs:
      anon: True

To Reproduce
Steps to reproduce the behavior:

1. Load the Conda Stats filesystem
2. Click 2025, 02
3. Right click on 2025-02-28.parquet, select Send Bytes to helper
4. In the notebook, execute the following:

from jupyter_fsspec import helper

helper.out

See error.

Expected behavior
Output of the bytes or a portion of the bytes.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: macOS
• Browser: Firefox
• Version: main

Additional context

----------------
Request [FAIL] made 0:00:02.220236s ago
Timestamp 2025-03-18T17:26:35.098507
................
Path: anaconda-package-data/conda/hourly/2025/02/2025-02-23.parquet
Data[:64] preview (total -1): <None>

.... ERROR! ....
Traceback (most recent call last):
  File "/Users/dyeaw/Projects/jupyter-fsspec/jupyter_fsspec/helper.py", line 155, in _request_bytes
    "value": filesys.open(path, mode="rb").read(),
             ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/fsspec/spec.py", line 1310, in open
    f = self._open(
        path,
    ...<4 lines>...
        **kwargs,
    )
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/s3fs/core.py", line 699, in _open
    return S3File(
        self,
    ...<11 lines>...
        size=size,
    )
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/s3fs/core.py", line 2238, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        s3,
        ^^^
    ...<6 lines>...
        size=size,
        ^^^^^^^^^^
    )
    ^
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/fsspec/spec.py", line 1884, in __init__
    self.size = self.details["size"]
                ^^^^^^^^^^^^
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/fsspec/spec.py", line 1897, in details
    self._details = self.fs.info(self.path)
                    ~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/fsspec/asyn.py", line 118, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/Users/dyeaw/miniconda3/envs/jupyter-fsspec/lib/python3.13/site-packages/fsspec/asyn.py", line 76, in sync
    raise RuntimeError("Loop is not running")
RuntimeError: Loop is not running

----------------

[martindurant]

The cause: the "manager" is designed to be called from within a server process, and sets backends to asynchronous=True if they support it. helper, however, if not calling them asynchronously.

I am a bit puzzled why the helper module instantiates the manager class. I thought the whole point of our server-client setup, was that the server would provide the kwargs to instantiate those filesystems locally. The kernel does not necessarily have access to the same config file that the server can see! That makes it equivalent (and possible redundant with) providing the "open" code snippet.

[danyeaw]

We talked about this offline. @ericsnekbytes proposal is to update _get_fs hook, so that we aren't constructing a new Manager.