Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileContainer: add .items(), deprecate __iter__ #128

Merged
merged 3 commits into from
Nov 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,12 @@
this column from the underlying `DataFrame` ([#113](https://github.com/mathause/filefinder/pull/113)).
- Added `meta` and `paths` properties to `FileContainer` which allow to iterate over them
([#121](https://github.com/mathause/filefinder/pull/121)).
- Deprecated `combine_by_key` ([#115](https://github.com/mathause/filefinder/pull/115)).
- Added `items()` method to `FileContainer`, which iterates over `path, meta`
([#128](https://github.com/mathause/filefinder/pull/128)).
- Deprecated iterating over `FileContainer`, use `.paths`, `.meta` or `items()` instead
([#128](https://github.com/mathause/filefinder/pull/128)).
- Deprecated `combine_by_key`, create a `pd.MultiIndex` instead
([#115](https://github.com/mathause/filefinder/pull/115)).
- Added the number of paths to the repr ([#116](https://github.com/mathause/filefinder/pull/116)).

- Explicitly test on python 3.13 ([#103](https://github.com/mathause/filefinder/pull/103)).
Expand Down
13 changes: 12 additions & 1 deletion filefinder/_filefinder.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
import os
import re
import warnings
from typing import Any
from typing import Any, Generator

import numpy as np
import pandas as pd
Expand Down Expand Up @@ -592,6 +592,13 @@ def __init__(self, df: pd.DataFrame):

def __iter__(self):

warnings.warn(
"iterating over a `FileContainer` is deprecated. Use ``.paths``, ``.meta``"
" or ``.items() instead.",
FutureWarning,
stacklevel=2,
)

for index, element in self.df.iterrows():
yield index, element.to_dict()

Expand All @@ -616,6 +623,10 @@ def meta(self) -> list[dict[str, Any]]:
def paths(self) -> list[str]:
return self.df.index.to_list()

def items(self) -> Generator[tuple[str, dict[str, Any]], None, None]:
for index, element in self.df.iterrows():
yield index, element.to_dict()

def combine_by_key(self, keys=None, sep="."):
warnings.warn(
"`combine_by_key` has been deprecated and will be removed in a future version",
Expand Down
25 changes: 22 additions & 3 deletions filefinder/tests/test_filecontainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def test_empty_filecontainer():
assert len(fc) == 0

with pytest.raises(StopIteration):
next(iter(fc))
next(iter(fc.items()))


def test_filecontainer(example_df, example_fc):
Expand All @@ -51,11 +51,30 @@ def test_filecontainer(example_df, example_fc):
def test_fc_iter(example_df, example_fc):

# test one manually
path, meta = next(iter(example_fc))
with pytest.warns(
FutureWarning, match="iterating over a `FileContainer` is deprecated"
):
path, meta = next(iter(example_fc))

assert path == "file0"
assert meta == {"model": "a", "scen": "d", "res": "r"}

with pytest.warns(
FutureWarning, match="iterating over a `FileContainer` is deprecated"
):
result = list(example_fc)
expected = list(zip(example_df.index.to_list(), example_df.to_dict("records")))
assert result == expected


def test_fc_items(example_df, example_fc):

# test one manually
path, meta = next(iter(example_fc.items()))
assert path == "file0"
assert meta == {"model": "a", "scen": "d", "res": "r"}

result = list(example_fc)
result = list(example_fc.items())
expected = list(zip(example_df.index.to_list(), example_df.to_dict("records")))
assert result == expected

Expand Down