Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dvcfs: handle NotADirectoryError from datafs.ls #9746

Merged
merged 3 commits into from
Jul 19, 2023
Merged

Conversation

dberenbaum
Copy link
Collaborator

@dberenbaum dberenbaum commented Jul 19, 2023

Before dvc_fs.ls(path) without dvc.lock would return path/path. Now it won't return anything since dvc_fs has no info about what's in that path.

Fixes #9745

@dberenbaum dberenbaum requested a review from efiop July 19, 2023 15:57
@dberenbaum
Copy link
Collaborator Author

Any idea for where to test this?

@codecov
Copy link

codecov bot commented Jul 19, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.01 ⚠️

Comparison is base (ea5f7d4) 90.48% compared to head (e5d6c6e) 90.47%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9746      +/-   ##
==========================================
- Coverage   90.48%   90.47%   -0.01%     
==========================================
  Files         480      480              
  Lines       36528    36534       +6     
  Branches     5252     5252              
==========================================
+ Hits        33051    33053       +2     
- Misses       2885     2887       +2     
- Partials      592      594       +2     
Impacted Files Coverage Δ
dvc/fs/dvc.py 95.18% <100.00%> (ø)
tests/unit/fs/test_dvc.py 99.45% <100.00%> (-0.55%) ⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@efiop
Copy link
Contributor

efiop commented Jul 19, 2023

For the record: I'll add a test, no worries. Thank you! 🙏

Comment on lines +165 to +172
def test_ls_dirty(tmp_dir, dvc):
tmp_dir.dvc_gen({"data": "data"})
(tmp_dir / "data").unlink()

tmp_dir.gen({"data": {"foo": "foo", "bar": "bar"}})

fs = DVCFileSystem(repo=dvc)
assert set(fs.ls("data")) == {"data/foo", "data/bar"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little different from what's described in the issue (because there is dvc.lock there), but it is the same situation of having a dirty workspace compared to what is recorded (or missing like with missing dvc.lock) in dvc files.

@efiop efiop changed the title dvc_fs: ignore NotADirectoryError in ls dvcfs: handle NotADirectoryError from datafs.ls Jul 19, 2023
@efiop efiop added the bugfix fixes bug label Jul 19, 2023
@efiop
Copy link
Contributor

efiop commented Jul 19, 2023

Remote Plugin Tests failure is unrelated, looks like has something to do with today's pyyaml release. Will take a look.

yaml/pyyaml#601

EDIT: fixed in iterative/dvc-s3#46

@efiop efiop marked this pull request as ready for review July 19, 2023 17:54
@efiop efiop merged commit 41194b7 into main Jul 19, 2023
19 of 20 checks passed
@efiop efiop deleted the ls-not-a-dir-error branch July 19, 2023 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix fixes bug
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

'.../plots/metrics/metrics' - file type error
2 participants