Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCAL-991: support passing a file to rdm.open #453

Merged
merged 2 commits into from
Jan 23, 2025

Conversation

braingram
Copy link
Collaborator

@braingram braingram commented Jan 22, 2025

Resolves RCAL-991

Closes #449

This PR restores (unofficial) support for passing file-like objects to rdm.open and updates docstrings to make it "official".

Tasks

  • Update or add relevant roman_datamodels tests.
  • Update relevant docstrings and / or docs/ page.
  • Does this PR change any API used downstream? (If not, label with no-changelog-entry-needed.)
News fragment change types:
  • changes/<PR#>.feature.rst: new feature
  • changes/<PR#>.bugfix.rst: fixes an issue
  • changes/<PR#>.doc.rst: documentation change
  • changes/<PR#>.removal.rst: deprecation or removal of public API
  • changes/<PR#>.misc.rst: infrastructure or miscellaneous change

Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.24%. Comparing base (087a60d) to head (45fbf66).
Report is 98 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #453      +/-   ##
==========================================
- Coverage   97.56%   97.24%   -0.32%     
==========================================
  Files          30       37       +7     
  Lines        2788     3302     +514     
==========================================
+ Hits         2720     3211     +491     
- Misses         68       91      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@braingram braingram changed the title support passing a file to rdm.open RCAL-991: support passing a file to rdm.open Jan 22, 2025
@braingram braingram marked this pull request as ready for review January 22, 2025 17:57
@braingram braingram requested a review from a team as a code owner January 22, 2025 17:57
Copy link
Collaborator

@schlafly schlafly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks Brett. I think your choice of "do nothing if it's not a path" is probably the right one, though it is a bit awkward that rdm.open(open(path)) and rdm.open(path) will do different things. Part of the motivation here was wanting to give files the right names by default if we we opened files from a particular name. Is there an appropriate hint for an S3 file that we should use instead, or should we give up? Thanks!

@braingram
Copy link
Collaborator Author

Looks good to me, thanks Brett. I think your choice of "do nothing if it's not a path" is probably the right one, though it is a bit awkward that rdm.open(open(path)) and rdm.open(path) will do different things.

Thanks!

This awkwardness exists for AsdfFile instances as well (on main) where rdm.open(path) differs from rdm.open(asdf.open(path)) (the first updates meta.filename the second does not).

Part of the motivation here was wanting to give files the right names by default if we we opened files from a particular name. Is there an appropriate hint for an S3 file that we should use instead, or should we give up? Thanks!

Both with this PR (for file-like input as is the case for s3fs returned objects) and with main (for AsdfFile instances returned by asdf.open) I say we give up since we'd have to make assumptions about what is the filename. These assumptions would complicate the code and would be incomplete (if an AsdfFile instance is created in memory or if the file-like object is an io.BytesIO instance neither will have a filename). By not attempting to update meta.filename for these (atypical) inputs we will hopefully keep things simple if the meta.filename handling issues in stpipe are ever taken on.

@schlafly schlafly merged commit ffd56b1 into spacetelescope:main Jan 23, 2025
17 of 19 checks passed
@braingram braingram deleted the open_file branch January 23, 2025 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roman_datamodels can no longer open files from S3 buckets
2 participants