Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add two records: CreateDirectories and DirectoryExists. Automatically create directories based on this PV like in AD #102

Merged
merged 15 commits into from
Jun 5, 2024

Conversation

jwlodek
Copy link
Contributor

@jwlodek jwlodek commented Mar 22, 2024

Marking as draft until I can write tests and perform a full test with a real PandA system, but this works in my initial testing.

Designed to work just like the AD CreateDirectory system: https://github.com/areaDetector/ADCore/blob/631fb284dc06d52b9286e62418a80f7f3a7a05e2/docs/ADCore/NDPluginFile.rst?plain=1#L57-L64

The DirectoryExists record is also meant to behave the same as in areaDetector, and prevent attempting to write the HDF file if the target does not exist or is not writable by the user running the IOC process.

@jwlodek jwlodek marked this pull request as ready for review May 16, 2024 14:11
@jwlodek
Copy link
Contributor Author

jwlodek commented May 16, 2024

Ok, I think this is ready for review. All tests are passing + tests added for new conditions, and I also tested with a real PandA, everything behaves as expected.

Copy link
Contributor

@AlexanderWells-diamond AlexanderWells-diamond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a couple of changes that I think should be made, and then a lot of nitpicking comments.

I also am not the person to give the final approval to this, as I'm somewhat detached from this project these days, so please wait on @evalott100 to give final approval.

src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
tests/test_hdf_ioc.py Outdated Show resolved Hide resolved
tests/test_hdf_ioc.py Outdated Show resolved Hide resolved
Comment on lines +422 to +433
await caput(
hdf5_test_prefix + ":CreateDirectory",
create_depth,
wait=True,
)
await caput(
hdf5_test_prefix + ":HDFDirectory",
target_path,
datatype=DBR_CHAR_STR,
wait=True,
)
exists = await caget(hdf5_test_prefix + ":DirectoryExists")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to add timeout=TIMEOUT to all these ca* calls, just to ensure a known maximum termination time. I do note we haven't been particularly good about always doing it in other tests.

tests/test_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Show resolved Hide resolved
tests/test_hdf_ioc.py Outdated Show resolved Hide resolved
Copy link
Contributor

@AlexanderWells-diamond AlexanderWells-diamond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me now

@GDYendell
Copy link
Collaborator

Is the directory depth here absolute rather than relative to how much of the requested path already exists? i.e. if directory /1/2/3 exists and I want it to create directory /1/2/3/4/5, does CreateDirectory need to be to 2 or 5? I had assumed it would be 2.

@jwlodek
Copy link
Contributor Author

jwlodek commented May 17, 2024

The dir depth can be either or, depending on the sign - this is the same as the behavior of the same record in areaDetector.

Basically, you can consider that the value represents the index in the path split into directories below which directories are permitted to be created by the IOC.

So in your example of /1/2/3/4/5, a value of 2 would mean that we can create any directories starting with index 2, meaning ./3/4/5 can be created if they don't already exist. A value of -2 would mean two from the back, meaning that we could create ./4/5.

From the areaDetector file plugin docs:

The CreateDirectory record controls whether directories are created if they don't exist. If it is zero (default), no directories are created. If it is negative, then the absolute value is the maximum of directories that will be created (i.e. -1 will create a maximum of one directory to complete the path, -2 will create a maximum of 2 directories). If it is positive, then at least that many directories in the path must exist (i.e. a value of 1 will create all directories below the root directory and 2 will not create a directory in the root directory).

@jwlodek
Copy link
Contributor Author

jwlodek commented May 31, 2024

@evalott100 Could we get this merged and included in a new release? I'd like to make some dependent changes in ophyd-async to add the option to use this feature.

Copy link

codecov bot commented May 31, 2024

Codecov Report

Attention: Patch coverage is 91.37931% with 5 lines in your changes missing coverage. Please review.

Project coverage is 90.97%. Comparing base (c08325a) to head (0289178).

Files Patch % Lines
src/pandablocks_ioc/_hdf_ioc.py 91.37% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #102      +/-   ##
==========================================
+ Coverage   90.87%   90.97%   +0.10%     
==========================================
  Files           8        8              
  Lines        1227     1285      +58     
  Branches      194      206      +12     
==========================================
+ Hits         1115     1169      +54     
- Misses         74       78       +4     
  Partials       38       38              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@evalott100 evalott100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! This will be really useful. I'm happy to release once it's merged.

It'd maybe be good to add docs on this: https://pandablocks.github.io/PandABlocks-ioc/main/how-to/capture-hdf.html

A copy of the areaDetector description of CreateDirectory should suffice.

src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
src/pandablocks_ioc/_hdf_ioc.py Outdated Show resolved Hide resolved
if create_dir_depth < 0:
max_dirs_to_create = abs(create_dir_depth)
elif create_dir_depth > len(new_path.parents):
max_dirs_to_create = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a test covering line 561

@evalott100
Copy link
Contributor

evalott100 commented Jun 3, 2024

Run tests/regenerate_test_bobfiles.sh to regenerate the test screens

@jwlodek jwlodek requested a review from evalott100 June 4, 2024 21:05
Copy link
Contributor

@evalott100 evalott100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 🙂 Lgtm if it's been tested on a real panda

@evalott100 evalott100 merged commit 8e25d64 into PandABlocks:main Jun 5, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants