Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Implement snapshot saving #57

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

shilorigins
Copy link
Collaborator

@shilorigins shilorigins commented Jul 16, 2024

Description

  • Implement Snapshot saving (closes Implement saving a Snapshot #32 )
    • include helper methods Client._build_snapshot and Client._gather_pvs
  • Extend Client._gather_data to accept any type of Entry, and make iterative rather than recursive

Motivation and Context

Client.snap itself is simple: it gathers all of the PVs in an entry, adds in the meta PVs, asynchronously fetches data for each PV, and then builds the snapshot. This method is named snap because save is currently used for the method that writes entries to the backend.

Since we want a function to gather all of the PVs from an entry, I decided to avoid code duplication by adding that functionality to Client._gather_data. This extension needed some messy code to handle all of the parameter permutations, which I was able to simplify by changing the method to be iterative. I also use hasattr rather than type checks to avoid length conditionals. I included Client._gather_pvs as a convenience method.

Fetching the data is straightforward due to ControlLayer._get_list.

Building the snapshot uses a simple depth-first process.

Alternative Designs

One major alternative was to have each Entry subclass implement method to gather its PVs. This conflicts with the current system because the entry classes don't have access to the backend to process UUIDs.

How Has This Been Tested?

New and updated tests in test_client.py

Where Has This Been Documented?

This PR, docstrings

Pre-merge checklist

  • Code works interactively
  • Code follows the style guide
  • Code contains descriptive docstrings, including context and API
  • New/changed functions and methods are covered in the test suite where possible
  • Test suite passes locally
  • Test suite passes on GitHub Actions
  • Ran docs/pre-release-notes.sh and created a pre-release documentation page
  • Pre-release docs include context, functional descriptions, and contributors as appropriate

@shilorigins shilorigins marked this pull request as ready for review July 19, 2024 17:47
@shilorigins shilorigins changed the title WIP: Implement snapshot saving ENH: Implement snapshot saving Jul 19, 2024
superscore/client.py Outdated Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/tests/conftest.py Outdated Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/tests/test_client.py Show resolved Hide resolved
superscore/tests/test_client.py Outdated Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/tests/test_client.py Outdated Show resolved Hide resolved
@shilorigins shilorigins force-pushed the devagr/save-snapshot branch 2 times, most recently from e5e7fd3 to 4ff42b6 Compare July 22, 2024 16:27
Copy link
Contributor

@tangkong tangkong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more thoughts. I think we need to think about how we handle meta_pvs, because at the very least, they are never initialized / configured in our current framework.

"""
logger.debug("Gathering PVs to read")
pvs = self._gather_pvs(entry)
pvs.extend(Collection.meta_pvs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm realizing now that we're defining meta PVs as a class variable, which prevents us from properly representing capturing changes to the meta_pv list. More concretely, if we:

  • define one collection expecting a set of meta_pvs
  • change the (global) list of meta_pvs
  • take a snapshot with that same old collection

The new snapshot will pull the new meta_pvs, and might inadvertently be missing / duplicate PVs. This could be the behavior we want, but I think we need to find a way to make it more obvious if we do. Alternately we record meta_pvs as an instance variable (not class variable), and fill it with the global configuration on creation.

We also need to determine how we capture this in the Filestore backend, as currently this won't be recorded properly. This probably needs to be a separate dictionary / store external to the Root tree that gets stored

superscore/client.py Outdated Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/client.py Show resolved Hide resolved
superscore/client.py Outdated Show resolved Hide resolved
superscore/client.py Show resolved Hide resolved
Copy link
Contributor

@tangkong tangkong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Let's get this in 👍

The test didn't consider Readback handling because sample_database doesn't
include any Readbacks. Rather than modify sample_database and possibly
affect other tests, I've added a minimal fixture for considering a single
Setpoint-Readback pair.
@shilorigins shilorigins merged commit 6d44dc9 into pcdshub:master Aug 2, 2024
11 checks passed
@shilorigins shilorigins deleted the devagr/save-snapshot branch August 2, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement saving a Snapshot
3 participants