Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to output ParameterNode as YAML #295

Draft
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

SylviaDu99
Copy link
Collaborator

WIP

fixes: #274

Added write_yaml function to output ParameterNode data to a YAML file; test_write_yaml test to produce a sample output

Copy link

codecov bot commented Oct 16, 2024

Codecov Report

Attention: Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.

Project coverage is 83.10%. Comparing base (cf53c6d) to head (d9a37d5).
Report is 20 commits behind head on master.

Files with missing lines Patch % Lines
policyengine_core/parameters/parameter_node.py 88.88% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #295      +/-   ##
==========================================
+ Coverage   79.73%   83.10%   +3.36%     
==========================================
  Files         192      193       +1     
  Lines       10057    10115      +58     
  Branches     1311     1051     -260     
==========================================
+ Hits         8019     8406     +387     
+ Misses       1753     1421     -332     
- Partials      285      288       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@anth-volk anth-volk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your continued work on this, @SylviaDu99. This is a challenging component, so it's great to have your experience here. I made a couple of structural recommendations, but a lot boils down to these two things:

  1. I'd move the meat of the function back to ParameterNode and limit the method you wrote in AtInstantLike to do just what it says - get a dictionary of attributes
  2. I'd programmatically iterate over all attrs, then remove ones we don't want, instead of hard-coding what we do want, as I think the former is more maintainable.

Thanks for your work, and I look forward to chatting with you about it tomorrow.


def get_attr_dict(self) -> dict:
attr_dict = {}
attr_list = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the best practice here would be to keep this method as simple as possible, then do a bit more work with it where we actually use it. What I would do here is:

  1. Programmatically loop over all attributes and feed into a relevant data structure
  2. Return said structure

I wouldn't recurse downward here, either. I think we can do that inside our custom ParameterNode method, where we're more broadly defining what we're doing.

@@ -274,3 +276,11 @@ def get_child(self, path: str) -> "ParameterNode":
f"Could not find the parameter (failed at {name})."
)
return node

def write_yaml(self, file_path: Path) -> yaml:
data = self.get_attr_dict()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's at this point where I'd do some of what you've done in AtInstantLike, though I'm not sure you need to hard-code the keys we need. I would just define a list of the keys we don't want, run the function from here, use the list to remove any keys we don't want, then recurse here, as you've done

@anth-volk
Copy link
Contributor

@SylviaDu99 Thanks for your revisions to this. I decided to test this out by printing out the entirety of the -us package's parameter structure within its "gov" folder, which I'd be happy to share with you. A couple issues came up:

  • The code still writes out the parent value of a relevant parameter, meaning there may be an issue in get_attr_dict's processing of the exclusion list. It also appears to add children at certain points, but I'm not certain on that, as the output file is extremely large
  • I'm not 100% sure this method is 100% recursive. I think - as long as it isn't memory-intensive enough to crash - you would probably want to follow the following flow:
    • Create exclusion list in ParameterNode; perhaps don't add children here
    • Run get_attr_dict, possible even without applying the exclusion list, because we'll use children a bit lower
    • Iterate over dict and delete items contained within exclusion list
    • For child in children, write a key equivalent to the child's name value, then have the output of write_yaml tacked onto the output at that key

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add fully-functional ParameterNode.__repr__
2 participants