Garble cherry pick #47
Conversation
| with open(metadata_file, "w") as original_metafile: | ||
| json.dump(metadata, original_metafile, indent=2) |
There was a problem hiding this comment.
This is the critical piece here -- not sure why the other changes were made, it seems like renaming the other files breaks in linkage-agent-tools. Can you revert the other changes and just do this? Or help me understand what the other changes are needed for
There was a problem hiding this comment.
@dehall The other changes are changes we'd have to make in anticipation of merging PR #32 in linkage-agent-tools. Those changes have been present as a part of this PR since before it was cherry-picked into a new PR but I'm not sure if there is any discussion there about them. Either way, they won't break linkage-agent-tools once This PR is merged
There was a problem hiding this comment.
When I try with this branch on data-owner-tools and that branch on linkage-agent-tools I still get errors, related to the metadata file name in validate.py:
$ py validate.py
site_a.zip
Traceback (most recent call last):
File "/Users/dehall/linkage-agent-tools/validate.py", line 67, in <module>
do_validate(config)
File "/Users/dehall/linkage-agent-tools/validate.py", line 23, in do_validate
missing, unexpected, metadata_issues = c.validate_all_present()
File "/Users/dehall/linkage-agent-tools/dcctools/config.py", line 129, in validate_all_present
metadata_issues.extend(self.validate_metadata(system_zip_path))
File "/Users/dehall/linkage-agent-tools/dcctools/config.py", line 59, in validate_metadata
timestamp = datetime.strptime(mname, "%Y%m%dT%H%M%S")
File "/usr/local/Cellar/[email protected]/3.9.2_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/_strptime.py", line 568, in _strptime_datetime
tt, fraction, gmtoff_fraction = _strptime(data_string, format)
File "/usr/local/Cellar/[email protected]/3.9.2_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/_strptime.py", line 349, in _strptime
raise ValueError("time data %r does not match format %r" %
ValueError: time data '' does not match format '%Y%m%dT%H%M%S'
And in projects.py, I believe this is because the clk file name format doesn't match what's expected:
$ py projects.py
Traceback (most recent call last):
File "/Users/dehall/linkage-agent-tools/projects.py", line 120, in <module>
run_projects(config, args.project)
File "/Users/dehall/linkage-agent-tools/projects.py", line 61, in run_projects
run_project(c, timestamp, project_name)
File "/Users/dehall/linkage-agent-tools/projects.py", line 87, in run_project
project.upload_clks(system, c.get_clks_raw(system, project_name))
File "/Users/dehall/linkage-agent-tools/dcctools/config.py", line 221, in get_clks_raw
with clk_zip.open(project_file) as clk_file:
UnboundLocalError: local variable 'project_file' referenced before assignment
Rather than try to solve all these together, let's just do one piece at a time, so for this PR let's just keep the necessary snippet and revisit the rest separately.
Adds "garble_time" field to metadata file both kept by data owner and sent to linkage agent.
For ticket No. 183864643