Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Registration #147

Merged
merged 36 commits into from
Jan 15, 2024
Merged

Registration #147

merged 36 commits into from
Jan 15, 2024

Conversation

liamhuber
Copy link
Member

Ahead of saving and loading workflows, we need the saved nodes to be able to know what node package they came from, so that workflows can register these on load, re-create the nodes, then the nodes can load their state from the data. (Note that Macros don't have the same problem, because they know how to create all their children and do so at initialization, so they can simply read off their saved states -- there is an edge case where a macro has replaced one or more of it's children, but we can deal with that later).

In principle each node package should have a unique identifier, but in practice I'm still just using the import module, e.g. "pyiron_workflow.node_library.standard". This "identifier" now gets stored on the node classes when they are registered as a package, so they'll be available to save later.

Otherwise there's a bunch of refactoring, including moving more responsibility onto NodePackage and making the (for now) equivalence of the package identifier and module name more clear.

There's also a bit of extra syntactic sugar so that you can register stuff to a deeply nested domain from the get-go, e.g. Workflow.create.register("my.deep.domain", "pyiron_workflow.node_library.standard"); Workflow.create.my.deep.domain.Add().

Now that __dir__ contains the domains!
Not the domain it can be found at on the creator, which is just a QoL convenience thing.
Maybe we eventually want to put in a softer check, so that if we find a module with `nodes` that _isn't_ `Iterable[type[Node]]` we raise a warning and keep going, but there's no need for that now.
We don't want the loaded package to get out of sync with its source
This is not actually anything to do with this PR, but rather pyiron/atomistics changed its API
There is an upstream bug that all the lammps jobs are printing "r" a million times and it's hideous.
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link

github-actions bot commented Jan 4, 2024

Binder 👈 Launch a binder notebook on branch pyiron/pyiron_workflow/registration_tracking

Copy link

codacy-production bot commented Jan 4, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
+0.04% (target: -1.00%) 95.56%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (1ea4ff6) 2503 2136 85.34%
Head commit (d44217b) 2559 (+56) 2185 (+49) 85.38% (+0.04%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#147) 90 86 95.56%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

Copy link

github-actions bot commented Jan 4, 2024

Pull Request Test Coverage Report for Build 7426980580

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.04%) to 90.301%

Totals Coverage Status
Change from base Build 7426966556: 0.04%
Covered Lines: 4646
Relevant Lines: 5145

💛 - Coveralls

@liamhuber liamhuber added the format_black trigger the Black formatting bot label Jan 4, 2024
@liamhuber liamhuber requested a review from samwaseda January 5, 2024 00:30
@liamhuber
Copy link
Member Author

Notebook failure appears to just be a versioning inconsistency with pyiron/atomistics; I'll fix it before merging but it doesn't impact the meat here

@liamhuber
Copy link
Member Author

Also, forgive me @samwaseda, for this PR is sinfully big. I should have decomposed it into some smaller steps, but I rushed. I hope the readability of the end product is high enough to compensate somewhat.

@liamhuber
Copy link
Member Author

Original notebook error seems to have been a caching error and outdated version of pyiron/atomistics. Now pyiron_atomistics is breaking in a strange (and still seemingly unrelated) way:

File /usr/share/miniconda3/envs/my-env/lib/python3.11/site-packages/pyiron_base/jobs/job/runfunction.py:700, in handle_failed_job(job, error)
    698     if job.server.run_mode.non_modal:
    699         state.database.close_connection()
--> 700     raise RuntimeError("Job aborted")
    701 else:
    702     return True, out

RuntimeError: Job aborted

liamhuber and others added 4 commits January 5, 2024 09:52
Just needed to update my env to purge a bug in an outdated version of h5io
It is a bug in my CI that this is not happening automatically.
@liamhuber
Copy link
Member Author

One problem was that updates to the extras_require in setup.py are not being automatically propagated to .ci_support/notebooks-env.yml. This is a CI bug, and I opened an issue for it: #148

@liamhuber
Copy link
Member Author

liamhuber commented Jan 5, 2024

Looking for: ['jupyter', 'papermill', 'coveralls', 'coverage', 'bidict=0.22.1', 'cloudpickle=3.0.0', 'graphviz=8.1.0', 'matplotlib=3.8.2', 'pympipool=0.7.9', 'python-graphviz=0.20.1', 'toposort=1.10', 'typeguard=4.1.5', 'ase=3.22.1', 'atomistics=0.1.20', 'lammps', 'phonopy=2.21.0', 'pyiron_atomistics=0.4.1', 'pyiron-data=0.0.24', 'numpy=1.26.2']


Could not solve for environment specs
The following packages are incompatible
├─ atomistics 0.1.20**  is requested and can be installed;
└─ pyiron_atomistics 0.4.1**  is uninstallable because it requires
   └─ atomistics >=0.1.12,<=0.1.18 , which conflicts with any installable versions previously reported.
Error: Process completed with exit code 1.

Right, the dependabot upgrade of atomistics to 0.1.19 got through the tests before because it only impacts the notebooks test, but changes to setup.py by dependabot were not getting propagated to the actual notebooks env so the conflict never arose. I'm going to go patch this in a separate PR.

# Conflicts:
#	.binder/environment.yml
#	.ci_support/environment-notebooks.yml
#	notebooks/atomistics_nodes.ipynb
#	notebooks/deepdive.ipynb
Copy link

codacy-production bot commented Jan 5, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
+0.04% (target: -1.00%) 95.56%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (908a724) 2503 2136 85.34%
Head commit (1964aa7) 2559 (+56) 2185 (+49) 85.38% (+0.04%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#147) 90 86 95.56%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

@liamhuber
Copy link
Member Author

Closes #152

@liamhuber liamhuber linked an issue Jan 8, 2024 that may be closed by this pull request
@liamhuber liamhuber merged commit d5fcb48 into main Jan 15, 2024
15 of 16 checks passed
@liamhuber liamhuber deleted the registration_tracking branch January 15, 2024 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
format_black trigger the Black formatting bot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Storage: nodes from a package should know their package
2 participants