Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alllow for consecutive (depedent) notebook executions #11

Open
chrisjsewell opened this issue Feb 25, 2020 · 1 comment
Open

Alllow for consecutive (depedent) notebook executions #11

chrisjsewell opened this issue Feb 25, 2020 · 1 comment
Labels
discussion enhancement New feature or request

Comments

@chrisjsewell
Copy link
Member

chrisjsewell commented Feb 25, 2020

Although this would introduce some extra complexity, it is (I think) a current requirement of Jupinx.

Once #10 is implemented it would be possible to 'orchestrate' consecutive notebook execution, whereby:

  1. A notebook is executed and its final state and artefacts committed.
  2. A subsequent notebook is staged for execution with (1)'s nbfile and/or artefacts as it's assets.
  3. That notebook is executed

The orchestration could possibly be done at the staging phase, from a top-level; stating which notebook relied on which, or from the notebook-level; stating a relative path dependency in the metadata (this may be less robust). Then the cache or executor would work out a 'branching tree' of execution paths (checking it is acyclic) and decide how to execute appropriately.

The artefacts from the first notebook could be either/or files that the notebook outputs or its final kernel state (using something like dill.dump_session)

@chrisjsewell chrisjsewell added discussion enhancement New feature or request labels Feb 25, 2020
@chrisjsewell
Copy link
Member Author

FYI, this come up from discussion with @mmcky, @choldgraf may also be interested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant