Skip to content

Commit

Permalink
Merge branch 'main' into noklam/databricks-asset-bundles-docs
Browse files Browse the repository at this point in the history
  • Loading branch information
noklam authored Nov 26, 2024
2 parents cbb8f56 + dfa0b79 commit b9c111c
Show file tree
Hide file tree
Showing 7 changed files with 61 additions and 63 deletions.
12 changes: 6 additions & 6 deletions docs/diagrams/installed-kedro-project.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
```mermaid
sequenceDiagram
title "Installed Kedro project"
title Installed Kedro project
participant script as "third-party Python script"
participant curr_dir as "Directory with Kedro conf/ in it"
participant session as "KedroSession"
participant script as Third-party Python script
participant curr_dir as Directory with Kedro conf/ in it
participant session as KedroSession
script->>script: run third-party script
script->>curr_dir: get path to the project config
script->>session: create a session with Kedro project name and project config dir
session->>run: run a pipeline and/or nodes
script->>session: create a session with Kedro project config dir
session->>session: run a pipeline and/or nodes
```
27 changes: 11 additions & 16 deletions docs/diagrams/kedro-ipython.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,25 @@
```mermaid
sequenceDiagram
title "$ kedro ipython"
title $ kedro ipython
participant cli as "$ kedro ipython"
participant env as "Environment variables"
participant ipython as "IPython"
participant entrypoint as "00-kedro-init.py\nreload_kedro"
participant hook_manager as "Hook manager"
participant project as "Kedro project directory"
participant session as "KedroSession"
participant context as "KedroContext"
participant cli as $ kedro ipython
participant env as Environment variables
participant ipython as IPython
participant entrypoint as ipython/__init__.py <br> reload_kedro
participant project as Kedro project directory
participant session as KedroSession
participant context as KedroContext
cli->>cli: Check if IPython is importable
cli->>env: Set IPYTHONDIR to metadata.project_path / ".ipython"
cli->>env: Set KEDRO_ENV to the chosen Kedro environment
cli->>cli: Print an info message
cli->>ipython: Start ipython
ipython->>entrypoint: load startup script
entrypoint->>entrypoint: import Kedro
entrypoint->>hook_manager: clear the hook manager
ipython->>entrypoint: load ipython extension
entrypoint->>project: find Kedro project
entrypoint->>project: bootstrap the project
entrypoint->>entrypoint: remove imported project package modules
entrypoint->>session: create a KedroSession
entrypoint->>session: activate the session
entrypoint->>session: load KedroContext
entrypoint->>context: get the data catalog
entrypoint->>entrypoint: expose session, context and catalog variables
entrypoint->>entrypoint: expose session, context, catalog and pipelines variables
entrypoint->>entrypoint: register reload_kedro line magic
```
16 changes: 8 additions & 8 deletions docs/diagrams/kedro-no-project.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
```mermaid
sequenceDiagram
title "$ kedro\ndirectory without Kedro project"
title $ kedro in directory without Kedro project
participant kedro as "$ kedro"
participant entrypoint as "setup.py\nkedro = kedro.framework.cli:main"
participant init_plugins as "Kedro Plugins\nentry_point = kedro.init"
participant kedro_cli as "Kedro CLI\nglobal commands\ninfo, new, docs, starter"
participant global_plugins as "Kedro Plugins\nentry_point = kedro.global_commands"
participant pyproject.toml as "Current directory\npyproject.toml"
participant click as "Click"
participant kedro as $ kedro
participant entrypoint as pyproject.toml <br> kedro = "kedro.framework.cli:main"
participant init_plugins as Kedro Plugins <br> [project.entry-points."kedro.init"]
participant kedro_cli as Kedro CLI <br> global commands <br> info, new, docs, starter
participant global_plugins as Kedro Plugins <br> [project.entry-points."kedro.global_commands"]
participant pyproject.toml as Current directory <br> pyproject.toml
participant click as Click
kedro->>entrypoint: Python calls this
Expand Down
14 changes: 7 additions & 7 deletions docs/diagrams/kedro-plugin.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
```mermaid
sequenceDiagram
title "$ kedro plugin"
title $ kedro plugin
participant cli as "$ kedro plugin"
participant prelude as "See kedro-with-project.puml for details"
participant project_plugin as "Kedro Plugin\nentry_point = kedro.project_commands"
participant click as "Click context"
participant session as "KedroSession"
participant cli as $ kedro plugin
participant prelude as See kedro-with-project.md for details
participant project_plugins as Kedro Plugins <br> [project.entry-points."kedro.project_commands"]
participant session as KedroSession
participant kedro_cli as KedroCLI
cli->>prelude: prepare click commands as prelude to this
prelude->>project_plugin: execute plugin click command
project_plugin->>click: get ProjectMetadata from the click context
project_plugin->>kedro_cli: get ProjectMetadata from the KedroCLI command collection group
project_plugin->>project_plugin: plugin code
project_plugin->>session: need to create KedroSession for all runtime config and info
```
22 changes: 11 additions & 11 deletions docs/diagrams/kedro-run.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
```mermaid
sequenceDiagram
title "$ kedro run"
title $ kedro run
participant cli as "$ kedro run"
participant prelude as "See kedro-with-project.puml for details"
participant project_cli as "Project directory\ncli.py"
participant session as "KedroSession"
participant context as "KedroContext"
participant runner as "Runner"
participant hooks as "Hook manager"
participant cli as $ kedro run
participant prelude as See kedro-with-project.md for details
participant project_cli as Project directory <br> cli.py
participant session as KedroSession
participant context as KedroContext
participant runner as Runner
participant hooks as Hook manager
cli->>prelude: prepare click commands as prelude to this
prelude->>project_cli: run
project_cli->>session: create KedroSession
session->>hooks: initialise hook manager
session->>session: run
session->>session: load KedroContext
session->>context: get the selected pipeline
context->>context: filter the pipeline based on command line arguments
session->>session: get the selected pipeline
session->>session: filter the pipeline based on command line arguments
session->>context: get catalog with load version / save version
session->>runner: create runner
session->>hooks: get hook manager
hooks->>hooks: before_pipeline_run
runner->>runner: run the filtered pipeline with the catalog
hooks->>hooks: on_pipeline_error (if runner fails)
Expand Down
21 changes: 11 additions & 10 deletions docs/diagrams/kedro-with-project.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
```mermaid
sequenceDiagram
title "$ kedro\ndirectory without Kedro project"
title $ kedro in directory with Kedro project
participant kedro as "$ kedro"
participant entrypoint as "setup.py\nkedro = kedro.framework.cli:main"
participant init_plugins as "Kedro Plugins\nentry_point = kedro.init"
participant kedro_cli as "Kedro CLI\nglobal commands\ninfo, new, docs, starter"
participant global_plugins as "Kedro Plugins\nentry_point = kedro.global_commands"
participant pyproject.toml as "Current directory\npyproject.toml"
participant project_plugins as "Kedro Plugins\nentry_point = kedro.project_commands"
participant kedro_project as "Current directory\nKedro Project: cli.py"
participant kedro as $ kedro
participant entrypoint as pyproject.toml <br> kedro = "kedro.framework.cli:main"
participant init_plugins as Kedro Plugins <br> [project.entry-points."kedro.init"]
participant kedro_cli as Kedro CLI <br> global commands <br> info, new, docs, starter
participant global_plugins as Kedro Plugins <br> [project.entry-points."kedro.global_commands"]
participant pyproject.toml as Current directory <br> pyproject.toml
participant project_plugins as Kedro Plugins <br> [project.entry-points."kedro.project_commands"]
participant kedro_project as "Current directory <br> Kedro Project: cli.py"
participant click as "Click"
kedro->>entrypoint: Python calls this
entrypoint->>init_plugins: load and run all installed
entrypoint->>init_plugins: load and run all installed plugins
entrypoint->>kedro_cli: collect built-in commands
entrypoint->>global_plugins: load and collect global plugin commands
entrypoint->>pyproject.toml: check current dir for a Kedro project
Expand Down
12 changes: 7 additions & 5 deletions docs/diagrams/python-m-project.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
```mermaid
sequenceDiagram
title "$ <project>\n$ python -m <project>.run"
title python -m {project}.run
participant cli as "$ <project>\n$ python -m <project>"
participant entrypoint as "setup.py\n<project> = <project>.__main__"
participant session as "KedroSession"
participant cli as $ <project> <br> $ python -m <project>
participant entrypoint as pyproject.toml <br> <project> = <project>.__main__:main
participant find_run as find_run_command()
participant session as KedroSession
cli->>entrypoint: Python calls the entrypoint
entrypoint->>session: create session
entrypoint->>find_run: Find run command
find_run->>session: create session (for default run)
session->>session: run
```

0 comments on commit b9c111c

Please sign in to comment.