Skip to content

Commit

Permalink
deployer inheritance (#2135)
Browse files Browse the repository at this point in the history
* Move DeployedFlow and Triggered run to a more inheritance model.

The _enrich_* methods were making it very hard to generate proper stubs.
This will hopefully improve that.

* Fix circular dependency and address comments

Some doc cleanup as well.

* Removed extraneous code

* Forgot file

* Update NBDeployer to forward to underlying deployer

* Fix circular deps and other crap

* Added stub-gen for deployer (try 2)

* Fix stub test

* Fix dataclients

* fix docstrings for deployer (#2119)

* Renamed metadata directory to metadata_provider (and related changes)

There was a clash between the `metadata` function and the `metadata` module
which caused issues with stubs (at least the new way of generating)

* Forgot one

* Typo

* Make nbdocs happy

* add docstrings for injected methods

---------

Co-authored-by: Romain Cledat <[email protected]>
  • Loading branch information
madhur-ob and romain-intel authored Nov 7, 2024
1 parent 81eb31b commit dfc4a71
Show file tree
Hide file tree
Showing 49 changed files with 1,874 additions and 1,125 deletions.
5 changes: 2 additions & 3 deletions metaflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,7 @@ class and related decorators.
# Flow spec
from .flowspec import FlowSpec

from .parameters import Parameter, JSONTypeClass

JSONType = JSONTypeClass()
from .parameters import Parameter, JSONTypeClass, JSONType

# data layer
# For historical reasons, we make metaflow.plugins.datatools accessible as
Expand Down Expand Up @@ -149,6 +147,7 @@ class and related decorators.
from .runner.metaflow_runner import Runner
from .runner.nbrun import NBRunner
from .runner.deployer import Deployer
from .runner.deployer import DeployedFlow
from .runner.nbdeploy import NBDeployer

__ext_tl_modules__ = []
Expand Down
4 changes: 2 additions & 2 deletions metaflow/client/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
from .filecache import FileCache

if TYPE_CHECKING:
from metaflow.metadata import MetadataProvider
from metaflow.metadata_provider import MetadataProvider

try:
# python2
Expand Down Expand Up @@ -143,7 +143,7 @@ def default_metadata() -> str:
if default:
current_metadata = default[0]
else:
from metaflow.plugins.metadata import LocalMetadataProvider
from metaflow.plugins.metadata_providers import LocalMetadataProvider

current_metadata = LocalMetadataProvider
return get_metadata()
Expand Down
2 changes: 1 addition & 1 deletion metaflow/clone_util.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import time
from .metadata import MetaDatum
from .metadata_provider import MetaDatum


def clone_task_helper(
Expand Down
Loading

0 comments on commit dfc4a71

Please sign in to comment.