Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wip][Flyte Deck] Streaming Decks #2779

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from
+68 −2
Draft

[wip][Flyte Deck] Streaming Decks #2779

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
01182b4
[Flyte Decl] Streaming Decks
Future-Outlier Oct 1, 2024
9616fc3
print
Future-Outlier Oct 1, 2024
98ae7c7
sleep more
Future-Outlier Oct 1, 2024
4e92bb0
add dummy deck
Future-Outlier Oct 2, 2024
4df18b5
nit
Future-Outlier Oct 2, 2024
ebb4d4e
dummy deck
Future-Outlier Oct 2, 2024
99522d9
update
Future-Outlier Oct 2, 2024
c19d67d
nit
Future-Outlier Oct 2, 2024
67cd829
test
Future-Outlier Oct 2, 2024
06da3df
return html
Future-Outlier Oct 2, 2024
6d99d69
Change Deck
Future-Outlier Oct 2, 2024
b805cd7
fix
Future-Outlier Oct 2, 2024
4c97758
fix recursion error
Future-Outlier Oct 2, 2024
7b3574a
remove redundant code
Future-Outlier Oct 2, 2024
9b60564
add dummy deck to deck init
Future-Outlier Oct 2, 2024
18c994f
Better Dummy Deck Logic
Future-Outlier Oct 2, 2024
39f39d1
Merge branch 'master' into flytekit-streaming-deck
Future-Outlier Oct 7, 2024
aabcbbb
Deck Publish
Future-Outlier Oct 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions flytekit/bin/entrypoint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ def _dispatch_execute(
input_proto = utils.load_proto_from_file(_literals_pb2.LiteralMap, local_inputs_file)
idl_input_literals = _literal_models.LiteralMap.from_flyte_idl(input_proto)

# # TEST DECK
# if task_def is not None and not getattr(task_def, "disable_deck", True):
# _output_deck(task_def.name.split(".")[-1], ctx.user_space_params)
# print("@@@ OUTPUT DECK @@@")

# Step2
# Invoke task - dispatch_execute
outputs = task_def.dispatch_execute(ctx, idl_input_literals)
Expand Down
2 changes: 1 addition & 1 deletion flytekit/deck/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
SourceCodeRenderer
"""

from .deck import Deck, DeckField
from .deck import Deck, DeckField, DummyDeck
from .renderer import MarkdownRenderer, SourceCodeRenderer, TopFrameRenderer
63 changes: 62 additions & 1 deletion flytekit/deck/deck.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,40 @@

OUTPUT_DIR_JUPYTER_PREFIX = "jupyter"
DECK_FILE_NAME = "deck.html"
DUMMY_DECK_HTML = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make it automatically refresh the page.. probably need that on the empty deck but not sure how to do the realtime behavior once it has content (you don't want to refresh while the user is browsing around or clicking through various tabs)

Suggested change
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="refresh" content="5" >

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COOL, will try it, thank you

<title>Flytekit Status</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
font-family: Arial, sans-serif;
background-color: #f0f0f0;
}
.message {
background-color: #ffffff;
padding: 20px;
border: 1px solid #cccccc;
border-radius: 5px;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
text-align: center;
}
</style>
</head>
<body>
<div class="message">
<p> Flyte decks have not been created yet. </p>
</div>
</body>
</html>
"""


class DeckField(str, enum.Enum):
Expand Down Expand Up @@ -86,6 +120,27 @@ def name(self) -> str:
def html(self) -> str:
return self._html

@classmethod
def publish(cls):
task_name = FlyteContextManager.current_context().user_space_params.task_id.name
new_user_params = FlyteContextManager.current_context().user_space_params
_output_deck(task_name, new_user_params)


class DummyDeck(Deck):
"""
The DummyDeck class is designed
"""

def __init__(self):
name = "dummy_deck"
html = DUMMY_DECK_HTML
super().__init__(name, html)

@property
def html(self) -> str:
return self._html


class TimeLineDeck(Deck):
"""
Expand Down Expand Up @@ -154,7 +209,13 @@ def _get_deck(
Get flyte deck html string
If ignore_jupyter is set to True, then it will return a str even in a jupyter environment.
"""
deck_map = {deck.name: deck.html for deck in new_user_params.decks}

deck_map = {deck.name: deck.html for deck in new_user_params.decks if deck.name != "dummy_deck"}

# If deck_map is empty after filtering, add DummyDeck
if not deck_map:
deck_map = {"dummy_deck": DUMMY_DECK_HTML}

nav_htmls = []
body_htmls = []

Expand Down
Loading