Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempting to view task data closes browser connection #257

Open
duane-space opened this issue Oct 21, 2023 · 4 comments
Open

Attempting to view task data closes browser connection #257

duane-space opened this issue Oct 21, 2023 · 4 comments

Comments

@duane-space
Copy link

duane-space commented Oct 21, 2023

First off, thanks for plombery! It's quite neat and also easy to get started with compared to alternatives. Excited to see where it goes.
 
I have a flow with 3 tasks:

The first returns a list of some class objects
The second returns a dictionary
The third returns a (filepath like) string

For the first task, if I click to view the 'data' from the web ui, it shows me a table indicating the length of the list, but nothing about the contents (probably the expected behavior)

However, I can't view the data for either the second or third tasks. For both, when I click to view the data, I get a blank white page and to get the UI to load again I need to reload the page. I see this in the terminal from which I ran plombery:

INFO:    "GET /api/runs/2/logs HTTP/1.1" 200 OK

INFO:    "GET /api/runs/2/data/task_2 HTTP/1.1" 200 OK

INFO:     connection closed
@duane-space
Copy link
Author

Realized I could gain some insight via developer tools.

Second task: (dictionary case)
vendor-a8602ee1.js:38 Uncaught TypeError: Cannot convert undefined or null to object at Function.keys (<anonymous>)

Third task: (string case)
vendor-a8602ee1.js:40 Error: loadData only accepts array of objects or array of arrays (string given)

@lucafaggianelli
Copy link
Owner

hi thanks for report this issue, indeed the data store functionality is not well defined/documented today. I only implemented the case in which the output of a task is a pandas DataFrame and in all other cases I try to json serialize the data, probably that doesn't alway work...

So this could be an occasion to improve the task view feature... what would you expect from plombery? :) to see the data in a json viewer? in a table? to plot a chart?

@duane-space
Copy link
Author

duane-space commented Oct 30, 2023

Hey - thanks for the response!

Tthere could be a lot of routes to go down, and some user configuration might be necessary to get the most out of it! Besides the error handling issue, I think a minimal default functionality should probably be something like:

  1. showing primitives
  2. show 'types' of non-primitives
  3. show lists, dictionaries, data-frames as tables (displaying either primitives or 'types' as the content)
  4. for data structures nested beyond 2D, show datatypes
  5. Json viewer is also a good fallback option for json serializable data.

Maybe embedding something like graphic walker could be quite powerful?
https://github.com/Kanaries/graphic-walker

Or pivot table?
https://react-pivottable.js.org/

@lucafaggianelli
Copy link
Owner

yes, makes sense to me, aside from the default data display behavior I'll add some helper function to write and read output data from tasks, also because today the data is stored into a file and it's not always the best solution... moreover someone may not be interested into storing task output data at all

wow I love graphic walker, I'll check how I can integrate it in plombery, thanks for the hint!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants