Review ways of speeding up the ESI UI

[tzumainn]

Can we cache data? Other ideas?

[joachimweyl]

@tzumainn 8 story points seem large for a research issue, I suggest time boxing initial research to 2-3 points and then creating new issues once initial research is done based on the outcome of initial research.

[tzumainn]

I talked to @sheldor1510 about this issue previously, listed several vaguely plausible ways this might be accomplished, and how it would definitely take a chunk of time to figure out. 2-3 points feels very low to me!

[joachimweyl]

I am not suggesting that the whole issue shrinks. The idea is 2-3 story points to get initial research done and then create more issues equaling maybe even more than 8 but at least the initial one will be small enough to decide on the initial direction to go. The point is to try to keep issues within one sprint as much as possible. If we think it will take more than one sprint it is best to break into multiple issues.

[ajamias]

Two ways I can think of speeding up the ui:

• Formatting the data in the Javascript instead of Python because Javascript is faster
  • For listing nodes, offers, leases, have the python retrieve the data using esisdk and other methods, send the data to the client, then have the Javascript in the frontend format the data nicely for the user.
• Using websockets
  • As soon as the user navigates to the dashboard, a websocket connection is created.
  • The Django backend polls requests at a configurable rate, and sends data to the frontend when data is recieved.
  • The frontend will display timestamps when the data was last updated

[tzumainn]

I'm curious about the websockets solution - can you expand on the implementation?

[ajamias]

I think that the main slow user experience comes from navigating to a page and having to wait 10+ seconds for the table to load.

Currently, every page navigation needs to establish a new connection to the Django backend, the backend will have to make several requests to esi-leap, esisdk, etc., and then send the data back to the client.

I am proposing to use websockets so only one persistent duplex connection is established.

The Django backend will have a websocket port listening for connections. As soon as the user logs in to the dashboard regardless if it's an esi-ui panel or not, the frontend will establish a websocket connection in esi.service.js. On connection establish, django will poll node, lease, and offer data and send it to the client at a configurable rate. esi.service.js will implement a handler function that updates the variables for lists of nodes, leases, and offers. The controllers will call a service function that returns a promise that indicates that new data has arrived, retrieve that data from the service, format the data to display with a timestamp, and then repeat.

When the user logs in, the first iteration of getting data might be slow, but all page navigations after will be faster since the data is cached in esi.service.js.

For now, other actions like provisioning, network operations, etc. will be using the already existing HTTP requests.

[tzumainn]

that sounds like a pretty good idea - would you like to work on a proof-of-concept? I can create a new issue if so.

[ajamias]

Yep, I can work on a proof-of-concept. A new issue would be great.

[tzumainn]

Followup issue created: #626