Asynchronous Building of Plugins

[smolck]

Currently, if you Plug a new Haskell plugin, the next time you open neovim you'll have to wait for the plugin to build before you can do anything. It'd be really nice if that could be done asynchronously.

[saep]

I originally wanted to do that when I wrote this very first vimL plugin of mine. I opted for the synchronous option for at least the following reasons:

• it was harder to implement :-)
• whle the plugin is compiling, you and other plugins cannot use the plugin's functionality; this can break the other plugins or it can be annoying that error messages keep popping up
• if neovim finally has started, errors are programming errors and not just intermittent while one or more plugins compile

[smolck]

@saep Ah okay, now I see what you mean! I didn't think about those cases.

What if a flag was set after a plugin or plugins were compiled, say g:nvim_hs_has_loaded_plugins, and then the plugin functions (for example NextRandom()) would be disabled until that value is true? Maybe an error message could even be displayed if the user tried to call a function before the plugin is loaded, something like Unable to call function, plugins aren't loaded.

if neovim finally has started, errors are programming errors and not just intermittent while one or more plugins compile

Would you mind elaborating on what you mean here? I think I see what you're saying, but I want to make sure.

[saep]

if neovim finally has started, errors are programming errors and not just intermittent while one or more plugins compile

Would you mind elaborating on what you mean here? I think I see what you're saying, but I want to make sure.

It's more of an extension of the second point. Say plugin A depends on plugin B which depends on C; visually A -> B -> C. Then if C is still compling, B might start in a broken state and you may not notice that. Then A seems to be broken and you try to find the issue in plugin A, even though the issue lies in B because it didn't properly communicate that it broke because C wasn't available.

It's probably a super unlikely case because plugins do not usually depend on other plugins and if they do, usually not that deeply. But these issues can be really annoying time sinks.

What if a flag was set after a plugin or plugins were compiled, say g:nvim_hs_has_loaded_plugins, and then the plugin functions (for example NextRandom()) would be disabled until that value is true? Maybe an error message could even be displayed if the user tried to call a function before the plugin is loaded, something like Unable to call function, plugins aren't loaded.

You should already get a similar error message because the function is only defined after nvim-hs hast started successfully which it can only do after it has compiled successfully. To provide a function, you have to connect to the RPC-Channel and tell Neovim which functions you provide. This can't done at compile time.

[smolck]

if neovim finally has started, errors are programming errors and not just intermittent while one or more plugins compile

Would you mind elaborating on what you mean here? I think I see what you're saying, but I want to make sure.

It's more of an extension of the second point. Say plugin A depends on plugin B which depends on C; visually A -> B -> C. Then if C is still compling, B might start in a broken state and you may not notice that. Then A seems to be broken and you try to find the issue in plugin A, even though the issue lies in B because it didn't properly communicate that it broke because C wasn't available.

It's probably a super unlikely case because plugins do not usually depend on other plugins and if they do, usually not that deeply. But these issues can be really annoying time sinks.

Thank you for the explanation! And yes, that would definitely be very annoying . . . hmm . . . yeah now I see why it's harder to implement async building!

You should already get a similar error message because the function is only defined after nvim-hs hast started successfully which it can only do after it has compiled successfully. To provide a function, you have to connect to the RPC-Channel and tell Neovim which functions you provide. This can't done at compile time.

Okay, so then really the only blocker is figuring out how to ensure all the plugins are built before the user can start using them?

[saep]

One of the main blockers is having an idea with a better user experience than the current one. The other one is finding someone to invest the tie implementing it.

[smolck]

I’m not so sure about the first blocker, but I’m definitely willing to help where I can! Unfortunately though, I’m not super well-versed in VimL, so I don’t know how much assistance I can offer.

I do however know a fair bit of Lua (at least enough to use it in a project/plugin), and I’m wondering if a port of this plugin from VimL -> Lua would be worth it, since nowadays Neovim has pretty great Lua support. That’s a separate topic (and project) entirely, but if I did this I could add this feature along the way, and fix other issues that may exist. It might also improve performance (LuaJIT is way faster than VimL).

I would need some direction there though, because conceptually I’m not really sure how this plugin works, nor what could be done to improve the user experience. More specifically, if you were to rewrite this plugin, what would you do differently?