support vscode-jupyter extension

[tianhaoyang]

Currently theia supports vscode-python extension, which is a basic notebook-like python execution UI. However, a brand-new jupyter extension vscode-jupyter has been released recently, which is much more powerful and user-friendly. Can theia support this?

[msujew]

Hi @tianhaoyang. I'll have to disappoint you there, the new Notebook API is not supported by Theia. As it's quite large - I looked into it a few weeks ago while it was still in a proposal state - with around 30k LOC, I don't expect Theia to support it in the near future.

Regarding support for it, while a lot of stuff can be simply copied from vscode's repo, the whole rendering and glue code will have to be rewritten to accommodate Theia's architecture. That'll be quite a large undertaking. Though, I won't stop anyone from trying it ;)

[vince-fugnitto]

Please also note that it is possible that ms-toolsai.jupyter is in fact proprietary, or uses proprietary dependencies which will not make it available to be published to open-vsx and may cause legal issues when consumed by non-vscode products:

• [python] Recent versions of ms.python have dependency to ms-toolsai.jupyter, that's not available on open-vsx.org theia-ide/theia-apps#441 (comment)
• latest version of theiaide/theia-python will not debug theia-ide/theia-apps#485 (comment)

[tianhaoyang]

@msujew @vince-fugnitto Thank you for your response. If you have any updates on this issue, please notify me. BTW,
@msujew do you have any tips for rewriting the Notebook API to accommodate Theia's arch?

[msujew]

@tianhaoyang Not really tips, just general differences between Theia and vscode. Generally, I don't think it's too difficult to rewrite, but just a rather long process due to the amount of code:

• Theia is using React for rendering, while vscode uses their custom DOM rendering API. (which you can find here) This in turn requires us to rewrite the whole rendering code.
• Theia is using inversify for dependency injection, vscode (again) uses its own implementation. This shouldn't be too hard to rewrite, it's just a lot of code. Probably not all vscode services match 100% with Theia's services, which could make this task a bit harder.

[tsmaeder]

@msujew there used to be a jupyter plugin that was based on a custom editor. I was under the impression that they basically moved that inside the VS Code base. Maybe that is an easier starting point?

[msujew]

@tsmaeder Mostly, yes. AFAIK previous versions of the jupyter extensions used a custom editor, which used the normal custom editor API. The basic architecture (notebooks, notebook serialization, notebook rendering) is quite similiar between the internals of the old jupyter extension and the new vscode native implementation. Nevertheless, the native implementation is way faster (beside having a better UI). So I can imagine they changed quite a lot.

The old custom editor based jupyter extension is working btw inside Theia (although not flawlessly) see the python extension 2020.10 version.

[tsmaeder]

Here's what you get installed when you install the ms-python extension from open-vsx:

[tsmaeder]

The ms-python extension declares an "extensionPack" with the ms-toolsai/jupyter extension. This extensions than declares an extension pack like so:

"extensionPack": [
        "ms-toolsai.jupyter-keymap",
        "ms-toolsai.jupyter-renderers",
        "ms-toolsai.vscode-jupyter-slideshow",
        "ms-toolsai.vscode-jupyter-cell-tags"
    ],

[tsmaeder]

the jupyter-renderers extension contributes a renderer for https://github.com/jupyter-widgets/ipywidgets. I would expect this is required to show many notebooks.

[tsmaeder]

vscode-jupyter-slideshow adds support for converting notebooks to slideshows. Something we don't need at first, IMO.

[tsmaeder]

ms-toolsai.jupyter-keymap just adds keybindings, so should have no problems.

[tsmaeder]

ms-toolsai.vscode-jupyter-cell-tags adds support for manipulation metadata tags on cells. Not needed at first, IMO.

[tsmaeder]

As mentioned in #12442 (comment) etc., in order to use python notebooks in a way we consider reasonable for a MVP would require us to

• implement cell execution
• implement custom notebook renderers (including support for interactive renderer through messaging)
• Reasonable cell editing experience, not part of this for a MVP is (IMO)
  • Drag & Drop of cells
  • Notebook & cell toolbars (can be done via menu/command palette) as a workaround
  • Performance improvements beyond for large files
• Sundries, like stubbing any "proposed" API not needed for starting the jupyter extension

[tsmaeder]

Note that the jupyter extension uses a largish list of "proposed" apis:

        "notebookDeprecated",
        "notebookMessaging",
        "notebookMime",
        "notebookCellExecutionState",
        "portsAttributes",
        "quickPickSortByLabel",
        "notebookKernelSource",
        "interactiveWindow",
        "notebookControllerAffinityHidden",
        "quickPickItemTooltip"

but unless we analyze the use of those API's in the extensions source code, it's hard to judge which ones need a real implementation until we can actually run jupyter notebooks. For that, we'll need the above list of features.

[msujew]

We have implemented a bunch of the proposed APIs in the recent months and the vscode-jupyter extension is now running fairly stable. Closing the issue.