Add support for vega-lite 5

[davidanthoff]

Vega-lite 5 is out, and it would be great if we could add support for it (and the new MIME type for it) to nteract :) I think nteract had this awesome system, where it essentially can support all vega and vega-lite versions at the same time, right? So fingers crossed this might be a relatively simple change?

CC @domoritz

[domoritz]

We made sure most of Vega-Lite 5 is backwards compatible (even if the schema is not backwards compatible for example for selections).

[davidanthoff]

I think here in nteract there is all the infrastructure there already so that we can just add support for a new vega-lite-5 MIME type, and then it ships the latest vega-lite 4 library for v4 MIME types, and vega-lite 5 library for v5 MIME types. So at least on the nteract side of things I believe backwards compat is actually not an issue :)

[domoritz]

Even better!

[captainsafia]

Yep -- the infrastructure for vega integration is over at the https://github.com/nteract/any-vega repo.

@CrystallineCat authored the package but I believe what's required is adding the transpiled JS sources for the new version and updating the https://github.com/nteract/any-vega/blob/472f50dd64c3b72f4a4f10db4c6ac4e963baeac8/packages/any-vega/index.ts file with the new media type and export.

[davidanthoff]

Bump :) We would like to ship vega-lite 5 in https://github.com/queryverse/VegaLite.jl but we can only do that once the various clients start to support the vega-lite 5 MIME type and also ship vega-lite 5. I don't understand the code here enough to open a PR, but would be fantastic if we could try to ship vega-lite 5 with nteract!

[davidanthoff]

@CrystallineCat is there a chance you could help us with this? I've stared at it for a while now, but I just don't understand what I would have to do to get this to work...

I think ideally we would a) add support for vega-lite 5, b) the new MIME type that corresponds to vega-lite 5, and c) I think generally update the vega and vega-lite versions that ship. On c), for example there are newer versions of vega 5 and vega-lite 4 than the ones referenced in any-vega.

And finally, it would be awesome if maybe @CrystallineCat could update the README of any-vega a little bit with instructions on how to keep things up-to-date. I'd be more than happy to occasionally bump some version numbers here and there, but at the moment I just have no clue where and how :) Just some short instructions a la "To update, make a change in file X, run tool Y and commit" or something like that would go a long way.

Thanks :)

[willingc]

Added an issue at #5

[captainsafia]

CrystallineCat isn't active on the project at the moment so we'll have to coordinate next steps for the moment.

Added an issue at #5

Let's go ahead and transfer the issue so we can retain the context.

[willingc]

I gave updating this a shot last night. I was running into some conflicts with pnp and node_modules on my Mac so I will try it on Linux later (I was having similar issues on troubleshooting memory leak on zmq.js).

[CrystallineCat]

Hi all, sorry for the lack of presence, I don't really have many free mental resources at the moment.

I got vega-lite@5 to build and bundle, and it imports fine. I didn't test this in nteract or publish or anything. I can do that if really needed, but I'd prefer if someone else did. I attached a .ipynb I used to test this last time: Vega.ipynb.zip

I noticed that several hand-written index.js files didn't get added to Git due to the .gitignore that assumed they were generated files. I corrected that. Apologies.

My process with packaging has basically been "mess around till it works", hence no docs. However, I'll highlight the changes that were needed for the new version (as images since I can't seem to get GitHub to behave regarding copying the diff):

...then yarn, yarn build, yarn bundle and it should generate the bundle, or tell you to update dependencies.

In this case, I had to update typescript and decided to update all possible packages.

Also, could someone please decide guidelines about things like bundle size/which versions to include? I don't really know much about the bundle structure and how much duplication there is, and if things are properly loaded lazily like it's supposed to, but Vega 2/Vega-Lite 1 has a completely different process than the newer versions, for example. (Sorry if I'm not making sense, can't really think straight right now.)

And also, there are those security advisories that GitHub constantly sends me e-mails about but that I'm not qualified to resolve...

[domoritz]

Thank you for writing up the process you went through!

[willingc]

Thanks @CrystallineCat. We'll handle it from here. We appreciate the screen shots of the process.