Improvements to pdlua graphics

[timothyschoen]

• Paths are now calculated as floating points instead of integers, allowing for better accuracy. This is especially noticeable for Bezier curves.
• Small optimisations, reduced the amount of memory allocation, this simplified it as well.
• Pd-vanilla and plugdata now use the same system for calculating paths.
• Small changes to plugdata's implementation so that pdlua graphics are GPU accelerated

[agraef]

Sorry, totally forgot about this -- too much other work to do. :( Merged, thanks!

[agraef]

@timothyschoen Ok, this is getting close now. Do you have more changes in the pipeline? Otherwise a new pd-lua release might be in order.

I just fixed the dysfunctional message forwarding on the signal inlets on the Purr Data side (finally!), with agraef/purr-data@0046a65 (cf. #43). So everything should now be in good working order in Purr Data, too (albeit without the graphics there).

I think that I should do a 0.12.0 release at this point, so that I can start porting the graphics callbacks to Purr Data afterwards. (Hopefully some time during the summer break.)

[agraef]

Alright, after a few final touches, including an update to the most recent Lua release, I'm ready to bump the version and roll a new release. If I don't hear anything from you in the next few days, I'll just go ahead with this.

[timothyschoen]

Nice! I'm all done now too :)

[agraef]

Great! :) I just released https://github.com/agraef/pd-lua/releases/tag/0.12.0. Maybe you'd like to review the release notes. I tried to mention all the major bits and pieces, but as you know there were a lot of commits leading up to 0.12.0, so it's quite possible that I missed something.

Also, it would be nice if you could update your pd-lua submodule to 0.12.0 now. It's already in my testing branch of Purr Data and will presumably be merged into the next Purr Data release.

Ok, I'm off to some well-deserved vacation now. :) After that I'll see whether I can implement the graphics routines for Purr Data.

[agraef]

Just a quick heads-up: I'm about to release 0.12.1 which (along with some recent additions to my purr-data release and testing branches) implements the graphics routines for Purr Data. That was easier than expected. :)

Seems to work fine on Linux and Windows, on Mac I still have some trouble. I suspect that the automatic redraws in hello-gui and osci3d~ clog the engine-gui communication socket, but this only happens with Purr Data on the Mac, I'll look into that asap.

[agraef]

Yeah, it seems that the (much newer) node-webkit version we use on the Mac just renders dynamic SVG documents much slower than the earlier nw.js versions we use on Linux and Windows (up to 0.42 seems to be fine). This slows down the animation examples a lot. The other examples are fine. Unfortunately, we can't use nw.js 0.42 on the Mac since it crashes. A lot. ;-) So I guess that we'll just have to live with it.

I have some versions of the hello-gui and osci3d~ examples which dial down the update rates to a point so that the animations work at least in some capacity also on Purr Data on the Mac. I also have some cosmetic changes for the circle-gui example so that it will look nicer in vanilla and Purr -- the vertical offset in that patch is wrong, and the colors in the example make my eyes bleed. ;-) I'll just commit these and roll a new version so that the examples will work better across all Pd variants and platforms.

Stay tuned...

[timothyschoen]

Cool! Thanks for working on the purr-data port, really cool to have this working in all Pd flavours :)

Good luck with the macOS issue, that sound tough...

[agraef]

Well, doing some profiling with the nw.js devtools I found that in fact the rendering itself does fine, it's the dynamic JS execution that we do in the gui that just became much slower after the NW2 rewrite. If nw.js wasn't such a monster, I'd see whether I can get 0.42.0 fixed on the Mac, but that's above my pay grade for now. :)

Anyway, I've just released pd-lua 0.12.2 where I did some minor refactoring of the purr-data interface (just for cosmetics and easier maintenance) and adjusted the animation speed of the animation examples. I think that this will also be useful for the other flavors if they run on lesser machines.

Which got me thinking about other ways to address this issue. Let's face it, even vanilla and plugdata will run out of steam eventually if you try to render lots of graphic objects on the canvas at a high frame rate. Pd was never designed to accommodate that kind of real-time drawing on the canvas, except in a few special cases like live array and data structure updates. Throw enough graphical objects at a high frame rate at it, and eventually there will be congestion on the pd->gui communication channel, and the interpretation and execution of the graphics code itself will slow down to a crawl.

But typically animations don't need the entire graphics to be repainted, you only change a few elements here and there, or simply update the coordinates of a path element or two. That's certainly true for the examples we have. In such a case it is just wasteful to redraw everything on each frame. Maybe we should have an update() method and corresponding API functions which would allow the coordinates of elements to be updated without having to redraw anything? Of course this would complicate the API, but it should also speed things up quite a bit. Food for thought...

[agraef]

Fixed the NW2 slowdown. Turns out that there's an option to disable it even in newer nw.js versions, making the JS execution performance what it was in earlier versions. So this issue is now sorted as well, and the animations can run at full throttle on the Mac. I'll still keep the changes in rev. daac188 since they will be useful for people trying to run this stuff on lesser machines like a Pi4.

[timothyschoen]

But typically animations don't need the entire graphics to be repainted, you only change a few elements here and there, or simply update the coordinates of a path element or two. That's certainly true for the examples we have. In such a case it is just wasteful to redraw everything on each frame. Maybe we should have an update() method and corresponding API functions which would allow the coordinates of elements to be updated without having to redraw anything? Of course this would complicate the API, but it should also speed things up quite a bit. Food for thought...

I've thought about this too: I think a nicer solution would be to have a "Drawable" object which has a drawing loop independent from the main drawing loop. So that you could render that separately, without having to update everything. I think that's easier to wrap your head around than having to update things.

So basically, you'd have a "drawable" type with it's own paint(g) and repaint() functions. Then you can call "add_drawable" or "remove_drawable" on your object. Then, call repaint() on your drawable whenever it needs to be repainted. A limitation of this would be that it always draws on top of your normal paint(g) function. But you can work around that by just defining 2 drawables.

Possibly, you could even do a full Subcomponent class, and pass on mouse events as well... But not sure about that yet.

[agraef]

Right. That sounds a lot like the grouping mechanism we have in purr-data's svg data structures. I admit that this is much better than my half-baked update idea. :)

But that's something for the future. I'm really happy with the features that we already have! That gives me lots of great new stuff to talk about in the tutorial. A chapter about signals and graphics is the next thing on my TODO list.

[agraef]

Once I'm done with the tutorial, I might just turn that chapter into a paper about the new stuff for the Linux Audio Conference next year at Lyon. Yeah, LAC is back! :) I think we owe it it the Linux audio and Pd crowd to present this there. If you're interested and maybe plan to attend, we could also do the paper together.

Also, I just released 0.12.3 with a few more bugfixes and improvements (mostly in the purr-data part).

[timothyschoen]

Once I'm done with the tutorial, I might just turn that chapter into a paper about the new stuff for the Linux Audio Conference next year at Lyon. Yeah, LAC is back! :) I think we owe it it the Linux audio and Pd crowd to present this there. If you're interested and maybe plan to attend, we could also do the paper together.

I would definitely be interested. This would be for June 2025 right?

[timothyschoen]

Oh, I received a bug report that some special characters (ě, š, č, ř, ž,á, and probably more) don't work when passed in as symbols to a pdlua inlet. This is very unfortunate for people who have this character in their user name, it makes it impossible to use file paths.

This was reported in plugdata, so I'll check if this also affects other pd flavours, but I'd assume so. I'll open an issue for it once I've verified that. I think that's worth fixing.

[agraef]

I would definitely be interested. This would be for June 2025 right?

That would be great! Yep, June 26-28. Papers are due March 24, 2025, so still ample time. :)

Oh, I received a bug report that some special characters (ě, š, č, ř, ž,á, and probably more) don't work when passed in as symbols to a pdlua inlet.

I don't think so. Could be a local issue on the machine, maybe localization issue on an ancient Windows version? Or maybe user error. Ask for a minimal patch exhibiting the issue. :)

In any case, it works fine over here with both vanilla and purr-data, on Linux (Arch), Mac (Ventura), and Windows (11). Here's a test patch: foo.zip

[agraef]

(Of course, I'm using UTF-8 here. I'm not sure how well any Pd flavor would cope with weird encodings.)

[agraef]

Sorry, forgot one line in the Lua script to make bang work as intended. Here's the corrected version: foo.zip

[agraef]

Hmm yes, I can reproduce that issue with vanilla (0.55.0) on Windows now. My test example works fine with pd-lua 0.11.6, but crashes Pd with 0.12.3 already while loading the patch. Works fine with the same pd-lua version in Purr Data, though. My guess would be that it's something in the object initialization in the vanilla part of the implementation. Maybe an uninitialized member variable in the pdlua object somewhere? Must be something recent, as 0.11.6 appears to work fine.

Maybe try with plugdata, so that we have another data point.

[agraef]

Crashes also without the message with the international characters in vanilla. So maybe this is another bug.

[timothyschoen]

Also struggling to reproduce the issue here. He specifically had issues with these symbols in file paths, so I'm trying this:

But it seems to work fine (on plugdata + macOS). I've asked for more details on how exactly to reproduce it.

[agraef]

Well, the crashes I had with vanilla 0.55.0 were simply due to using a pd-lua GH CI build against 0.53. After re-compiling against 0.55 everything worked fine. I'll update the pd-lua Windows CI build shortly.

He specifically had issues with these symbols in file paths, so I'm trying this:

Yeah, nothing to see there, works fine over here, too. You'll have to ask the guy for a test patch that exhibits the issue he's seeing, so that we can figure out what's going on there.