handling of dollar args

[ben-wes]

this is more of a discussion since it's not a certain issue and probably there are different ways of handling this. therefore the rather generic title.

---

... the more i look into this, the more complex it seems to become for me. one rather simple use case might be:

• user wants to make his pd_lua object's receive name configurable through a message

this can be easily done with a mechanism like (omitting all checks here):

function test:in_1_receive(x)
  self.recv:destruct()
  self.recv = pd.Receive:new():register(self, x[1], "receive")
end

it becomes more challenging though if there are dollar arguments involved. a typical option would be a message like receive \$0-foo to set the name to $0-foo, where $0 gets internally expanded to the actual canvas id. this is the mechanism that's common for Pd's gui objects like toggle, sliders etc.

if i send receive \$0-bla to the mechanism above, the dollar argument isn't expanded, which is probably expected, since $0 is obviously dynamic and shouldn't be forwarded in its expanded form, which would negate its purpose. but in that case, some kind of canvas_realizedollar()¹ function would be needed on the lua side, i assume?

when i use set_args() with the received symbol, it ends up in the patch file as \\\$0-foo - which means that the backslash gets escaped as well. might be expected, but prevents Pd from properly expanding the symbol on load.

attaching a pdlua object and patch for testing here² - the updated arguments of [test_dollar] are obviously only visible on reload or cut/paste.

Footnotes

1. https://github.com/pure-data/pure-data/blob/master/src/g_canvas.c#L279-L292 ↩

2. test_dollar.zip ↩

[agraef]

Well, canvas_realizedollar is in g_canvas.h which, at least at the time of this writing, is among the public header files, so in principle it would be possible to expose it in pd-lua's Lua API in some way.

But I don't really see why you need to to expand the symbol on the Lua side when Pd can do it just fine. Admittedly, the [bang( - [f $0] - [receive $1-foo( - [your Lua object] construct is a bit clunky, but it's what Miller's mandates in his html manual. (Incidentally, the clumsiness of this construct is probably the reason why in Pd-l2ork / Purr Data we can just write [receive $0-foo( and be done with it.)

Regarding the extra level of escapes in set_args, I'm sure that this is not in set_args itself, so it must happen inside Pd's guts (probably somewhere in the binbuf operations). Alas, I don't think that there's much that we can do about that. :(

Finally, I'm afraid that escaping $ and other special chars in object and message boxes, the way you have done here, is a bit of a murky area. This isn't something that has always worked, and Purr Data still doesn't have it to this day (although I'm currently looking into this).

[ben-wes]

thanks for your quick response, albert! i'll try to elaborate a bit more on the use case.

But I don't really see why you need to expand the symbol on the Lua side when Pd can do it just fine.

certainly true - i can achieve a patch-local receiver name in other ways already:

• if i create a lua object with a $0 argument, i will get the expanded symbol on the lua side when the object is initialized - so this works well and whenever i reload the patch
• if i don't set the receiver on creation, i can still set it the way you describe via [f $0] -> [receive $1-foo( - but this would need to be done whenever i load the patch, too
• the case i'm addressing here is when i want to change this parameter only once after creation, which might be nice to have for more complex, graphical objects (where the arguments are not visible). in that case, the only option is through messages. but if i set the arguments with the expanded symbol from the pd side, it will obviously be useless after reopening the patch since its id will change.

Regarding the extra level of escapes in set_args, I'm sure that this is not in set_args itself, so it must happen inside Pd's guts (probably somewhere in the binbuf operations).

likely so, yes. the iemguis somehow manage to handle that differently. i did not yet fully understand what's going on there - but of course, this comes with a bit of complexity if the expanded symbols are used, but the unexpanded ones get stored. this seems related - not sure though:
https://github.com/pure-data/pure-data/blob/master/src/g_all_guis.c#L246

[porres]

but it's what Miller's mandates in his html manual

where do we have this in the manual? Section 2.4.5 mentions escaping characters and how it can be used with dollar signs file:///Applications/Pd-0.55-1.app/Contents/Resources/doc/1.manual/resources/chapter2.htm#s2.4.5 but the use case is not clear, it should and could be better presented at 2.7.5 file:///Applications/Pd-0.55-1.app/Contents/Resources/doc/1.manual/resources/chapter2.htm#s2.7.5 though.

But the thing is that escaping dollar signs here in this case is just crucial and needed.

I'm afraid that escaping $ and other special chars in object and message boxes, the way you have done here, is a bit of a murky area. This isn't something that has always worked

It's not murky though, it's officially documented and works pretty fine. The use case is also documented in the help files of iemguis.

Anyway, the fact that messages can expand $0 in Purr data is not related to this issue. The problem is not to have a way that $0 works in messages, simply not at all. We just need a literal escaped $0 and this can be achieved/constructed in a message or an object box, it doesn't matter.

Well, Ben already explainded the use case, but let me also explain this.

We want to set something like a receive name with "$0" in it. For the object to store "$0" literally, it needs an unexpanded "$0" symbol. The only way to achieve that is if we escape the dollarsign. Then it's ok, the object stores that argument as a literal "$0" and not an expanded symbol.

Now, in order to expand the $0 symbol we need the "canvas_realizedollar" function, and we just need to support it on the lua side

if i create a lua object with a $0 argument, i will get the expanded symbol on the lua side

how does that happen without "canvas_realizedollar"?

cheers

[ben-wes]

Section 2.4.5 mentions [...]

@porres clickable links would be nice in these cases. ;)

• https://msp.ucsd.edu/Pd_documentation/resources/chapter2.htm#s2.4.5
• https://msp.ucsd.edu/Pd_documentation/resources/chapter2.htm#s2.7.5

[...] escaping dollar signs here in this case is just crucial and needed.

this is actually not the issue though. i obviously need to send the escaped version in the message anyway (like receive \$0-foo). there is not a lacking escaping, but a double escaping if i just forward the incoming atoms to set_args() - the backslash also gets escaped and therefore Pd won't expand the dollar symbol when initializing the object next time (it's saved as test_dollar \\\$0-foo; in the pd file).

Important

EDIT: correcting the following paragraphs. i have no idea what i did there before, but self:set_args({"\$0-foo"}) is obviously not possible!

but actually (thank you both for making me reconsider this!), i now realized that i can achieve correct escaping in the pd file if i escape it manually, which can be done in Lua: if i do self:set_args({"\$0-foo"}), i get the correct result. for convenience, there could be a Lua function to achieve this - but it's not crucial.

so one the only thing that i currently can't do on the Lua side is expanding the dollar symbols (with some realizedollar() function) to immediately apply such a receive name.

how does that happen without "canvas_realizedollar"?

@porres : AFAICT, this still happens on the Pd side. the pd_lua object gets the expanded arguments in its initialize() function. you can try that with the examples i included above.

[ben-wes]

@agraef : i tried exposing canvas_realizedollar() now. works for me. not sure if that's the correct way though - let me know if this looks good to you:

master...ben-wes:pd-lua:feature/realizedollar

tested with:

function test_dollar:in_1_test(x)
  pd.post("received symbol: " .. x[1])
  pd.post("expanded symbol: " .. self:canvas_realizedollar(x[1]))
end

sending a message test \$0-foo in Pd gives me the following log output:

received symbol: $0-foo
expanded symbol: 1003-foo

[ben-wes]

i added another commit in my branch now to avoid the symbol conversion for set_args(): 48f0053

... not sure if this might break other relevant cases though. ;)
@timothyschoen : since you added that feature, maybe you see potential problems with this change?

currently the problem is that i have no way to achieve \$0-foo as an argument in the pd file. the symbol conversion makes \\\$0-foo from this. i still don't grasp all the details here - but i assume that setting arguments might require this "lower level" string handling?

with these changes, i could handle $0 names for receivers by simply doing something like this:

function test_dollar:set_receiver(name)
  if self.recv then self.recv:destruct() end
  local name_expanded = self:canvas_realizedollar(name)
  self.recv = pd.Receive:new():register(self, name_expanded, "receive")
  self:set_args({name})
  pd.post(string.format("test_dollar: receiver name set to '%s', receiving from '%s'", name, name_expanded))
end

[porres]

clickable links would be nice

damn, thought they'd be

this is actually not the issue though.

Not in Vanilla or lua of course, but it is an issue for Purr Data, which needs to have this implemented

AFAICT, this still happens on the Pd side.

that's what I thought, we couldn't expand dollar signs in lua

i tried exposing canvas_realizedollar() now. works for me.

Yay!

[ben-wes]

that's what I thought, we couldn't expand dollar signs in lua

when given as creation arguments, one can already use the expanded dollar arguments in Lua as i mentioned above. but obviously, these other options here would be nice if the code looks good to @agraef and @timothyschoen!

btw ... sorry for the confusing title change (and undo)! i hadn't checked how all of this behaves in an abstraction with $1 etc. but it actually does work there as well. if there is no $1 value, it just remains unexpanded.

the only thing i currently don't understand in my own tests: when i'm setting the arguments like in the following lua object, i would expect it to set an expanded version on initialization (since this it not properly managed in this code) when the patch contains a [test_dollar $0-foo] object ... for some reason, i still have \$0-foo in the pd file after saving. that's cool - but i'm not sure why. :)

local test_dollar = pd.Class:new():register("test_dollar")

function test_dollar:initialize(name, args)
  self.inlets = 1
  self:set_receiver(args[1])
  return true
end

function test_dollar:set_receiver(name)
  if self.recv then self.recv:destruct() end
  local name_expanded = self:canvas_realizedollar(name)
  self.recv = pd.Receive:new():register(self, name_expanded, "receive")
  self:set_args({name})
  pd.post(string.format("test_dollar: receiver name set to '%s', receiving from '%s'", name, name_expanded))
end

function test_dollar:receive(sel, atoms)
  pd.post(string.format("test_dollar: received '%s %s'", sel, table.concat(atoms, " ")))
end

function test_dollar:in_1_receive(x)
  self:set_receiver(x[1])
end

[agraef]

Ben, sorry I got side-tracked with finishing off the upcoming Purr Data 2.19.4 release, still need to look at all your remarks and your PR.

But FYI, I just backported all the \-quoting stuff from vanilla for 2.19.4, so all these escaped dollar symbols etc. should now work pretty much like they do in vanilla.

[agraef]

for some reason, i still have $0-foo in the pd file after saving. that's cool - but i'm not sure why. :)

That's the way binbuf_savetext() works. It escapes ';', ',', and '$' in objects, which gets unescaped when loading the object so that you get [test_dollar $0-foo] again -- the object itself then expands the $0 during instantiation. Otherwise the binbuf parser would give you an immediate expansion like [test_dollar 1003-foo] which is obviously not what you want; you can try that for yourself by removing the \ in a text editor.

But enough already about backslashes -- implementing them in Purr Data made my brain hurt, so I hope that I never have to revisit that code again. ;-) As we all know from shell usage, \-quoting gets insane pretty quickly (do I need single, double, or quadruple escapes now? -- that kind of thing), so it's not something that you want to think about all that often.

[agraef]

That's the way binbuf_savetext() works.

Or some similar routine which helps with the saving of all the "text" objects on a canvas. In Pd, you can never be too sure where something actually happens, because the internals are insanely complicated and most of the time functions just delegate their task to other functions, possibly in an entirely different module. That's what we call the "great hall of mirrors". "Lasciate ogne speranza, voi ch'intrate." 😄

[agraef]

Feature added in #54.