hangs Pd on close

[umlaeute]

compilation

• Pd-0.51-2
• python3.8
• g++ 10.2.0-16
• Debian/GNU linux (amd64; sid)

run

pd -lib py -path scripts/ pd/pak.pd

everything works great. however:

• quitting Pd hangs
• (closing the patch does not; only when trying to quit Pd afterwards, it hangs)

[ahihi]

i can confirm that this happens for me as well, but not sure about the cause.

[marclava]

Same issue here, on my laptop with Ubuntu 20.04LTS and on a RPI 400 with Debian 11.
Last time I used pyext was 18 years ago, and it was working great...

[grybouilli]

Looking at the problem rn, it seems the problem occurs when building pyext with a multithhreaded build of flext. My bet is on a thread that isn't joined / exited correctly ; i'm still troubleshooting tho.

[grybouilli]

Looking at the problem rn, it seems the problem occurs when building pyext with a multithhreaded build of flext. My bet is on a thread that isn't joined / exited correctly ; i'm still troubleshooting tho.

My intuition seems to be right : if you comment the following line (~275) of pybase.cpp:
LaunchThread(quworker,NULL);

Pd doesn't hang on close. But of course, it disables the good functioning of threading in py/pyext. I'm digging into quworker and what causes the issue now.

[grybouilli]

I solved the issue and commited it on my fork; you can check the following pull request #8. See complete fix here b3b5878.

Explanation
When loading the py/pyext library into puredata, a call to LaunchThread(quworker,NULL); is made. This launches the function quworker to live forever in a thread. This would be fine as flext has a good management of threads, but the problem finds its source in the algorithm of the quworker function itself. The function relies on an infinite loop :

for(;;) {
        while((el = qufifo.Get())) {
            ++el->th->thrcount;
            {
                ThrLock lock(my);
                el->th->docall(el->fun,el->args);
                Py_XDECREF(el->fun);
                Py_XDECREF(el->args);
            }
            --el->th->thrcount; 
            qufifo.Free(el);
        }
        qucond.Wait(); // <--- the source of all our problems
    }

From my understanding, quworker manages the pool of threads to be executed by the diverse py/pyext externals. When no thread is created - i.e no python externals is running things in parallel to the main pd process-, the thread of quworker is on hold because the call to qucond.Wait(). Problem is, when puredata is closed, no signal is sent - with qucond.Signal() - hence the thread stays in a waiting state, leading the program to hang forever.

The solution

We need to make a call to Signal with qucond before it is destroyed - remember it's a static member of pybase, so it is destroyed when we leave pd's main - AND we need to make sure the thread won't be put into a waiting state again -i.e no more call to Wait().

The only solution I've found for now is to create a class which sole purpose is to be destroyed at some point before qucond, to send a signal and break the loop. The class is called ThrCtrl and holds a pointer to a ThrCond and a pointer to a bool. I added two static variables to pybase : a ThrCtrl and bool. I changed the infinite loop of quworker to a while loop which condition is the newly added boolean variable. The static ThrCtrl variable holds a pointer to that boolean and a pointer to qucond. In the destructor of ThrCtrl, we change the value of the boolean to false and we make a call to Signal.

This ensures to break the loop and hence, to let puredata exit when we make a call for it, without having to kill the process :)

[ahihi]

thank you @grybouilli for your contributions!

i just wanted to note that i am no longer working at SOPI and im not sure if this repo has a maintainer now. perhaps the best way forward would be to make a pull request to the original https://github.com/grrrr/py repo? the main reason i never got around to that was because the cmake build was still a bit messy, but it seems your fork has some improvements to that. it would be great to get the python3 support into the hands of more people.

[grybouilli]

thank you @grybouilli for your contributions!

i just wanted to note that i am no longer working at SOPI and im not sure if this repo has a maintainer now. perhaps the best way forward would be to make a pull request to the original https://github.com/grrrr/py repo? the main reason i never got around to that was because the cmake build was still a bit messy, but it seems your fork has some improvements to that. it would be great to get the python3 support into the hands of more people.

Sure enough, I will clean things a bit and make a PR asap :)