Question: Periodic BrokenBarrierError

[fpf3]

I have an application where I periodically call an async function that lives in JS land, poll().
It calls it from the main runtime of Python, not an async context.
This makes various API calls and returns a promise for the results of those calls.

It works for the most part, but occasionally I need to restart the application because the call to poll results in BrokenBarrierError.
I read through a bit of stdlib.d.ts, and it looks like the cases where this can happen is when something calls reset() or when something calls abort(). Is there a smarter way that I should be handling this?

Should I make sure that all promises on the JS side have returned before trying to inspect them in Python?
Should I have something running in the async Python context to interact with async javascript objects?
How do I minimize my exposure to this issue?

Any guidance you can provide is appreciated!

[fpf3]

I think the issue is that the Executor is timing out waiting on javascript. Is there a way to increase the timeout, or better yet let it sit forever?

[extremeheat]

Hello

Can you provide a stack trace (or reproducible code sample) for when you get the error? It's possible there's a race condition somewhere. To make things simple internally we automatically await everything turned over from JS to Python.

As for the timeout issue: if you do a function call with the keyword argument timeout=None at the end, no timeout check is done.

fn(timeout=None)
# with normal args + no timeout:
fn('normal', 'arguments', timeout=None)

[fpf3]

I think, honestly, a simpler explanation is that the API I'm calling into is not responding due to load.

I will try and do some more digging, and get back to you soon.

[kencyke]

@extremeheat @fpf3

I encountered a very similar issue where BrokenBarrierError occurred during a long-running process that repeatedly invoked a JavaScript async function in Python's future objects created by ThreadPoolExecutor with max_workers=16.

I suspect that the timeout for the Barrier is too short for our use case.

JSPyBridge/src/javascript/events.py

Lines 179 to 180 in bd17b0d

	lock, timeout = self.requests[r]
	barrier = threading.Barrier(2, timeout=5)

To address this, I am considering the following adjustments:

lock, timeout = self.requests[r]
barrier = threading.Barrier(2, timeout=timeout)

Would this approach resolve this issue without compromising the overall architecture?

[extremeheat]

Sure, you can make a PR if you can confirm it fixes your use case

[kencyke]

Thank @extremeheat for your encouragement: #35 (comment).

At the moment, I cannot confirm whether my hypothesis is correct.
Your suggestion (#35 (comment)), combined with some more work on our code, might resolve our issue.

If necessary, I will submit a PR. Thank you for the excellent work on this project.