Multithreading error.

[nihilityworld]

I have used the c_predict_api to build a predict api in the web. but when I conduct the pressure test using 50 requests in 1s, 49 return right result and 1 blocked(10 or 20 requests will also block 1 request). I find that it is blocked in MXPredSetInput and MXPredGetOutputI check the c_predict_api.cc and its include file. I find a unique_lock before a memcpy function and it will be unlock after the memcpy finished in MXPredSetInput and MXPredGetOutput. Why there is 1 request blocked? And how to slove it.

[frankfliu]

Thank you for submitting the issue! @sandeep-krishnamurthy requesting this be labeled.

[chsin]

I have the same issue where its hanging at https://github.com/apache/incubator-mxnet/blob/v1.2.0/src/engine/threaded_engine.cc#L395. I'm using v1.2.0, any fixes or updates?

[chsin]

More specifically, I'm running a server like the @nihilityworld and so far I've traced that the hanging is because:

• it never returns from nd.SyncCopyToCPU(data, size); in MXPredGetOutput https://github.com/apache/incubator-mxnet/blob/v1.2.0/src/c_api/c_predict_api.cc#L398.
• because it never returned from this->WaitToRead(); in NDArray::SyncCopyToCPU
  https://github.com/apache/incubator-mxnet/blob/v1.2.0/src/ndarray/ndarray.cc#L1975
• because it never returned from Engine::Get()->WaitForVar(ptr_->var); in inline void WaitToRead()
  https://github.com/apache/incubator-mxnet/blob/v1.2.0/include/mxnet/ndarray.h#L307
• because it never returned from finished_cv_.wait(lock, [this, &done]() { return done.load() || kill_.load(); in ThreadedEngine::WaitForVar
  https://github.com/apache/incubator-mxnet/blob/v1.2.0/src/engine/threaded_engine.cc#L395

[gartentrio]

Had the same problem. But after we ran model.predict() once in the main thread, the hangers disappeared. Maybe this helps.

[ddavydenko]

@mxnet-label-bot, [Thread Safety, Bug]

[JohnLee168]

Had dead lock problem too using c_predict_api.
I use 10 different PredictorHandles created by MXPredCreate() and for each PredictorHandle calling MXPredSetInput(), MXPredForward() and MXPredGetOutput() functions to inference in 10 threads, and after several hours, I got dead lock in MXPredForward, MXPredSetInput or MXPredGetOutput.

[PapaMadeleine2022]

meet the same error. need help...
@JohnLee168 @nihilityworld @chsin have you solved it?

@gartentrio can you describe the detail how to get it work?

[BogdanovKirill]

Same issues on v1.5. Did someone find a solution to the problem?

[slavah]

Anyone found solution for this issue?

[BogdanovKirill]

It seems that this problem is connected with:
#10948
I could say that when your program start another child process (any type of process, even something like "ps aux") in linux it could lead to hangs in
MXPredSetInput/MXPredGetOutput in another thread. Locking could help to avoid this bug.

[BogdanovKirill]

I think that this issue is a serious bug, not question.