You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I want to use python to call weka in parallel, but weke_python_wrapper3 uses javabridge, and javabridge is None in sub_f. It is not clear why.
follow code is a demo to show the problem I mentioned。
from joblib import Parallel, delayed
import weka.core.jvm as jvm
import javabridge._javabridge as _javabridge
def sub_f(j):
print(jvm.started)
print(_javabridge.get_env())
print(j)
def joblib_process():
print(jvm.started)
print(_javabridge.get_env())
res = Parallel(n_jobs=6, require='sharedmem')(
delayed(sub_f)(i) for i in range(6)
)
return res
if __name__ == '__main__':
jvm.start()
joblib_process()
jvm.stop()
The text was updated successfully, but these errors were encountered:
I am guessing the problem is the process model for joblib. Maybe Javabridge isn't open in the subprocess? See #176 for a tested multiprocess mechanism using Javabridge.
I am guessing the problem is the process model for joblib. Maybe Javabridge isn't open in the subprocess? See #176 for a tested multiprocess mechanism using Javabridge.
Thank you for your reply. As you said, the problem is in the process of parallelization. Now that the problem has been solved, thank you very much for your guidance.
Below is my corrected code.
with parallel_backend("multiprocessing", n_jobs=6):
res = Parallel(require='sharedmem')(delayed(sub_f)(j) for j in range(6))
I want to use python to call weka in parallel, but weke_python_wrapper3 uses javabridge, and javabridge is None in sub_f. It is not clear why.
follow code is a demo to show the problem I mentioned。
The text was updated successfully, but these errors were encountered: