FileNotFoundError: [Errno 2] No such file or directory: '/personal/dpgen/init bulk/iter.000002/00.train/fba971ce63ccb5a4d315ee901c5ba63ae9eebd88_back.zip'

[WYX-X]

Dear developers,
I have problems in using dpgen run in Bohrium. The dpgen:0.12.0 image is used. The training process has completed two iterations, and the following error appears in the process of iter.000002\00.train, which seems to be the lack of files. How to modify?

INFO:dpgen:-------------------------iter.000002 task 01--------------------------
DeepModeling
Version: 0.12.0
Path: /opt/mamba/lib/python3.10/site-packages/dpgen

Dependency
numpy 1.26.2 /opt/mamba/lib/python3.10/site-packages/numpy
dpdata 0.2.17 /opt/mamba/lib/python3.10/site-packages/dpdata
pymatgen unknown version or path
monty 2022.9.9 /opt/mamba/lib/python3.10/site-packages/monty
ase 3.22.1 /opt/mamba/lib/python3.10/site-packages/ase
paramiko 3.0.0 /opt/mamba/lib/python3.10/site-packages/paramiko
custodian 2023.10.9 /opt/mamba/lib/python3.10/site-packages/custodian

Reference
Please cite:
Yuzhi Zhang, Haidi Wang, Weijie Chen, Jinzhe Zeng, Linfeng Zhang, Han Wang, and Weinan E,
DP-GEN: A concurrent learning platform for the generation of reliable deep learning
based potential energy models, Computer Physics Communications, 2020, 107206.
Description
2024-09-01 11:45:16,189 - INFO : Find old submission; recover submission from json file;submission.submission_hash:b194c64184db13d11d528ceba91cf6dedd7f701a; machine.context.remote_root:.; submission.work_base:iter.000002/00.train;
2024-09-01 11:45:17,248 - ERROR : request error status_code:404 reason: Not Found body:

NoSuchKey The specified key does not exist. 66D3E34D244FB331364FE0B4 bohrium.oss-cn-zhangjiakou.aliyuncs.com 34881/33047/job/35215/11824024/out.zip 0026-00000001 https://api.aliyun.com/troubleshoot?q=0026-00000001
2024-09-01 11:45:17,283 - ERROR : request error status_code:404 reason: Not Found body:

NoSuchKey The specified key does not exist. 66D3E34D41DBDB373902F9A1 bohrium.oss-cn-zhangjiakou.aliyuncs.com 34881/33047/job/35215/11824024/out.zip 0026-00000001 https://api.aliyun.com/troubleshoot?q=0026-00000001
2024-09-01 11:45:18,311 - ERROR : request error status_code:404 reason: Not Found body:

NoSuchKey The specified key does not exist. 66D3E34E49A6C93934D0AB3D bohrium.oss-cn-zhangjiakou.aliyuncs.com 34881/33047/job/35215/11824024/out.zip 0026-00000001 https://api.aliyun.com/troubleshoot?q=0026-00000001
Traceback (most recent call last):
File "/opt/mamba/bin/dpgen", line 8, in
sys.exit(main())
File "/opt/mamba/lib/python3.10/site-packages/dpgen/main.py", line 255, in main
args.func(args)
File "/opt/mamba/lib/python3.10/site-packages/dpgen/generator/run.py", line 5411, in gen_run
run_iter(args.PARAM, args.MACHINE)
File "/opt/mamba/lib/python3.10/site-packages/dpgen/generator/run.py", line 4742, in run_iter
run_train(ii, jdata, mdata)
File "/opt/mamba/lib/python3.10/site-packages/dpgen/generator/run.py", line 864, in run_train
submission.run_submission()
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/submission.py", line 221, in run_submission
self.update_submission_state()
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/submission.py", line 345, in update_submission_state
job.get_job_state()
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/submission.py", line 831, in get_job_state
job_state = self.machine.check_status(self)
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/machines/dp_cloud_server.py", line 214, in check_status
self._download_job(job)
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/machines/dp_cloud_server.py", line 228, in _download_job
zip_file.unzip_file(target_result_zip, out_dir=self.context.local_root)
File "/opt/mamba/lib/python3.10/site-packages/dpdispatcher/utils/dpcloudserver/zip_file.py", line 81, in unzip_file
obj = ZipFile(zip_file, "r")
File "/opt/mamba/lib/python3.10/zipfile.py", line 1249, in init
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '/personal/dpgen/init bulk/iter.000002/00.train/fba971ce63ccb5a4d315ee901c5ba63ae9eebd88_back.zip'

and
2024-08-31 22:01:39,194 - ERROR : request error status_code:404 reason: Not Found body:

NoSuchKey The specified key does not exist. 66D322430F564C3037C3FCDC bohrium.oss-cn-zhangjiakou.aliyuncs.com 34881/33047/job/35215/11824024/out.zip 0026-00000001 https://api.aliyun.com/troubleshoot?q=0026-00000001
machine (1)(1).json

param.json

[njzjz]

request error status_code:404 reason: Not Found body:

This seems to be a Bohrium issue.

[WYX-X]

Thank you, the problem has been resolved.