error when run task 08 that is OSError: Unable to synchronously open file (file signature not found)

[Luckaswww]

Summary

When this iteration reaches the fp phase, the OSError: Unable to synchronously open file (file signature not found) problem occurs.

NFO:dpgen:-------------------------iter.000026 task 08--------------------------
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
warnings.warn(
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:Try to parse from vasprun.xml
INFO:dpgen:failed frame: 21 in 509 4.13 %
Traceback (most recent call last):
File "/public/software/apps/miniconda3/envs/dpgen/bin/dpgen", line 10, in
sys.exit(main())
^^^^^^
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/main.py", line 255, in main
args.func(args)
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/generator/run.py", line 5394, in gen_run
run_iter(args.PARAM, args.MACHINE)
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/generator/run.py", line 4749, in run_iter
post_fp(ii, jdata)
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/generator/run.py", line 4622, in post_fp
post_fp_check_fail(iter_index, jdata)
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/generator/run.py", line 4117, in post_fp_check_fail
sys_paths = expand_sys_str(ii)
^^^^^^^^^^^^^^^^^^
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/util.py", line 67, in expand_sys_str
with h5py.File(root_dir, "r") as f:
^^^^^^^^^^^^^^^^^^^^^^^^
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/h5py/_hl/files.py", line 562, in init
fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/h5py/_hl/files.py", line 235, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 102, in h5py.h5f.open
OSError: Unable to synchronously open file (file signature not found)

DeePMD-kit Version

DeePMD-kit v3.0.0b0

Backend and its version

dpdata :0.2.19; dpgen:0.12.1

Python Version, CUDA Version, GCC Version, LAMMPS Version, etc

python = 3.12 h5py=3.11.0

Details

Hello,everyone. I'm very confused. When this iteration reaches the fp phase, the OSError: Unable to synchronously open file (file signature not found) problem occurs. I manually modified it in the record, and the next round still brought up OSError: Unable to synchronously open file (file signature not found). I don't know what causes this problem and how to solve it. Please help me.

[njzjz]

Could you take a look what is in your 02.fp directory?

[Luckaswww]

Could you take a look what is in your 02.fp directory?

Thank you very much for your reply. I processed different MD trajectories in batches. Firstly, I run six MD trajectory and fp from1 to 6 that form 01.model_devi.bk000 and 02.fp.bk000 folder, respectively. Next, I run six MD trajectory and fp from 7 to 12 that form 01model_devi and 02.fp folder. Finally, I put the data.00x file from 02.fp.bK001 into 02.fp. Now, 02.fp folder have data.001 ~ data.012 files and some task.007.xxxx ~ 012.xxxxx files

[niuniu800]

It is possible that the data folder obtained from your first-principles calculation is missing during some iteration process, or the deepmd folder generated by the init function does not exist. You may want to check on that.
The script used is as follows:
import h5py
import os

file_paths = [
"iter.000010/02.fp/data.000",
"iter.000010/02.fp/data.001",

# add more folder path

]

for file_path in file_paths:
if os.path.exists(file_path):
try:
with h5py.File(file_path, "r") as f:
print(f"{file_path} is a valid HDF5 file.")
except OSError as e:
print(f"Error opening {file_path}: {e}")
else:
print(f"File does not exist: {file_path}")

[Luckaswww]

OK. I'll try that right away. Thank you very much for you reply. I really appreciate your help.

[Luckaswww]

Hello! I've already done this script run. this result show that is "Error opening iter.000026/02.fp/data.000: [Errno 21] Unable to synchronously open file (file read failed: time = Wed Dec 4 18:45:28 2024, filename = 'iter.000026/02.fp/data.000', file descriptor = 4, errno = 21, error message = 'Is a directory', buf = 0x7fffffe74990, total read size = 8, bytes this sub-read = 8, bytes actually read = 18446744073709551615, offset = 0)".

I also tested the previous normal iteration using this script, and the output was the same as what I observed now. It's perplexing that in previous iterations, the 26th iteration proceeded normally after the 25th. However, now, after round 26, iteration 27 cannot continue, and instead, I receive this error. This script is following:

import h5py
import os

file_paths = [
"iter.000026/02.fp/data.000",
"iter.000026/02.fp/data.001",
"iter.000026/02.fp/data.002",
"iter.000026/02.fp/data.003",
"iter.000026/02.fp/data.004",
"iter.000026/02.fp/data.005",
"iter.000026/02.fp/data.006",
"iter.000026/02.fp/data.007",
"iter.000026/02.fp/data.008",
"iter.000026/02.fp/data.009",
"iter.000026/02.fp/data.010",
"iter.000026/02.fp/data.011",
"iter.000026/02.fp/data.012",
]

for file_path in file_paths:
if os.path.exists(file_path):
try:
with h5py.File(file_path, "r") as f:
print(f"{file_path} is a valid HDF5 file.")
except OSError as e:
print(f"Error opening {file_path}: {e}")
else:
print(f"File does not exist: {file_path}")

[njzjz]

File "/public/software/apps/miniconda3/envs/dpgen/lib/python3.12/site-packages/dpgen/util.py", line 67, in expand_sys_str
with h5py.File(root_dir, "r") as f:

Please modify this file and print(root_dir), which I think is the quickest way to locate the problem.