Simulation Error

[npiroozan]

Hello,

I'm trying to run a LAMMPS simulation for silicon but encounter an error relating to the Allegro trained model. I'm using a CPU-only release of Pytorch, LAMMPS is integrated to libtorch through conda, lmp was built successfully, model was trained and deployed, and lammps input script was detailed properly using pair_style allegro.

However when trying to run the LAMMPS script the following error is observed:

Created 4032 atoms
using lattice units in orthogonal box = (0.0000000 0.0000000 0.0000000) to (38.014900 43.445600 48.876300)
create_atoms CPU = 0.000 seconds
Allegro is using device cpu
Allegro: Loading model from ../../../../allegro/work/Silicon/results/silicon-tutorial/si/log/silicon-deployed.pth
terminate called after throwing an instance of 'c10::Error'
what(): Unrecognized data format
Exception raised from load at /localdisk/npirooza/Allegro_Source/pytorch/torch/csrc/jit/serialization/import.cpp:449 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x55 (0x7efdc5b622c5 in /localdisk/npirooza/anaconda3/envs/Allegro_Source/lib/python3.9/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0xb1 (0x7efdc5b5f461 in /localdisk/npirooza/anaconda3/envs/Allegro_Source/lib/python3.9/site-packages/torch/lib/libc10.so)
frame #2: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&) + 0x191 (0x7efdbe01e6a1 in /localdisk/npirooza/anaconda3/envs/Allegro_Source/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so)
frame #3: ../../../build/lmp() [0x7b1c19]
frame #4: ../../../build/lmp() [0x455dd8]
frame #5: ../../../build/lmp() [0x45992b]
frame #6: ../../../build/lmp() [0x459b73]
frame #7: ../../../build/lmp() [0x433828]
frame #8: __libc_start_main + 0x102 (0x7efdb9e27362 in /lib64/libc.so.6)
frame #9: ../../../build/lmp() [0x433e8e]

Aborted (core dumped)

I appreciate your time and apologies if this is a dumb mistake on my part :)

[Linux-cpp-lisp]

Hm... a weird error like this probably indicates an upstream backward/forward compatibility bug in PyTorch, which we have seen at least for the case of train in 1.13 and load in C++ in 1.11 (though with a different error). Is your libtorch the same version as your PyTorch for training?

See for example pytorch/pytorch#47917

[npiroozan]

Hello,

Thank you for taking the time to respond to the message. It seems this was as you indicated. A slight mismatch between the version of libtorch I was using and the torch version. I apologize for taking your time with this. Thank you again!

[Linux-cpp-lisp]

Good to know! Out of curiosity what were the torch and libtorch versions? Helps us to have a record. Unfortunately, while PyTorch in general made us hope that the deployed models would be entirely backward and forward compatible, PyTorch's own edge case bugs do appear to make not work sometimes.

[npiroozan]

Certainly, the torch version I am using now is the cpu release of 1.12.0. The libtorch version which gave the error was 2.0. I failed to notice that PyTorch updated the current release and paid the price for that. I also just tried libtorch 1.13.1 and it gave no error. I hope that helps!

[Linux-cpp-lisp]

Please note that we are only currently confident in PyTorch 1.11 due to various JIT bugs in 1.12 and 1.13 (and probably 2.0 also has the 1.13 bug). See mir-group/nequip#311.

[npiroozan]

Thank you, this is good to know! I will revert to 1.11.