sqlite3.ProgrammingError sometimes occurs with parallelization

[timcallow]

When using multiple cores with joblib, sometimes the following error is raised:

Exception during reset or similar
Traceback (most recent call last):
  File "/home/callow46/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 697, in _finalize_fairy
    fairy._reset(pool)
  File "/home/callow46/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 893, in _reset
    pool._dialect.do_rollback(self)
  File "/home/callow46/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 543, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139675633149760 and this is thread id 139675095406336.
Exception closing connection <sqlite3.Connection object at 0x7f08a8e3e030>

The code continues to run after the error is raised and it does not affect the results. However it's annoying since it doesn't look good (and I'm not sure but maybe is capable of causing something critical). It seems random as to whether this error appears or not.

It seems the sqlalchemy library is called indirectly through the mendeleev library imported into atoMEC. This only performs some initialization of the Atom object and joblib is supposed to work in a self-contained way, i.e. multiple threads should only be used during the diagonalization and then the code should return to single threading. So I'm not sure why joblib is interfering with sqalchemy. Either way it would be good to prevent this error from happening, or at the very least tell atoMEC just to ignore this error (I'm not sure how to do this since I don't know when exactly the error is induced).

@DanielKotik any ideas?