Skip to content

Race conditions can occur when initializing DB hash table #11

Open
@toolness

Description

@toolness

This was introduced by #10.

I thought perhaps this might not be possible if CREATE TABLE IF NOT EXISTS were atomic, but maybe it's not? In any case, all but one of the scheduled tasks last night failed with the following traceback:

Alas, an error occurred when loading the dataset `oath_hearings`.
Traceback (most recent call last):
File "load_dataset.py", line 254, in <module>
main()
File "load_dataset.py", line 250, in main
raise e
File "load_dataset.py", line 247, in main
load_dataset(dataset)
File "load_dataset.py", line 206, in load_dataset
dbhash = SqlDbHash(conn, 'nycdb_k8s_loader.dbhash')
File "/app/dbhash.py", line 62, in __init__
self._init_db()
File "/app/dbhash.py", line 71, in _init_db
"""
File "/app/dbhash.py", line 77, in _exec_sql
cur.execute(sql, params)
psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL: Key (typname, typnamespace)=(dbhash, 4093702) already exists.

Oof.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions