Race condition/Duplicates

[jpswade]

I've implemented this in Laravel, I've setup supervisord to run 4 workers for his on each server (there's 3).

This means there will be multiple threads running at the same time.

What I do with the queue is pick the item off the queue, parse it, then drop the entry into the database.

Laravel should do an update or Create, which means it will update if the entry already exists or create.

However, if there's a race condition where mysql is trying to do the same thing at the same time, you get the following error:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'xxx' for key 'xxx_id_unique'

I believe this is because multiple threads are picking up the same item off the queue at the same time.

There's multiple ways that this can be resolved, including switching to something like upserts at MySQL level, however I wanted to report this issue incase there's something I've missed.

Cheers.