Handle job gone errors and improve promise management

[eduard-fa]

Changes

• Ignore job missing errors on data purging: prevents errors when the job is already deleted before the purging process
• Resolves a race condition that could occur when removing a completed promise from the promises list (removing job using potentially outdated index)
• Ensures that promises are properly removed from the list even if an error occurs during the purging process

Checklist

• Apply one of following labels; major, minor, patch or skip-release
• I've updated the documentation, or no changes were necessary
• I've updated the tests, or no changes were necessary

[kibertoad]

you may want to use allSettled instead of all with try-catch, if you want to wait for all promises to resolve regardless of how successful that is