-
Notifications
You must be signed in to change notification settings - Fork 0
Reliable queueing
This feature is available in Sidekiq Pro 0.9.0
Sidekiq uses BRPOP to pop a message off the queue in Redis. This is very efficient and simple but it has one drawback: the message is now gone from Redis. If Sidekiq crashes while processing that message, it is gone forever. This is not a problem for many but some businesses need absolute reliability when processing messages.
Sidekiq Pro offers an alternative strategy for message processing using Redis' RPOPLPUSH command which ensures that a crash will not result in lost messages. To enable "reliable fetch" you must tag each process on a machine with a unique index and require the strategy:
Start Sidekiq with a unique index for each process on the machine:
sidekiq -e production -i 0
sidekiq -e production -i 1
sidekiq -e production -i 2
Require the alternative fetch strategy:
Sidekiq.configure_server do |config|
require 'sidekiq/reliable_fetch'
end
When Sidekiq starts, you should see ReliableFetch activated:
INFO: Booting Sidekiq 2.6.2 with Redis at redis://localhost:6379/0
INFO: Running in ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2]
INFO: Sidekiq Pro 0.9.0, commercially licensed. Thanks for your support!
INFO: ReliableFetch activated
INFO: Starting processing, hit Ctrl-C to stop
Enjoy your newfound reliability!