Problems and Troubleshooting

Threading

Sidekiq is multithreaded so your Workers must be thread-safe.

Thread-safe libraries

Most popular Rubygems are thread-safe in my experience. A few exceptions to this rule:

• right_aws
• aws-sdk (According to a somewhat old post in the discussion group, this gem is thread-safe with the exception of the use of autoload. More details here. Explicitly calling AWS.eager_autoload! during initialization should allow it to be used with Sidekiq)
• aws-s3 (For S3 and other AWS work, use Fog instead, it is substantially better and under active development (aws-s3 is old and hasn't been updated in years). There is a guide to using S3 with Fog available, and it has been tested to be thread safe).
• basecamp

Some gems can be troublesome:

• pg (the postgres driver, make sure PG::Connection.isthreadsafe returns true)
• RMagick (see #338, try mini_magick instead)
• therubyracer, versions before 0.11 can cause Sidekiq to hang

Writing thread-safe code

Well-factored code is typically thread-safe without any changes. Always prefer instance variables and methods to class variables and methods, for instance. Also remember that Ruby's require statement is not atomic, so you should require any files your threads will need before your worker starts up, as explained in this Stack Overflow answer.

Any other tips or notes? Please add it to this page.

Forking

When using Passenger or Unicorn, you should configure the Sidekiq client within a block that runs after the child process is forked. If you use custom connection arguments like namespace for the server you must adjust the below code snippets to fit your setup:

# config/unicorn.rb
after_fork do |server, worker|
  Sidekiq.configure_client do |config|
    config.redis = { :size => 1 }
  end
end

# config/initializers/sidekiq.rb
if defined?(PhusionPassenger)
  PhusionPassenger.on_event(:starting_worker_process) do |forked|
    Sidekiq.configure_client do |config|
      config.redis = { :size => 1 }
    end if forked
  end
end

ActiveRecord

Take care to avoid unsafe code related to the ActiveRecord database connection or connection pool. Calls like verify_active_connections! manipulate the ConnectionPool in a thread-unsafe way. Avoid these calls from inside of your jobs' perform method. See issue #267 for an example.

"Cannot find ModelName with ID=12345"

Sidekiq is so fast that it is quite easy to get transactional race conditions where a job will try to access a database record that has not committed yet. The clean solution is to use after_commit:

class User < ActiveRecord::Base
  after_commit :greet, :on => :create

  def greet
    UserMailer.delay.send_welcome_email(self.id)
  end
end

Note: after_commit will not be invoked in your tests if you have use_transactional_fixtures enabled.

If you aren't using ActiveRecord models, use a scheduled perform to run after you can be sure the transaction has committed:

MyWorker.perform_in(5.seconds, 1, 2, 3)

Either way, Sidekiq's retry mechanism's got your back. The first time might fail with RecordNotFound but the retry will succeed.

Heroku + Redis To Go

"ERR max number of clients reached"

You've hit the max number of Redis connections allowed by your plan: https://addons.heroku.com/redistogo

Limit the number of redis connections per process in config/sidekiq.yml. For example, if you're on Redis To Go's free Nano plan and want to use the Sidekiq web client, you'll have to set the concurrency down to 3.

:concurrency:  3

See #117 for a discussion on the topic.

Why does the Sidekiq Web UI look terrible / not render correctly in production but works fine in development?

Sidekiq Web wants to serve CSS/JS assets out of the gem. Your production web server is not forwarding CSS/JS requests to your app so Sidekiq Web can serve them but instead returning a 404 if they aren't found on the filesystem.

The workers are not starting

If you are migrating from Resque make sure there the Redis database does not contain any old tasks. You can clear all data with redis-cli FLUSHALL.

Another common problem is that you might have defined a namspace in Sidekiq.configure_server but not in Sidekiq.configure_client or named it something else. Also make sure to configure both!

Too many connections to MongoDB

If you are using Mongoid you'll also want to use the kiqstand middleware to properly disconnect workers so your connections aren't overloaded.