Advanced Options

sidekiq offers a few advanced options:

Location of Redis

By default, sidekiq assumes Redis is at localhost:6379. This is fine for development but for many deployments you'll need to point sidekiq to an external Redis server and an optional namespace by throwing this in config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { :url => 'redis://redis.example.com:7372/12', :namespace => 'mynamespace' }
end

Next, you need to configure the Sidekiq client, which is similar. If you're using the client with a single-threaded Rails (or other ruby) process, add a size of 1, which will provide one Redis connection for the client:

Sidekiq.configure_client do |config|
  config.redis = { :url => 'redis://redis.example.com:7372/12', :namespace => 'mynamespace', :size => 1 }
end

If you're running a multi-threaded Rails/ruby process, the default size is (Sidekiq.options[:concurrency] + 2), which is a good default. You can lower this if you need to, but setting it too low will cause the client threads to fight over too few connections.

Queues

By default, sidekiq uses a single queue called "default" in Redis. If you want to use multiple queues, you can pass them to sidekiq with an optional weight. A queue with a weight of 2 will be checked twice as often as a queue with a weight of 1:

sidekiq -q critical,2 -q default

You can specify a queue to use for a given worker just by declaring it:

class ImportantWorker
  include Sidekiq::Worker
  sidekiq_options :queue => :critical

  def perform(*important_args)
    puts "Doing critical work"
  end
end

Concurrency

You can tune the amount of concurrency in your sidekiq process. By default, sidekiq creates 25 Processors. If you have a lot of I/O heavy processing, why not try 100?

sidekiq -c 100

Note that ActiveRecord has a connection pool which needs to be properly configured in config/database.yml to work well with heavy concurrency. Set the pool setting to something close or equal to the number of Processors:

production:
  adapter: mysql2
  database: foo_production
  pool: 25

Connection Pooling

sidekiq includes the connection_pool gem which your Workers can use. With a connection pool, you can share a limited number of I/O connections among a larger number of threads.

class HardWorker
  include Sidekiq::Worker

  REDIS_POOL = ConnectionPool.new(:size => 10, :timeout => 3) { Redis.new }
  def perform(args)
    REDIS_POOL.with_connection do |redis|
      redis.lsize(:foo)
    end
  end
end

This ensures that even if you have lots of Processors, you'll only have 10 connections open to Redis.

Airbrake

If you have Airbrake configured for your Rails 3 application, sidekiq will use it to send any exceptions caught during processing. The message will also be sent as part of the context for easier debugging.