Skip to content

Benchmark

Jump to bottom
Andrey Bulygin edited this page Jun 6, 2013 · 14 revisions

Home

Several benchmark test cases were performed. Only .NET clients that support pipelining and are listed on Redis.io took part in benchmarking.

Clients

Test cases

Pipelining mode and synchronous mode were tested. Pipelining offers the greatest performance and it fits well for asynchronous clients, so it is expected clients that are asynchronous by nature should win the competition. So, let's see.

  • Pipelining mode tests
    • Payload size dependent test cases. Measures the time that is needed to send SET command N times. Each command is pipelined. At the end of the test a single synchronous GET command is executed, and one condition should be fulfilled - the result of GET operation equal to a value in SET command. This test is performed with three payloads of different sizes.
    • Mixed packet size test case. The rules are the same. Middle-sized payload is used for SET command, and one INCR command is pipeliend on each iteration.
  • Synchronous mode tests
    • Exactlly the same test as for pipelining mode is used, but the SET command is not pipelined but sent directly to Redis. Only test case with small payload was performed.

The code of benchmars is available and is a part of the [solution] (https://github.com/andrew-bn/RedisBoost)

Benchmark results:

  • Small payload size average results(payload size is 1 byte, test is executed 5 times on 2'000'000 iterations)
    • redisboost ~6437ms
    • servicestack ~11371ms
    • booksleeve ~17006ms
    • csredis ~14911ms
  • Medium payload size average results(payload size is 2053 bytes, test is executed 5 times on 50'000 iterations)
    • redisboost ~8846ms
    • servicestack ~12142ms
    • booksleeve ~19612ms
    • csredis ~8833ms
  • Large payload size average results (payload size is 20012 bytes, test is executed 5 times on 10'000 iterations)
    • redisboost ~16971ms
    • servicestack ~20788ms
    • booksleeve ~17104ms
    • csredis - condition failed (Result == Payload)
  • Mixed payload size average results (test is executed 5 times on 50'000 iterations)
    • redisboost ~9031ms
    • servicestack ~12421ms
    • booksleeve ~19796ms
    • csredis ~8949ms
  • Sync execution on small payload size average results (test is executed 5 times on 10'000 iterations)
    • redisboost ~11193ms
    • servicestack ~10362ms
    • booksleeve ~10905ms
    • csredis (sync) ~10338ms

The interpretation of the results

The more efficiently network is used the faster the client is. Not only asynchronous clients were tested, ServiceStack.Redis is a synchronous one, but it showed good results, and it is very fast in pipelining mode.

The test case with large payload size shows that csredis has some internal problems and some unknown error occured within, so the last GET operation returned not expected data from Redis.

As for synchronous execution. All clients showed almost the same results. For csredis and for servicestack.redis synchronous client was used. For redisboost and booksleeve command execution was synchronized with Task.Wait() call.

Benchmark execution date - May 21, 2013

Clone this wiki locally