Document how to benchmark Dragonfly #101
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Hi @romange, please let me know if this is roughly what you had in mind, and what should be added / changed if not. |
docs/development/benchmarking.md
Outdated
|
|
||
| ```shell | ||
| sudo apt install linux-tools-common linux-tools-generic | ||
| sudo cpupower frequency-set --governor performance |
There was a problem hiding this comment.
Are you sure you need it on cloud instances?
i have never done it, actually.
There was a problem hiding this comment.
sorry I missed the sentence before :(
I would frame the whole page around the cloud or on-prem datacenter server. I do not advise benchmarking locally.
There was a problem hiding this comment.
Sure, I removed this whole paragraph with the command line example
|
|
||
| ## Choosing an Environment | ||
|
|
||
| A benchmark is done to assess the performance aspects of a system. In the case of Dragonfly, a |
There was a problem hiding this comment.
Was this paragraph generated by ChatGPT? :)
I thought about providing specific requirements of how to say reach 1M qps on m5 family instance.
- Specifiying Linux os version, even distribution, mentioning that Dragonfly runs best with iouring,
- talking about what's the minimal instance size that allows reaching 1M qps on m5.
- Talking about running the load test on a separate machine. machine size for the load test .
- Specifying that it is better to choose a bigger size for load test machine to avoid bottlenecks on the client side.
- Provide rule of thumb for
--threadsshould not be higher than number of vcpus on that machine. - Provide suggested configuration for memtier to reach 1M qps on Dragonfly.
- Provide suggested configuration + instance type to reach 2M qps on GCP instance etc
i.e. keep everything very technical and specific.
There was a problem hiding this comment.
or any other instance family and interesting target goal.
There was a problem hiding this comment.
It was not generated by ChatGPT, but I've been called worse :)
re/ 1: done
re/ 2: what is indeed the minimal instance?
re/ 3: I already talk about it below
re/ 4: done
re/ 5: done (but I think you meant --proactor_threads?)
re/ 6+7: do you have these? or would you like me to run them until I figure it out?
There was a problem hiding this comment.
Sorry 😞 , I was joking.
re/5 - I meant -t on memtier side. Dragonfly actually spans all the cpus automatically. memtier always uses 4 by default.
| traffic, you should definitely use them. That would be the closest estimation to what a real | ||
| production deployment with a backing Dragonfly would look like. | ||
|
|
||
| If, like many others, you do not (yet) have such a tool, you could either write your own tool to |
There was a problem hiding this comment.
Lets be opinionated and say - we usually use memtier - and this is how we do it.
I would not suggest folks to write their own loadtest tool :)
There was a problem hiding this comment.
We can also mention redis-benchmark - from my experience (have not been using it for the last two years) it's less efficient than memtier but it has more predefined loadtest options specific to redis.
but each such statement must be checked verified and run personally by you.
There was a problem hiding this comment.
I would actually suggest that users write their on load tests, as an end-to-end kind of thing, but that's really beside the point here :)
| --requests=<requests per client> | ||
| ``` | ||
|
|
||
| ## Having Troubles? Anything Unclear? |
There was a problem hiding this comment.
btw, every docs page has "edit page" button at the end...
There was a problem hiding this comment.
Shall I remove this then?
docs/development/benchmarking.md
Outdated
| Then, when you're done with the benchmark you could reboot your machine or run the following: | ||
|
|
||
| ```shell | ||
| sudo cpupower frequency-set --governor powersave |
There was a problem hiding this comment.
I would not suggest it unless you saw difference in Dragonfly's performance with and without this option on the cloud.
There was a problem hiding this comment.
Removed as per discussed
docs/development/benchmarking.md
Outdated
| consider running the benchmark in a similar way. | ||
|
|
||
| In practice, it means that any other systems in your setup (like other services & databases) should | ||
| run in other machines. Importantly, also the software that sends the traffic should run in another |
There was a problem hiding this comment.
i would even make the last sentence bold or move it to the start because it's the most relevant sentence here for practical purposes.
|
|
||
| Dragonfly supports both `epoll` and [`io_uring`](https://en.wikipedia.org/wiki/Io_uring) Linux APIs. | ||
| `io_uring` is a newer API, which is faster. Dragonfly runs best with `io_uring`, but it is only | ||
| available with Linux kernels >= 5.1. |
There was a problem hiding this comment.
Dragonfly requires 5.10 or later for iouring. Before that iouring API was partial and not reliable.
| `io_uring` is available in Debian versions Bullseye (11) or later, Ubuntu 21.04 or later, Red Hat | ||
| Enterprise Linux 9.3 or later, Fedora 37 or later. | ||
|
|
||
| To find if your machine has `io_uring` support you could run the following: |
There was a problem hiding this comment.
since we require iouring starting from specific version the most straightforward approach would be to use uname -a in this case.
|
@chakaz is this ready to merge? |
|
I'm afraid not. It's going to take a while, as I'm actively learning how to do some more advanced benchmarking from Roman.. |
|
@chakaz should we close this PR? |
|
Yeah, let's close it. I'll create another PR at a later time. |
Fixes #60