Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to target (i) memcached servers that support the binary protocol over UDP, and (ii) resource-contrained memcached servers (supporting only short keys); and to evaluate the latency of each transaction. #12

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

niksu
Copy link
Contributor

@niksu niksu commented Jun 1, 2016

This PR contains three extensions to memtier_benchmark:

  1. --transaction_latency command-line flag causes memtier_benchmark to print the raw latency of each operation.
    "Latency" here is the time taken for the server to carry out an operation (GET or SET) from the client's perspective.
    The "client's perspective" might involve two sorts of overhead: the trip-time from the server to the client, and the time taken for the client's architecture (including its network stack) to make the result of the operation available to the client.
    This patch adds an flag in memtier_benchmark's configuration, and if this flag is set then the latency is printed to stdout.
  2. --key-width command-line parameter allows the user to specify the width of the keys that are generated by memtier_system.
    This is intended to be used on constrained servers that don't support the standard key-width (250 bytes), and that support smaller keys.
    This patch adds a range-check on key sizes that are requested by the user (to ensure that they don't exceed the maximum, 250), and modifies the behaviour of object_generator to take into account the requested key width. (If not key width is specified by the user, then memtier_benchmark defaults to the default maximum size, 250 bytes.) Finally, runtime checks are made to ensure that the generated keys do not exceed the maximum size requested by the user.
  3. --udp command-line flag causes memtier_benchmark to generate traffic over UDP, rather than its usual TCP transport. This only works with the memcached binary protocol -- that is, it won't work with redis or memcached_text. This patch's code threads a socktype value, defaulting to SOCK_STREAM (for TCP) but which is changed to SOCK_DGRAM (for UDP) when the --udp flag is provided. The use of UDP has been added as a constructor parameter to protocol_factory. Using the binary memcached protocol over UDP necessitates a small header to be included -- this has been specified in libmemcached_protocol/binary.h. This header is generated by the function append_binary_udp_header, which currently produces a constant header -- this can be improved in future work if this feature is developed/needed further.

These patches were tested using memcached 1.4.25 (cloned at the tip memcached/memcached@6b65e45), which was run using the following command:
./memcached -p 0 -U 11211 -B binary

Many thanks to @marcinwoj who made the original extensions;
and to @luomai, @salvatorg, @noaz, @richardclegg, and @mgrosvenor who
worked on, or helped with, testing/experimentation/evaluation in which
these extensions were used.

niksu added 7 commits May 24, 2016 14:08
…tes. Added checks to ensure that the prefix and maximum key index will fit in this width, and that the width does not exceed the current maximum (250 bytes);
…ly tested with Memcached. When using the binary protocol, a header is inserted between the UDP header and memcached payload;
@itamarhaber
Copy link
Member

@niksu et al. (@marcinwoj, @luomai, @salvatorg, @noaz, @richardclegg, and @mgrosvenor) - firstly thank you and secondly wow.

Due to the outstanding scope of this PR it requires a thorough review before merging. That said, our developers are currently focused on Redis and the memcached side of the house gets lower priority. I can not commit on an ETA for the review but we'll keep this PR open.

If more people using memtier_benchmark want to provide their feedback on the feature, please feel free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants