Features

Server aliases support

The original problem can be found at https://github.com/twitter/twemproxy/issues/25 .

If you're using libmc in production. It's advised to add alias for each memcached instance, this will make it easier for you to maintain/migrate/monitor a memcached cluster.

A sample of using server alias:

import libmc


def libmc_from_config(config):
    '''
    Sample config:
    {
        'SERVERS': ['localhost:11211 mc-server111',
                    'localhost:11212 mc-server112'],
        'HASH_FUNCTION': 'crc32',
        'PREFIX': '',
        'CONNECT_TIMEOUT': 10,
        'POLL_TIMEOUT': 300,
        'RETRY_TIMEOUT': 5,
        'COMPRESSION_THRESHOLD': 1024
    }
    '''

    _HASH_FN_MAPPING = {
        'md5': libmc.MC_HASH_MD5,
        'crc32': libmc.MC_HASH_CRC_32,
        'fnv1_32': libmc.MC_HASH_FNV1_32,
        'fnv1a_32': libmc.MC_HASH_FNV1A_32
    }
    servers = config['SERVERS']
    hash_fn = _HASH_FN_MAPPING[config.get('HASH_FUNCTION', 'md5')]
    prefix = config.get('PREFIX', '')
    connect_timeout = config.get('CONNECT_TIMEOUT', 10)
    poll_timeout = config.get('POLL_TIMEOUT', 300)
    retry_timeout = config.get('RETRY_TIMEOUT', 5)
    comp_threshold = config.get('COMPRESSION_THRESHOLD', 0)
    client = libmc.Client(
        servers,
        comp_threshold=comp_threshold,
        prefix=prefix,
        hash_fn=hash_fn,
    )
    client.config(libmc.MC_CONNECT_TIMEOUT, connect_timeout)
    client.config(libmc.MC_POLL_TIMEOUT, poll_timeout)
    client.config(libmc.MC_RETRY_TIMEOUT, retry_timeout)
    return client

Lazy-connecting

The libmc client may be initialized but never used, for example, in a background daemon process. If we establish the TCP connections immediately after the client is initialized, there will be a lot of idle TCP connection to memcached server, which is not friendly. So, the internal TCP connections in libmc will not be established until the first command(mc.get, mc.set, ...) is called.

gevent compatibility

If you're using libmc under gevent, you can use greenify to patch the libmc to make sure it will not block.

import greenify
greenify.greenify()
import libmc
for so_path in libmc.DYNAMIC_LIBRARIES:
    assert greenify.patch_lib(so_path)

There's also a test case for gevent.

Large values

Memcached server will refuse to store value if size >= 1MB, if do_split is enabled, large value (< 10 MB) will be splitted into several blocks. If the value is too large (>= 10 MB), it will not be stored. This feature is enabled by default.

This feature is deprecated since retrieving a large value may raise high network bandwidth.

Auto failover

If failover is enabled, when a connection is broken, it will be mark as dead and ejected out of the consistent hash ring for a few seconds(5s by default, controlled by libmc.MC_RETRY_TIMEOUT). If failover is disabled, when a connection is broken,it will be mark as dead for a few seconds, but it will not be ejected out of the consistent hash ring. During the dead period, requesting keys routed to that connection(memcached server) will return failures.

Failed is disabled by default. If you're using libmc in a distributed environment, it's also not recommended to enable this feature. If one of the libmc client failed-over but others don't, consistency issue will raise. It's recommended to implement failover in higher level.