Tuesday, December 15, 2009

python-libmemcached replication

While adding some Nagios monitoring checks I was requiring a simple mechanism to replicate data betwen multiple memcached servers. Since libmemcached 0.34 this is possible using the 'replica' behavior.

Using the memcached_set_behavior function and setting the MEMCACHED_BEHAVIOR_NUMBER_OF_REPLICAS to some integer you are ensuring that multiple copies of the stored values exist throughout your memcached farm.
My only issue was that I am using python-libmemcached to access my memcache servers from python and the 'replicas' behavior has not yet (as of 0.17.0) been incorporated. So using I had to patch the sources (RPM package is available on OpenSuse Build Service, patch is attached).

Usage example:
>>> import cmemcached as memcache
>>> mc=memcache.Client(['netmona:11211','netmonb:11211'],debug=1,behaviors={'replicas':2,'binary':1})
>>> mca=memcache.Client(['netmona:11211'],debug=1,behaviors={})
>>> mcb=memcache.Client(['netmonb:11211'],debug=1,behaviors={})
>>> mc.set('replicated-data','yeah',time=600)
>>> mc.get('replicated-data')
>>> mca.get('replicated-data')
>>> mcb.get('replicated-data')

Of course, if you set the data on 1 server only it wont be replicated. Never the less ,this simple replication system opens a number of opportunities for some of my projects where I use memcache as a 'shared' storage between processes.

Warning: memcache does not have any security mechanism embedded (access,confidentiality...), use it only on trusted networks.

No comments: