Django – sharing a memcached instance

Mads Sülau Jørgensen — Tue, 23 Jun 2009 11:23:00 +0000

Update: ~~Some Curious User brought to my attention, that a ticket has been opened which, when implemented, will add a setting for a cache prefix. It will also allow other cache key manipulations.~~ ~~Django has implemented KEY_PREFIX in the development version, which currently means, that it will be out in 1.4 iirc.~~ Django 1.3 has implemented KEY_PREFIX which solves the problem once and for all.

Until recently I’ve been using the file:// django cache, but that has a “problem” when multiple users needs to manipulate the cache (think uid 80 writes a key, that uid 1000 wants to delete).

My problem with the memcached:// django cache provider has been, that it cannot handle being used on a shared memcached instance, because of the danger of key collissions.

If project A and project B would share a memcached instance, they basiclly share the same global namespace. So if they both write a key called actor there is no telling what will happen.

So I wrote a little cache backend for django, that uses the current memcached backend, but adds a pre-defined prefix to all keys.

Usage: Put the code somewhere inside your project in a file called memcachedkeyprefix.py, and set your CACHE_BACKEND to something like: path.to.memcached_key_prefix:///127.0.0.1:11211/?keyprefix=sewc&foo=bar&timeout=3600

"Memcached cache backend with key prefixing"
 
from django.core.cache.backends.base import InvalidCacheBackendError
from django.core.cache.backends.memcached import CacheClass as MemcachedCacheClass
from django.utils.encoding import smart_unicode, smart_str
 
class CacheClass(MemcachedCacheClass):
    def __init__(self, server, params):
        try:
            self._key_prefix = smart_str(params['key_prefix'])
        except KeyError:
            raise InvalidCacheBackendError('key_prefix not specified')
 
        super(CacheClass, self).__init__(server, params)
 
    def _get_key(self, key):
        return self._key_prefix + smart_str(key)
 
    def add(self, key, value, timeout=0):
        return super(CacheClass, self).add(self._get_key(key), value, timeout)
 
    def get(self, key, default=None):
        return super(CacheClass, self).get(self._get_key(key), default)
 
    def set(self, key, value, timeout=0):
        return super(CacheClass, self).set(self._get_key(key), value, timeout)
 
    def delete(self, key):
        return super(CacheClass, self).delete(self._get_key(key))
 
    def get_many(self, keys):
        keys = [self._get_key(key) for key in keys]
        return super(CacheClass, self).get_many(keys)
 
    def incr(self, key, delta=1):
        return super(CacheClass, self).incr(self._get_key(key), delta)
 
    def decr(self, key, delta=1):
        return super(CacheClass, self).decr(self._get_key(key), delta)

Thoughts from Mads Sülau Jørgensen » memcached

Django – sharing a memcached instance