django-memoize

About

django-memoize is an implementation of memoization technique for Django. You can think of it as a cache for function or method results.

Installation

Install the extension with one of the following commands:

$ easy_install django-memoize

or alternatively if you have pip installed:

$ pip install django-memoize

Set Up

Add ‘memoize’ to your INSTALLED_APPS in settings.py:

INSTALLED_APPS = [
    '...',
    'memoize',
]

Memoization is managed through a Memoizer instance:

from memoize import Memoizer

memoizer = Memoizer()

However, we recommend to use already defined instance of Memoizer and use its methods:

from memoize import memoize, delete_memoized, delete_memoized_verhash

@memoize(timeout=60)
def count_objects():
    pass

delete_memoized(count_objects)

Memoization

See memoize()

In memoization, the functions arguments are also included into the cache_key.

Memoize is also designed for methods, since it will take into account the repr of the ‘self’ or ‘cls’ argument as part of the cache key.

The theory behind memoization is that if you have a function you need to call several times in one request, it would only be calculated the first time that function is called with those arguments. For example, a model object that determines if a user has a role. You might need to call this function many times during a single request. To keep from hitting the database every time this information is needed you might do something like the following:

class Person(models.Model):
    @memoize(timeout=50)
    def has_membership(self, role_id):
            return Group.objects.filter(user=self, role_id=role_id).count() >= 1

Warning

Using mutable objects (classes, etc) as part of the cache key can become tricky. It is suggested to not pass in an object instance into a memoized function. However, the memoize does perform a repr() on the passed in arguments so that if the object has a __repr__ function that returns a uniquely identifying string for that object, that will be used as part of the cache key.

For example, a model person object that returns the database id as part of the unique identifier.:

class Person(models.Model):
    def __repr__(self):
        return "%s(%s)" % (self.__class__.__name__, self.id)

Deleting memoize cache

You might need to delete the cache on a per-function bases. Using the above example, lets say you change the users permissions and assign them to a role, but now you need to re-calculate if they have certain memberships or not. You can do this with the delete_memoized() function.:

delete_memoized('user_has_membership')

Note

If only the function name is given as parameter, all the memoized versions of it will be invalidated. However, you can delete specific cache by providing the same parameter values as when caching. In following example only the user-role cache is deleted:

user_has_membership('demo', 'admin')
user_has_membership('demo', 'user')

delete_memoized('user_has_membership', 'demo', 'user')

API

class memoize.Memoizer(cache=<django.utils.connection.ConnectionProxy object>, cache_prefix='memoize', default_cache_value=<memoize.DefaultCacheObject object>)

This class is used to control the memoizer objects.

memoize(timeout=<object object>, make_name=None, unless=None)

Use this to cache the result of a function, taking its arguments into account in the cache key.

Information on Memoization.

Example:

@memoize(timeout=50)
def big_foo(a, b):
    return a + b + random.randrange(0, 1000)

>>> big_foo(5, 2)
753
>>> big_foo(5, 3)
234
>>> big_foo(5, 2)
753

Note

The returned decorated function now has three function attributes assigned to it.

uncached

The original undecorated function. readable only

cache_timeout

The cache timeout value for this function. For a custom value to take affect, this must be set before the function is called.

readable and writable

make_cache_key

A function used in generating the cache_key used.

readable and writable

Parameters

• timeout – Default: 300. If set to an integer, will cache for that amount of time. Unit of time is in seconds.

• make_name – Default None. If set this is a function that accepts a single argument, the function name, and returns a new string to be used as the function name. If not set then the function name is used.

• unless – Default None. Cache will always execute the caching facilities unelss this callable is true. This will bypass the caching entirely.

delete_memoized(f, *args, **kwargs)

Deletes the specified functions caches, based by given parameters. If parameters are given, only the functions that were memoized with them will be erased. Otherwise all versions of the caches will be forgotten.

Example:

@memoize(50)
def random_func():
    return random.randrange(1, 50)

@memoize()
def param_func(a, b):
    return a+b+random.randrange(1, 50)

>>> random_func()
43
>>> random_func()
43
>>> delete_memoized('random_func')
>>> random_func()
16
>>> param_func(1, 2)
32
>>> param_func(1, 2)
32
>>> param_func(2, 2)
47
>>> delete_memoized('param_func', 1, 2)
>>> param_func(1, 2)
13
>>> param_func(2, 2)
47

Delete memoized is also smart about instance methods vs class methods.

When passing a instancemethod, it will only clear the cache related to that instance of that object. (object uniqueness can be overridden by defining the __repr__ method, such as user id).

When passing a classmethod, it will clear all caches related across all instances of that class.

Example:

class Adder(object):
    @memoize()
    def add(self, b):
        return b + random.random()

>>> adder1 = Adder()
>>> adder2 = Adder()
>>> adder1.add(3)
3.23214234
>>> adder2.add(3)
3.60898509
>>> delete_memoized(adder.add)
>>> adder1.add(3)
3.01348673
>>> adder2.add(3)
3.60898509
>>> delete_memoized(Adder.add)
>>> adder1.add(3)
3.53235667
>>> adder2.add(3)
3.72341788

Parameters

• fname – Name of the memoized function, or a reference to the function.

• *args – A list of positional parameters used with memoized function.

• **kwargs – A dict of named parameters used with memoized function.

Note

django-memoize uses inspect to order kwargs into positional args when the function is memoized. If you pass a function reference into fname instead of the function name, django-memoize will be able to place the args/kwargs in the proper order, and delete the positional cache.

However, if delete_memoized is just called with the name of the function, be sure to pass in potential arguments in the same order as defined in your function as args only, otherwise django-memoize will not be able to compute the same cache key.

Note

django-memoize maintains an internal random version hash for the function. Using delete_memoized will only swap out the version hash, causing the memoize function to recompute results and put them into another key.

This leaves any computed caches for this memoized function within the caching backend.

It is recommended to use a very high timeout with memoize if using this function, so that when the version has is swapped, the old cached results would eventually be reclaimed by the caching backend.

delete_memoized_verhash(f, *args)

Delete the version hash associated with the function.

Warning

Performing this operation could leave keys behind that have been created with this version hash. It is up to the application to make sure that all keys that may have been created with this version hash at least have timeouts so they will not sit orphaned in the cache backend.

Changelog

Version 2.2.0

• Enable wheels

• Py3 support improvements

Version 2.1.1

• Deprecates unsupported Django versions

Version 2.1.0

• memoize now caches also None values

• Py3 fixes

Version 2.0.0

• Deprecates Django<1.7

• Improves unicode stability

Version 1.3.0

• Fixes a bug #8 (*args are ignored by memoize)

Version 1.2.0

• Fixed a serious issue where memoize function ignored special f(arg, **kwargs) input use-case. See test test_10ab_arg_kwarg_memoize for more details.

• Added new Django==1.8 Travis env test

Version 1.1.2

• Memoizer now accepts cache_prefix and cache parameter

• Support for Django >= 1.4

Version 1.1.1

• Default timeout set to Django’s cache timeout

Version 1.1.0

• Port to Python >= 3.3 (requiring Python 2.6/2.7 for 2.x).

• Fixed bug with using per-memoize timeouts greater than the default timeout

• Added better support for per-instance memoization.

• Various bug fixes

Version 1.0.0

• Initial public release