Hypothesis for Django users

Hypothesis offers a number of features specific for Django testing, available in the hypothesis[django] extra. This is tested against each supported series with mainstream or extended support - if you’re still getting security patches, you can test with Hypothesis.

class hypothesis.extra.django.TestCase

Using it is quite straightforward: All you need to do is subclass hypothesis.extra.django.TestCase or hypothesis.extra.django.TransactionTestCase or LiveServerTestCase or StaticLiveServerTestCase and you can use @given as normal, and the transactions will be per example rather than per test function as they would be if you used @given with a normal django test suite (this is important because your test function will be called multiple times and you don’t want them to interfere with each other). Test cases on these classes that do not use @given will be run as normal.

class hypothesis.extra.django.TransactionTestCase

class hypothesis.extra.django.LiveServerTestCase

class hypothesis.extra.django.StaticLiveServerTestCase

We recommend avoiding TransactionTestCase unless you really have to run each test case in a database transaction. Because Hypothesis runs this in a loop, the performance problems it normally has are significantly exacerbated and your tests will be really slow. If you are using TransactionTestCase, you may need to use @settings(suppress_health_check=[HealthCheck.too_slow]) to avoid errors due to slow example generation.

Having set up a test class, you can now pass @given a strategy for Django models:

hypothesis.extra.django.from_model(model, /, **field_strategies)

Return a strategy for examples of model.

Warning

Hypothesis creates saved models. This will run inside your testing transaction when using the test runner, but if you use the dev console this will leave debris in your database.

model must be an subclass of Model. Strategies for fields may be passed as keyword arguments, for example is_staff=st.just(False). In order to support models with fields named “model”, this is a positional-only parameter.

Hypothesis can often infer a strategy based the field type and validators, and will attempt to do so for any required fields. No strategy will be inferred for an AutoField, nullable field, foreign key, or field for which a keyword argument is passed to from_model(). For example, a Shop type with a foreign key to Company could be generated with:

shop_strategy = from_model(Shop, company=from_model(Company))

Like for builds(), you can pass ... (python:Ellipsis) as a keyword argument to infer a strategy for a field which has a default value instead of using the default.

For example, using the trivial django project we have for testing:

>>> from hypothesis.extra.django import from_model
>>> from toystore.models import Customer
>>> c = from_model(Customer).example()
>>> c
<Customer: Customer object>
>>> c.email
'jaime.urbina@gmail.com'
>>> c.name
'\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
>>> c.age
-873375803

Hypothesis has just created this with whatever the relevant type of data is.

Obviously the customer’s age is implausible, which is only possible because we have not used (eg) MinValueValidator to set the valid range for this field (or used a PositiveSmallIntegerField, which would only need a maximum value validator).

If you do have validators attached, Hypothesis will only generate examples that pass validation. Sometimes that will mean that we fail a HealthCheck because of the filtering, so let’s explicitly pass a strategy to skip validation at the strategy level:

>>> from hypothesis.strategies import integers
>>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
>>> c
<Customer: Customer object>
>>> c.age
5

hypothesis.extra.django.from_form(form, form_kwargs=None, **field_strategies)

Return a strategy for examples of form.

form must be an subclass of Form. Strategies for fields may be passed as keyword arguments, for example is_staff=st.just(False).

Hypothesis can often infer a strategy based the field type and validators, and will attempt to do so for any required fields. No strategy will be inferred for a disabled field or field for which a keyword argument is passed to from_form().

This function uses the fields of an unbound form instance to determine field strategies, any keyword arguments needed to instantiate the unbound form instance can be passed into from_form() as a dict with the keyword form_kwargs. E.g.:

shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})

Like for builds(), you can pass ... (python:Ellipsis) as a keyword argument to infer a strategy for a field which has a default value instead of using the default.

Tips and tricks

Custom field types

If you have a custom Django field type you can register it with Hypothesis’s model deriving functionality by registering a default strategy for it:

>>> from toystore.models import CustomishField, Customish
>>> from_model(Customish).example()
hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
    customish for model Customish
>>> from hypothesis.extra.django import register_field_strategy
>>> from hypothesis.strategies import just
>>> register_field_strategy(CustomishField, just("hi"))
>>> x = from_model(Customish).example()
>>> x.customish
'hi'

Note that this mapping is on exact type. Subtypes will not inherit it.

hypothesis.extra.django.register_field_strategy(field_type, strategy)

Add an entry to the global field-to-strategy lookup used by from_field().

field_type must be a subtype of django.db.models.Field or django.forms.Field, which must not already be registered. strategy must be a SearchStrategy.

hypothesis.extra.django.from_field(field)

Return a strategy for values that fit the given field.

This function is used by from_form() and from_model() for any fields that require a value, or for which you passed ... (python:Ellipsis) to infer a strategy from an annotation.

It’s pretty similar to the core from_type() function, with a subtle but important difference: from_field takes a Field instance, rather than a Field subtype, so that it has access to instance attributes such as string length and validators.

Generating child models

For the moment there’s no explicit support in hypothesis-django for generating dependent models. i.e. a Company model will generate no Shops. However if you want to generate some dependent models as well, you can emulate this by using the flatmap function as follows:

from hypothesis.strategies import just, lists


def generate_with_shops(company):
    return lists(from_model(Shop, company=just(company))).map(lambda _: company)


company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)

Let’s unpack what this is doing:

The way flatmap works is that we draw a value from the original strategy, then apply a function to it which gives us a new strategy. We then draw a value from that strategy. So in this case we’re first drawing a company, and then we’re drawing a list of shops belonging to that company: The just strategy is a strategy such that drawing it always produces the individual value, so from_model(Shop, company=just(company)) is a strategy that generates a Shop belonging to the original company.

So the following code would give us a list of shops all belonging to the same company:

from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))

The only difference from this and the above is that we want the company, not the shops. This is where the inner map comes in. We build the list of shops and then throw it away, instead returning the company we started for. This works because the models that Hypothesis generates are saved in the database, so we’re essentially running the inner strategy purely for the side effect of creating those children in the database.

Generating primary key values

If your model includes a custom primary key that you want to generate using a strategy (rather than a default auto-increment primary key) then Hypothesis has to deal with the possibility of a duplicate primary key.

If a model strategy generates a value for the primary key field, Hypothesis will create the model instance with update_or_create(), overwriting any existing instance in the database for this test case with the same primary key.

On the subject of MultiValueField

Django forms feature the MultiValueField which allows for several fields to be combined under a single named field, the default example of this is the SplitDateTimeField.

class CustomerForm(forms.Form):
    name = forms.CharField()
    birth_date_time = forms.SplitDateTimeField()

from_form supports MultiValueField subclasses directly, however if you want to define your own strategy be forewarned that Django binds data for a MultiValueField in a peculiar way. Specifically each sub-field is expected to have its own entry in data addressed by the field name (e.g. birth_date_time) and the index of the sub-field within the MultiValueField, so form data for the example above might look like this:

{
    "name": "Samuel John",
    "birth_date_time_0": "2018-05-19",  # the date, as the first sub-field
    "birth_date_time_1": "15:18:00",  # the time, as the second sub-field
}

Thus, if you want to define your own strategies for such a field you must address your sub-fields appropriately:

from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))