Writing custom model fields¶

ModelForms and custom fields¶

If you use SubfieldBase, to_python() will be called every time an instance of the field is assigned a value. This means that whenever a value may be assigned to the field, you need to ensure that it will be of the correct datatype, or that you handle any exceptions.

This is especially important if you use ModelForms. When saving a ModelForm, Django will use form values to instantiate model instances. However, if the cleaned form data can’t be used as valid input to the field, the normal form validation process will break.

Therefore, you must ensure that the form field used to represent your custom field performs whatever input validation and data cleaning is necessary to convert user-provided form input into a to_python()-compatible model field value. This may require writing a custom form field, and/or implementing the formfield() method on your field to return a form field class whose to_python() returns the correct datatype.

Custom database types¶

Field.db_type(self, connection)¶

Returns the database column data type for the Field, taking into account the connection object, and the settings associated with it.

Say you’ve created a PostgreSQL custom type called mytype. You can use this field with Django by subclassing Field and implementing the db_type() method, like so:

from django.db import models

class MytypeField(models.Field):
    def db_type(self, connection):
        return 'mytype'

Once you have MytypeField, you can use it in any model, just like any other Field type:

class Person(models.Model):
    name = models.CharField(max_length=80)
    something_else = MytypeField()

If you aim to build a database-agnostic application, you should account for differences in database column types. For example, the date/time column type in PostgreSQL is called timestamp, while the same column in MySQL is called datetime. The simplest way to handle this in a db_type() method is to check the connection.settings_dict['ENGINE'] attribute.

For example:

class MyDateField(models.Field):
    def db_type(self, connection):
        if connection.settings_dict['ENGINE'] == 'django.db.backends.mysql':
            return 'datetime'
        else:
            return 'timestamp'

The db_type() method is only called by Django when the framework constructs the CREATE TABLE statements for your application – that is, when you first create your tables. It’s not called at any other time, so it can afford to execute slightly complex code, such as the connection.settings_dict check in the above example.

Some database column types accept parameters, such as CHAR(25), where the parameter 25 represents the maximum column length. In cases like these, it’s more flexible if the parameter is specified in the model rather than being hard-coded in the db_type() method. For example, it wouldn’t make much sense to have a CharMaxlength25Field, shown here:

# This is a silly example of hard-coded parameters.
class CharMaxlength25Field(models.Field):
    def db_type(self, connection):
        return 'char(25)'

# In the model:
class MyModel(models.Model):
    # ...
    my_field = CharMaxlength25Field()

The better way of doing this would be to make the parameter specifiable at run time – i.e., when the class is instantiated. To do that, just implement django.db.models.Field.__init__(), like so:

# This is a much more flexible example.
class BetterCharField(models.Field):
    def __init__(self, max_length, *args, **kwargs):
        self.max_length = max_length
        super(BetterCharField, self).__init__(*args, **kwargs)

    def db_type(self, connection):
        return 'char(%s)' % self.max_length

# In the model:
class MyModel(models.Model):
    # ...
    my_field = BetterCharField(25)

Finally, if your column requires truly complex SQL setup, return None from db_type(). This will cause Django’s SQL creation code to skip over this field. You are then responsible for creating the column in the right table in some other way, of course, but this gives you a way to tell Django to get out of the way.

Converting database values to Python objects¶

Field.to_python(self, value)¶

Converts a value as returned by your database (or a serializer) to a Python object.

The default implementation simply returns value, for the common case in which the database backend already returns data in the correct format (as a Python string, for example).

If your custom Field class deals with data structures that are more complex than strings, dates, integers or floats, then you’ll need to override this method. As a general rule, the method should deal gracefully with any of the following arguments:

• An instance of the correct type (e.g., Hand in our ongoing example).
• A string (e.g., from a deserializer).
• Whatever the database returns for the column type you’re using.

In our HandField class, we’re storing the data as a VARCHAR field in the database, so we need to be able to process strings and Hand instances in to_python():

import re

class HandField(models.Field):
    # ...

    def to_python(self, value):
        if isinstance(value, Hand):
            return value

        # The string case.
        p1 = re.compile('.{26}')
        p2 = re.compile('..')
        args = [p2.findall(x) for x in p1.findall(value)]
        if len(args) != 4:
            raise ValidationError("Invalid input for a Hand instance")
        return Hand(*args)

Notice that we always return a Hand instance from this method. That’s the Python object type we want to store in the model’s attribute. If anything is going wrong during value conversion, you should raise a ValidationError exception.

Remember: If your custom field needs the to_python() method to be called when it is created, you should be using The SubfieldBase metaclass mentioned earlier. Otherwise to_python() won’t be called automatically.

Converting Python objects to query values¶

Field.get_prep_value(self, value)¶

This is the reverse of to_python() when working with the database backends (as opposed to serialization). The value parameter is the current value of the model’s attribute (a field has no reference to its containing model, so it cannot retrieve the value itself), and the method should return data in a format that has been prepared for use as a parameter in a query.

This conversion should not include any database-specific conversions. If database-specific conversions are required, they should be made in the call to get_db_prep_value().

For example:

class HandField(models.Field):
    # ...

    def get_prep_value(self, value):
        return ''.join([''.join(l) for l in (value.north,
                value.east, value.south, value.west)])

Converting query values to database values¶

Field.get_db_prep_value(self, value, connection, prepared=False)¶

Some data types (for example, dates) need to be in a specific format before they can be used by a database backend. get_db_prep_value() is the method where those conversions should be made. The specific connection that will be used for the query is passed as the connection parameter. This allows you to use backend-specific conversion logic if it is required.

The prepared argument describes whether or not the value has already been passed through get_prep_value() conversions. When prepared is False, the default implementation of get_db_prep_value() will call get_prep_value() to do initial data conversions before performing any database-specific processing.

Field.get_db_prep_save(self, value, connection)¶

Same as the above, but called when the Field value must be saved to the database. As the default implementation just calls get_db_prep_value(), you shouldn’t need to implement this method unless your custom field needs a special conversion when being saved that is not the same as the conversion used for normal query parameters (which is implemented by get_db_prep_value()).

Preprocessing values before saving¶

Field.pre_save(self, model_instance, add)¶

This method is called just prior to get_db_prep_save() and should return the value of the appropriate attribute from model_instance for this field. The attribute name is in self.attname (this is set up by Field). If the model is being saved to the database for the first time, the add parameter will be True, otherwise it will be False.

You only need to override this method if you want to preprocess the value somehow, just before saving. For example, Django’s DateTimeField uses this method to set the attribute correctly in the case of auto_now or auto_now_add.

If you do override this method, you must return the value of the attribute at the end. You should also update the model’s attribute if you make any changes to the value so that code holding references to the model will always see the correct value.

Preparing values for use in database lookups¶

As with value conversions, preparing a value for database lookups is a two phase process.

Field.get_prep_lookup(self, lookup_type, value)¶

get_prep_lookup() performs the first phase of lookup preparation, performing generic data validity checks

Prepares the value for passing to the database when used in a lookup (a WHERE constraint in SQL). The lookup_type will be one of the valid Django filter lookups: exact, iexact, contains, icontains, gt, gte, lt, lte, in, startswith, istartswith, endswith, iendswith, range, year, month, day, isnull, search, regex, and iregex.

Your method must be prepared to handle all of these lookup_type values and should raise either a ValueError if the value is of the wrong sort (a list when you were expecting an object, for example) or a TypeError if your field does not support that type of lookup. For many fields, you can get by with handling the lookup types that need special handling for your field and pass the rest to the get_db_prep_lookup() method of the parent class.

If you needed to implement get_db_prep_save(), you will usually need to implement get_prep_lookup(). If you don’t, get_prep_value will be called by the default implementation, to manage exact, gt, gte, lt, lte, in and range lookups.

You may also want to implement this method to limit the lookup types that could be used with your custom field type.

Note that, for range and in lookups, get_prep_lookup will receive a list of objects (presumably of the right type) and will need to convert them to a list of things of the right type for passing to the database. Most of the time, you can reuse get_prep_value(), or at least factor out some common pieces.

For example, the following code implements get_prep_lookup to limit the accepted lookup types to exact and in:

class HandField(models.Field):
    # ...

    def get_prep_lookup(self, lookup_type, value):
        # We only handle 'exact' and 'in'. All others are errors.
        if lookup_type == 'exact':
            return self.get_prep_value(value)
        elif lookup_type == 'in':
            return [self.get_prep_value(v) for v in value]
        else:
            raise TypeError('Lookup type %r not supported.' % lookup_type)

Field.get_db_prep_lookup(self, lookup_type, value, connection, prepared=False)¶

Performs any database-specific data conversions required by a lookup. As with get_db_prep_value(), the specific connection that will be used for the query is passed as the connection parameter. The prepared argument describes whether the value has already been prepared with get_prep_lookup().

Specifying the form field for a model field¶

Field.formfield(self, form_class=forms.CharField, **kwargs)¶

Returns the default form field to use when this field is displayed in a model. This method is called by the ModelForm helper.

All of the kwargs dictionary is passed directly to the form field’s __init__() method. Normally, all you need to do is set up a good default for the form_class argument and then delegate further handling to the parent class. This might require you to write a custom form field (and even a form widget). See the forms documentation for information about this, and take a look at the code in django.contrib.localflavor for some examples of custom widgets.

Continuing our ongoing example, we can write the formfield() method as:

class HandField(models.Field):
    # ...

    def formfield(self, **kwargs):
        # This is a fairly standard way to set up some defaults
        # while letting the caller override them.
        defaults = {'form_class': MyFormField}
        defaults.update(kwargs)
        return super(HandField, self).formfield(**defaults)

This assumes we’ve imported a MyFormField field class (which has its own default widget). This document doesn’t cover the details of writing custom form fields.

Emulating built-in field types¶

Field.get_internal_type(self)¶

Returns a string giving the name of the Field subclass we are emulating at the database level. This is used to determine the type of database column for simple cases.

If you have created a db_type() method, you don’t need to worry about get_internal_type() – it won’t be used much. Sometimes, though, your database storage is similar in type to some other field, so you can use that other field’s logic to create the right column.

For example:

class HandField(models.Field):
    # ...

    def get_internal_type(self):
        return 'CharField'

No matter which database backend we are using, this will mean that syncdb and other SQL commands create the right column type for storing a string.

If get_internal_type() returns a string that is not known to Django for the database backend you are using – that is, it doesn’t appear in django.db.backends.<db_name>.creation.DATA_TYPES – the string will still be used by the serializer, but the default db_type() method will return None. See the documentation of db_type() for reasons why this might be useful. Putting a descriptive string in as the type of the field for the serializer is a useful idea if you’re ever going to be using the serializer output in some other place, outside of Django.

Converting field data for serialization¶

Field.value_to_string(self, obj)¶

This method is used by the serializers to convert the field into a string for output. Calling Field._get_val_from_obj(obj) is the best way to get the value to serialize. For example, since our HandField uses strings for its data storage anyway, we can reuse some existing conversion code:

class HandField(models.Field):
    # ...

    def value_to_string(self, obj):
        value = self._get_val_from_obj(obj)
        return self.get_prep_value(value)