Django URL Filter

Contents

Usage

Vanilla

In its simplest form, Django URL Filter usage revolves around FilterSet. They can be used manually:

from django import forms
from url_filter.filters import Filter
from url_filter.filtersets import FilterSet

class ProfileFilterSet(FilterSet):
    lives_in_country = Filter(form_field=forms.CharField())

class UserFilterSet(FilterSet):
    username = Filter(form_field=forms.CharField(), lookups=['exact'])
    email = Filter(form_field=forms.CharField())
    joined = Filter(form_field=forms.DateField())
    profile = ProfileFilterSet()

query = QueryDict(
  'email__contains=gmail'
  '&joined__year=2015'
  '&profile__lives_in_country__iexact=us'
)
fs = UserFilterSet(data=query, queryset=User.objects.all())
filtered_users = fs.filter()

Notable things to mention from above:

  • FilterSet can be used as a Filter within another FilterSet hence allowing filtering by related models.
  • form_field is used to validate the filter value. Each lookup however can overwrite validation. For example year lookup will use IntegerField rather then DateField.
  • Filter can restrict allowed lookups for that field by using lookups parameter

Django

Instead of manually creating FilterSet, Django URL Filter comes with ModelFilterSet which greatly simplifies the task:

from django import forms
from url_filter.filtersets import ModelFilterSet

class UserFilterSet(ModelFilterSet):
    class Meta(object):
        model = User
        fields = ['username', 'email', 'joined', 'profile']

Notable things:

  • fields can actually be completely omitted. In that case FilterSet will use all fields available in the model, including related fields.

  • filters can be manually overwritten when custom behavior is required:

    class UserFilterSet(ModelFilterSet):
        username = Filter(form_field=forms.CharField(max_length=15))
    
        class Meta(object):
            model = User
            fields = ['username', 'email', 'joined', 'profile']
    

SQLAlchemy

SQLAlchemy works very similar to how Django backend works. Additionally SQLAlchemyModelFilterSet is available to be able to easily create filter sets from SQLAlchemy models. For example:

from django import forms
from url_filter.backend.sqlalchemy import SQLAlchemyFilterBackend
from url_filter.filtersets.sqlalchemy import SQLAlchemyModelFilterSet

class UserFilterSet(SQLAlchemyModelFilterSet):
    filter_backend_class = SQLAlchemyFilterBackend

    class Meta(object):
        model = User  # this model should be SQLAlchemy model
        fields = ['username', 'email', 'joined', 'profile']

fs = UserFilterSet(data=QueryDict(), queryset=session.query(User))
fs.filter()

Notable things:

  • this works exactly same as ModelFilterSet so refer above for some of general options.
  • filter_backend_class must be provided since otherwise DjangoFilterBackend will be used which will obviously not work with SQLAlchemy models.
  • queryset given to the queryset should be SQLAlchemy query object.

Plain Filtering

In addition to supporting regular ORMs django-url-filter also allows to filter plain Python lists of either objects or dictionaries. This feature is primarily meant to filter data-sources without direct filtering support such as lists of data in redis. For example:

from django import forms
from url_filter.backend.plain import PlainFilterBackend
from url_filter.filtersets.plain import PlainModelFilterSet

class UserFilterSet(PlainModelFilterSet):
    filter_backend_class = PlainFilterBackend

    class Meta(object):
        # the filterset will generate fields from the
        # primitive Python data-types
        model = {
            'username': 'foo',
            'password': bar,
            'joined': date(2015, 1, 2),
            'profile': {
                'preferred_name': 'rainbow',
            }
        }

fs = UserFilterSet(data=QueryDict(), queryset=[{...}, {...}, ...])
fs.filter()

Integrations

Django URL Filters tries to be usage-agnostic and does not assume how FilterSet is being used in the application. It does however ship with some common integrations to simplify common workflows.

Django Class Based Views

FilterSet or related classes can directly be used within Django class-based-views:

class MyFilterSet(ModelFilterSet):
    class Meta(object):
        model = MyModel

class MyListView(ListView):
    queryset = MyModel.objects.all()
    def get_queryset(self):
        qs = super(MyListView, self).get_queryset()
        return MyFilterSet(data=self.request.GET, queryset=qs).filter()
Django REST Framework

Django URL Filter can rather easily be integrated with DRF. For that, a DRF-specific filter backend DjangoFilterBackend is implemented and can be used in settings:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': [
        'url_filter.integrations.drf.DjangoFilterBackend',
    ]
}

or manually set in the viewset:

class MyViewSet(ModelViewSet):
    queryset = MyModel.objects.all()
    serializer_class = MyModelSerializer
    filter_backends = [DjangoFilterBackend]
    filter_fields = ['field1', 'field2']

Note in the example above, fields to be filtered on are explicitly specified in the filter_fields attribute. Alternatively if more control over FilterSet is required, it can be set explicitly:

class MyFilterSet(FilterSet):
    pass

class MyViewSet(ModelViewSet):
    queryset = MyModel.objects.all()
    serializer_class = MyModelSerializer
    filter_backends = [DjangoFilterBackend]
    filter_class = MyFilterSet

For more available options, please refer to DjangoFilterBackend documentation.

Big Picture

This document explains all of the concepts used in Django URL Filter in context hence providing a “big picture” of how it works.

Basics

In order to filter any data, this library breaks the process into 3 phases:

  1. Parse the URL querystring into LookupConfig
  2. Loop through all the configs and generate FilterSpec when possible
  3. Use the list of specs to actually filter data

And here is a bit more information about each phase.

Parsing

Fundamentally a querystring is a collection of key-pairs. As such, this data is naturally flat and is usually represented as a simple dictionary:

?foo=bar&happy=rainbows => {
    'foo': 'bar',
    'happy': 'rainbows',
}

Note

Technically, this is not 100% true since key can be repeated which is why Django uses QueryDict but for the purposes of this discussion, let’s assume no duplicate keys are present.

The filtering however is not flat. Each querystring key can be nested when using nested FilterSet and in addition it can optionally contain lookup. For example:

?foo=bar
?foo__contains=bar
?foo__nested=bar
?foo__nested__contains=bar
?foo__nested__othernested=bar
?foo__nested__othernested__contains=bar

In order to accomodate the nested structure of querystring keys, Django URL Filter parses all querystring key-value pairs into nested dictionaries. For example:

?foo__nested__othernested=bar => {
    'foo': {
        'nested': {
            'othernested': 'bar'
        }
    }
}
?foo__nested__othernested__contains=bar => {
    'foo': {
        'nested': {
            'othernested': {
                'contains': 'bar'
            }
        }
    }
}

That is essentially what LookupConfig stores. Since these dictionaries are flat (each dictionaty has at most one key), it also provides some utility properties for dealing with such data. You can refer to the LookupConfig API documentation for more information.

Filter Specification

As mentioned in README, Django URL Filter decouples parsing of querystring and filtering. It achieves that by constructing filter specifications which have all necessary information to filter data without actually filtering data. Thats what FilterSpec is. It stores 3 required pieces of information on how to filter data:

  • Which attribute to filter on. Since models can be related by attributes of related models, this actually ends up being a list of attributes which we call components.
  • Lookup to use to filter data. This specifies how the value should be compared while doing filtering. Example is exact, contains. By default only lookups from Django ORM are supported however custom CallableFilter can be used to define custom lookups.
  • If the filter is negated. For example, to filter when username is 'foo' or filter when username is not 'foo'.
Filtering

Since filtering is decoupled from the FilterSet, the filtering honors all go to a specified filter backend. The backend is very simple. It takes a list of filter specifications and a data to filter and its job is to filter that data as specified in the specifications.

Note

Currently we only support a handful of backends such as Django ORM, SQLAlchemy and plain Python interables filter backends but you can imagine that any backend can be implemented. Eventually filter backends can be added for more exotic sources like Mongo, Redis, etc.

Steps

Above information hopefully puts things in perspective and here is more detailed step-by-step guide what Django URL Filter does behind the scenes:

  1. FilterSet is instantiated with querystring data as well as queryset to filter.
  2. FilterSet is asked to filter given data via filter method which kicks in all the steps below.
  3. FilterSet finds all filters it is capable of Filtering via get_filters. This is where custom filtersets can hook into to do custom stuff like extracting filters from a Django model.
  4. FilterSet binds all child filters to itself via bind. This practically sets parent and name attributes.
  5. Root FilterSet loops through all querystring pairs and generates LookupConfig for all of them.
  6. Root FilterSet loops through all generated configs and attemps to find appropriate filter to use to generate a spec fo the given config. The matching happens by the first key in the LookupConfig dict. If that key is found in available filters, that filter is used and otherwise that config is skipped. This is among the reasons why LookupConfig is used since it allows this step to be very simple.
  7. If appropriate filter is found, it is passed nested config to the child filter which then goes through very similar process as in previous step until it gets to a leaf filter.
  8. Leaf Filter gets the config. In then checks if the config is still nested. For example if the config is simply a value (e.g. 'bar') or is still a dictionary (e.g. {'contains': 'bar'}). If the config is just a value, it then uses a default lookup for that filter as provided in default_lookup parameter when instantiating Filter. If the config is a dictionary, it makes sure that it is a valid lookup config (e.g. its not {'rainbows': {'contains': 'bar'}} since it would not know what to do with rainbows since it is not a valid lookup value).
  9. Now that Filter validated the lookup itself, it cleans the actual filter value by using either form_field as passed as parameter when instantiating Filter or by using lookup overwrite. Overwrites are necessary for more exotic lookups like in or year since they need to validate data in a different way.
  10. If the value is valid, then the leaf filter constructs a FilterSpec since it has all the necessary information to do that - 1) all filter component names from all ancestors (e.g. all attributes which should be accessed on the queryset to get to the value to be filtered on); 2) the actual filter value and 3) if the filter is negated.
  11. At this point, root FilterSet will get the FilterSpec as bubbled up from the leaf filter. If any ValidationError exceptions are raised, then depending on strict_mode, it will either ignore errors or will propagate them up to the caller of the filterset.
  12. Once all specs are collected from all the querystring key-value-pairs, root FilterSet instantiates a filter backend and passes it all the specs.
  13. Finally root FilterSet uses the filter backend to filter given queryset and returns the results to the user.

Some important things to note:

  • Root FilterSet does all the looping over querystring data and generated configurations.
  • Children filters of a root FilterSet are only responsible for generating FilterSpec and in the process of validating the data.

History

0.3.11 (2018-12-06)

  • Not modifying queryset in Django backend if no filters were applied. See #73.

0.3.10 (2018-11-14)

  • Only running distinct on queryset when one of filters is on one-to-many relation. This should help with performance. See #26.

0.3.9 (2018-11-12)

  • Adding iin form field overwrite for SQLAlchemy as otherwise by default iin lookup is not validated correctly.

0.3.8 (2018-08-08)

  • Fixed SQLAlchemyFilterBackend by not joining nested models when they are already eager loaded via query.options().

0.3.7 (2018-07-27)

  • Added StrictModel.empty which is new default. It returns empty queryset when any filter validations fail.
  • Fixed in lookup. Previously if any of the items were invalid whole filter would fail and depending on strict mode would either return all results, no results or will raise exception. Now in StrictMode.empty and StrictMode.drop any invalid items are ignored which will filter results for valid items. See #63.
  • Added ability in ModelFilterSet to customize filter names by providing extra_kwargs with field source. See #66.

0.3.6 (2018-07-23)

  • Added support for extra_kwargs in ModelFilterSet.Meta.
  • Added CoreAPIURLFilterBackend which enables documented filters in swagger docs.
  • Added iin lookup in plain and sqlalchemy backends.
  • Fixing inconsistency between plain and Django week_day lookup. Now both are consistent with 1-Monday and 7-Sunday.

0.3.5 (2018-02-27)

  • Django 2 support.
  • Using tox-travis for travis builds.
  • Fixed negated queries in Django backend. Previously negation did NOT (condition1 and condition2) vs expected NOT condition1 and NOT condition2. See #53.

0.3.4 (2017-08-17)

  • Py36 compatibility by switching to enum-compat from enum34
  • Improvement to README by including imports in code examples
  • Defaulting SQLAlchemyModelFilterSet to use SQLAlchemyFilterBackend
  • Defaulting PlainModelFilterSet to use PlainFilterBackend
  • Using universal wheels for distribution

0.3.3 (2017-06-15)

  • Fixed bug which did not allow to use SQLAlchemy backend fully without having django.contrib.contenttypes in installed apps. See #36.
  • Improved SQLAlchemy versions compatibility.
  • Added URLFilterBackend alias in DRF integration for backend to reduce confusion with DjangoFilterBackend as in url filter core backend.

0.3.2 (2017-05-19)

  • Fixed plain backend to return list in Python 3 vs filter() generator which is not compatible with Django pagination since it requires len() to be implemented.

0.3.1 (2017-05-18)

  • Fixed bug where default filters were used in root filtersets. As a result additional querystring parameters were validation which broke other functionality such as pagination.

0.3.0 (2017-01-26)

  • Added plain objects filtering support. More in docs and GitHub issue #8.
  • Added CallableFilter which allows to implement custom filters.
  • Normalizing to DRF’s ValidationError when using StrictMode.Fail since filterset raises Django’s ValidationError which caused 500 status code.
  • Fixes ModelFilterSet automatic introspection to ignore GenericForeignKey since they dont have form fields associated with them. See #20.
  • Releasing with wheels.

0.2.0 (2015-09-12)

  • Added SQLAlchemy support.

  • FilterSet instances have much more useful __repr__ which shows all filters at a glance. For example:

    >>> PlaceFilterSet()
    PlaceFilterSet()
      address = Filter(form_field=CharField, lookups=ALL, default_lookup="exact", is_default=False)
      id = Filter(form_field=IntegerField, lookups=ALL, default_lookup="exact", is_default=True)
      name = Filter(form_field=CharField, lookups=ALL, default_lookup="exact", is_default=False)
      restaurant = RestaurantFilterSet()
        serves_hot_dogs = Filter(form_field=BooleanField, lookups=ALL, default_lookup="exact", is_default=False)
        serves_pizza = Filter(form_field=BooleanField, lookups=ALL, default_lookup="exact", is_default=False)
        waiter = WaiterFilterSet()
          id = Filter(form_field=IntegerField, lookups=ALL, default_lookup="exact", is_default=True)
          name = Filter(form_field=CharField, lookups=ALL, default_lookup="exact", is_default=False)
    

0.1.1 (2015-09-06)

  • Fixed installation issue where not all subpackages were installed.

0.1.0 (2015-08-30)

  • First release on PyPI.

url_filter

url_filter package

Subpackages
url_filter.backends package
Submodules
url_filter.backends.base module
class url_filter.backends.base.BaseFilterBackend(queryset, context=None)[source]

Bases: object

Base filter backend from which all other backends must subclass.

Parameters:
  • queryset – Iterable which this filter backend will eventually filter. The type of the iterable depends on the filter backend. For example for DjangoFilterBackend, Django’s QuerySet needs to be passed.
  • context (dict) – Context dictionary. It could contain any information which potentially could be useful to filter given queryset. That can include, request, view, view kwargs, etc. The idea is similar to DRF serializers. By passing the context, it allows custom filters to reference all the information they need to be able to effectively filter data.
bind(specs)[source]

Bind the given specs to the filter backend.

This allows the filter backend to be instantiated first before filter specs are constructed and later, specs can be binded to the backend.

Parameters:specs (list) – List of FilterSpec to be binded for the filter backend for filtering
callable_specs

Property for getting custom filter specifications which have a filter callable for filtering querysets. These specifications cannot be directly used by filter backend and have to be called manually to filter data.

See also

regular_specs

empty()[source]

Method for returning empty queryset when any validations failed.

enforce_same_models = True

Whether same models should be enforced when trying to use this filter backend.

More can be found in BaseFilterBackend.model()

filter()[source]

Main public method for filtering querysets.

filter_by_callables(queryset)[source]

Method for filtering queryset by using custom filter callables as given in the Filter definition.

This is really meant to accommodate filtering with simple filter keys having complex filtering logic behind them. More about custom callables can be found at CallableFilter

filter_by_specs(queryset)[source]

Method for filtering queryset by using standard filter specs.

Note

MUST be implemented by subclasses

get_model()[source]

Get the queryset model.

Note

MUST be implemented by subclasses. model() property uses this method to get the model.

See also

model()

model

Property for getting model on which this filter backend operates.

This is meant to be used by the integrations directly shipped with django-url-filter which need to be able to validate that the filterset will be able to filter given queryset. They can do that by comparing the model they are trying to filter matches the model the filterbackend got. This primarily will have misconfigurations such as using SQLAlchemy filterset to filter Django’s QuerySet.

name = None

Name of the filter backend.

This is used by custom callable filters to define callables for each supported backend. More at CallableFilter

regular_specs

Property for getting standard filter specifications which can be used directly by the filter backend to filter queryset.

See also

callable_specs

supported_lookups = set([])

Set of supported lookups this filter backend supports.

This is used by leaf Filter to determine whether it should construct FilterSpec for a particular key-value pair from querystring since it if constructs specification but then filter backend will not be able to filter it, things will blow up. By explicitly checking if filter backend supports particular lookup it can short-circuit the logic and avoid errors down the road. This is pretty much the only coupling between filters and filter backends.

url_filter.backends.django module
class url_filter.backends.django.DjangoFilterBackend(queryset, context=None)[source]

Bases: url_filter.backends.base.BaseFilterBackend

Filter backend for filtering Django querysets.

Warning

The filter backend can ONLY filter Django’s QuerySet. Passing any other datatype for filtering will kill happy bunnies under rainbow.

empty()[source]

Get empty queryset

excludes

Property which gets list of negated filters

By combining all negated filters we can optimize filtering by calling QuerySet.exclude once rather then calling it for each filter specification.

filter_by_specs(queryset)[source]

Filter queryset by applying all filter specifications

The filtering is done by calling QuerySet.filter and QuerySet.exclude as appropriate.

get_model()[source]

Get the model from the given queryset

includes

Property which gets list of non-negated filters

By combining all non-negated filters we can optimize filtering by calling QuerySet.filter once rather then calling it for each filter specification.

name = u'django'
supported_lookups = set([u'gt', u'year', u'month', u'isnull', u'second', u'week_day', u'in', u'regex', u'gte', u'contains', u'lt', u'startswith', u'iendswith', u'icontains', u'iexact', u'exact', u'day', u'minute', u'hour', u'iregex', u'endswith', u'range', u'istartswith', u'lte'])
url_filter.backends.plain module
class url_filter.backends.plain.PlainFilterBackend(queryset, context=None)[source]

Bases: url_filter.backends.base.BaseFilterBackend

Filter backend for filtering plain Python iterables.

Warning

The filter backend does filtering inside a regular loop by comparing attributes of individual objects within iterable. As a result, this is NOT efficient method for filtering any large amounts of data. In those cases, it would probably be better to find more appropriate and efficient way to filter data.

empty()[source]

Get empty queryset

enforce_same_models = False
filter_by_specs(queryset)[source]

Filter queryset by applying all filter specifications

The filtering is done by calling manually loping over all items in the iterable and comparing inner attributes with the filter specification.

get_model()[source]

Get the model from the given queryset

Since there is no specific model for filtering Python lists, this simply returns object

name = u'plain'
supported_lookups = set([u'gt', u'year', u'month', u'isnull', u'second', u'week_day', u'in', u'regex', u'gte', u'contains', u'lt', u'startswith', u'iendswith', u'icontains', u'iexact', u'iin', u'exact', u'day', u'minute', u'hour', u'iregex', u'endswith', u'range', u'istartswith', u'lte'])
url_filter.backends.sqlalchemy module
class url_filter.backends.sqlalchemy.SQLAlchemyFilterBackend(*args, **kwargs)[source]

Bases: url_filter.backends.base.BaseFilterBackend

Filter backend for filtering SQLAlchemy query objects.

Warning

The filter backend can ONLY filter SQLAlchemy’s query objects. Passing any other datatype for filtering will kill happy bunnies under rainbow.

Warning

The filter backend can ONLY filter query objects which query a single entity (e.g. query a single model or model column). If query object queries multiple entities, AssertionError will be raised.

build_clause(spec)[source]

Construct SQLAlchemy binary expression filter clause from the given filter specification.

Parameters:spec (FilterSpec) – Filter specification for which to construct filter clause
Returns:Tuple of filter binary expression clause and and a list of model attributes/descriptors which should be joined when doing filtering. If these attributes are not joined, SQLAlchemy will not join appropriate tables hence wont be able to correctly filter data.
Return type:tuple
empty()[source]

Get empty queryset

filter_by_specs(queryset)[source]

Filter SQLAlchemy query object by applying all filter specifications

The filtering is done by calling filter with all appropriate filter clauses. Additionally if any filter specifications filter by related models, those models are joined as necessary.

get_model()[source]

Get the model from the given queryset

name = u'sqlalchemy'
supported_lookups = set([u'startswith', u'gt', u'gte', u'contains', u'lt', u'iendswith', u'icontains', u'iexact', u'isnull', u'range', u'istartswith', u'lte', u'in', u'endswith', u'iin', u'exact'])
url_filter.filtersets package
Submodules
url_filter.filtersets.base module
class url_filter.filtersets.base.FilterSet(data=None, queryset=None, context=None, strict_mode=None, *args, **kwargs)[source]

Bases: url_filter.filters.BaseFilter

Main user-facing classes to use filtersets.

It primarily does:

  • takes queryset to filter
  • takes querystring data which will be used to filter given queryset
  • from the querystring, it constructs a list of LookupConfig
  • loops over the created configs and attempts to get FilterSpec for each
  • in the process, it delegates the job of constructing spec to child filters when any match is found between filter defined on the filter and lookup name in the config
Parameters:
  • data (QueryDict, optional) – QueryDict of querystring data. Only optional when FilterSet is used as a nested filter within another FilterSet.
  • queryset (iterable, optional) – Can be any iterable as supported by the filter backend. Only optional when FilterSet is used as a nested filter within another FilterSet.
  • context (dict, optional) – Context for filtering. This is passed to filtering backend. Usually this would consist of passing request and view object from the Django view.
  • strict_mode (str, optional) – Strict mode how FilterSet should behave when any validation fails. See url_filter.constants.StrictMode doc for more information. Default is empty.
default_filter

Cached property for looking up default filter. Default filter is a filter which is defined with is_default=True. Useful when lookup config references nested filter without specifying which field to filter. In that case default filter will be used.

default_strict_mode = u'empty'

Default strict mode which should be used when one is not provided in initialization.

filter()[source]

Main method which should be used on root FilterSet to filter queryset.

This method:

  • asserts that filtering is being done on root FilterSet and that all necessary data is provided
  • creates LookupConfig from the provided data (querystring)
  • loops over all configs and attempts to get FilterSet for all of them
  • instantiates filter backend
  • uses the created filter specs to filter queryset by using specs
Returns:Filtered queryset
Return type:querystring
filter_backend

Property for getting instantiated filter backend.

Primarily useful when accessing filter_backend outside of the filterset such as leaf filters or integration layers since backend has useful information for both of those examples.

filter_backend_class

alias of url_filter.backends.django.DjangoFilterBackend

filter_options_class

Class to be used to construct Meta during FilterSet class creation time in its metaclass.

alias of FilterSetOptions

filters

Cached property for accessing filters available in this filteset. In addition to getting filters via get_filters(), this property binds all filters to the filterset by using BaseFilter.bind().

get_filter_backend()[source]

Get instantiated filter backend class.

This backend is then used to actually filter queryset.

get_filters()[source]

Get all filters defined in this filterset.

By default only declared filters are returned however this method is meant to be used as a hook in subclasses in order to enhance functionality such as automatically adding filters from model fields.

get_spec(config)[source]

Get FilterSpec for the given LookupConfig.

If the config is non leaf config (it has more nested fields), then the appropriate matching child filter is used to get the spec. If the config however is a leaf config, then default_filter is used to get the spec, when available, and if not, this filter is skipped.

Parameters:config (LookupConfig) – Config for which to generate FilterSpec
Returns:Individual filter spec
Return type:FilterSpec
get_specs()[source]

Get list of FilterSpec for the given querystring data.

This function does:

  • unpacks the querystring data to a list of LookupConfig
  • loops through all configs and uses appropriate children filters to generate list of FilterSpec
  • if any validations fails while generating specs, all errors are collected and depending on strict_mode it re-raises the errors or ignores them.
Returns:List of FilterSpec
Return type:list
repr(prefix=u'')[source]

Custom representation of the filterset

Parameters:prefix (str) – Prefix with which each line of the representation should be prefixed with. This allows to recursively get the representation of all descendants with correct indentation (children are indented compared to parent)
validate_key(key)[source]

Validate that LookupConfig key is correct.

This is the key as provided in the querystring. Currently key is validated against a regex expression.

Useful to filter out invalid filter querystring pairs since not whole querystring is not dedicated for filter purposes but could contain other information such as pagination information. In that case if the key is invalid key for filtering, we can simply ignore it without wasting time trying to get filter specification for it.

Parameters:key (str) – Key as provided in the querystring
class url_filter.filtersets.base.FilterSetOptions(options=None)[source]

Bases: object

Base class for handling options passed to FilterSet via Meta attribute.

class url_filter.filtersets.base.ModelFilterSetOptions(options=None)[source]

Bases: url_filter.filtersets.base.FilterSetOptions

Custom options for FilterSet used for model-generated filtersets.

model

Model – Model class from which FilterSet will extract necessary filters.

fields

None, list, optional – Specific model fields for which filters should be created for. By default it is None in which case for all fields filters will be created for.

exclude

list, optional – Specific model fields for which filters should not be created for.

bool, optional – Whether related/nested fields should be allowed when model fields are automatically determined (e.g. when explicit fields is not provided).

extra_kwargs

dict, optional – Additional kwargs to be given to auto-generated individual filters

url_filter.filtersets.django module
class url_filter.filtersets.django.ModelFilterSet(data=None, queryset=None, context=None, strict_mode=None, *args, **kwargs)[source]

Bases: url_filter.filtersets.base.BaseModelFilterSet

FilterSet for Django models.

The filterset can be configured via Meta class attribute, very much like Django’s ModelForm is configured.

Meta = <url_filter.filtersets.django.DjangoModelFilterSetOptions object>
filter_options_class

alias of DjangoModelFilterSetOptions

class url_filter.filtersets.django.DjangoModelFilterSetOptions(options=None)[source]

Bases: url_filter.filtersets.base.ModelFilterSetOptions

Custom options for ``FilterSet``s used for Django models.

bool, optional – Flag specifying whether reverse relationships should be allowed while creating filter sets for children models.

url_filter.filtersets.plain module
class url_filter.filtersets.plain.PlainModelFilterSet(data=None, queryset=None, context=None, strict_mode=None, *args, **kwargs)[source]

Bases: url_filter.filtersets.base.BaseModelFilterSet

FilterSet for plain Python objects.

The filterset can be configured via Meta class attribute, very much like Django’s ModelForm is configured.

Meta = <url_filter.filtersets.base.ModelFilterSetOptions object>
filter_backend_class

alias of url_filter.backends.plain.PlainFilterBackend

url_filter.filtersets.sqlalchemy module
class url_filter.filtersets.sqlalchemy.SQLAlchemyModelFilterSet(data=None, queryset=None, context=None, strict_mode=None, *args, **kwargs)[source]

Bases: url_filter.filtersets.base.BaseModelFilterSet

FilterSet for SQLAlchemy models.

The filterset can be configured via Meta class attribute, very much like Django’s ModelForm is configured.

Meta = <url_filter.filtersets.base.ModelFilterSetOptions object>
filter_backend_class

alias of url_filter.backends.sqlalchemy.SQLAlchemyFilterBackend

url_filter.integrations package
Submodules
url_filter.integrations.drf module
class url_filter.integrations.drf.DjangoFilterBackend[source]

Bases: rest_framework.filters.BaseFilterBackend

DRF filter backend which integrates with django-url-filter

This integration backend can be specified in global DRF settings:

# settings.py
REST_FRAMEWORK = {
    'DEFAULT_FILTER_BACKENDS': [
        'url_filter.integrations.drf.DjangoFilterBackend',
    ]
}

Alternatively filter backend can be specified per view/viewset bases:

class MyViewSet(ModelViewSet):
    queryset = MyModel.objects.all()
    filter_backends = [DjangoFilterBackend]
    filter_fields = ['field1', 'field2']

The following attributes can be specified on the view:

  • filter_class - explicit filter (FilterSet to be specific) class which should be used for filtering. When this attribute is supplied, this filterset will be used and all other attributes described below will are ignored.

  • filter_fields - list of strings which should be names of fields which should be included in the generated FilterSet. This is equivalent:

    class MyFilterSet(ModelFilterSet):
        class Meta(object):
            model = MyModel
            fields = ['fields1', ...]
    
  • filter_class_meta_kwargs - additional kwargs which should be passed in Meta for the generated FilterSet.

  • filter_class_default - base class to use while creating new FilterSet. This is primarily useful when using non-Django data-sources. By default default_filter_set is used.

default_filter_set

alias of url_filter.filtersets.django.ModelFilterSet

filter_queryset(request, queryset, view)[source]

Main method for filtering query object

Parameters:
  • request (HttpRequest) – Request object from the view
  • queryset – Query object for filtering
  • view (View) – View where this filter backend is being used
Returns:

Filtered query object if filtering class was determined by get_filter_class(). If not given queryset is returned.

Return type:

object

get_filter_class(view, queryset=None)[source]

Get filter class which will be used for filtering.

Parameters:
  • view (View) – DRF view/viewset where this filter backend is being used. Please refer to DjangoFilterBackend documentation for list of attributes which can be supplied in view to customize how filterset will be determined.
  • queryset – Query object for filtering
Returns:

  • FilterSetFilterSet class either directly specified in the view or dynamically constructed for the queryset model.
  • None – When appropriate FilterSet cannot be determined for filtering

get_filter_context(request, view)[source]

Get context to be passed to FilterSet during initialization

Parameters:
  • request (HttpRequest) – Request object from the view
  • view (View) – View where this filter backend is being used
Returns:

Context to be passed to FilterSet

Return type:

dict

url_filter.integrations.drf.URLFilterBackend

alias of url_filter.integrations.drf.DjangoFilterBackend

url_filter.integrations.drf_coreapi module
class url_filter.integrations.drf_coreapi.CoreAPIURLFilterBackend[source]

Bases: url_filter.integrations.drf.DjangoFilterBackend

Same as url_filter.integrations.drf.DjangoFilterBackend except this backend also implements coreapi interface for autogenerated API docs.

get_schema_fields(view)[source]

Get coreapi filter definitions

Returns all filters including their supported lookups.

Submodules
url_filter.exceptions module
exception url_filter.exceptions.Empty[source]

Bases: exceptions.Exception

Exception to be used when filter backend should return empty queryset when any of filter validations has failed.

exception url_filter.exceptions.SkipFilter[source]

Bases: exceptions.Exception

Exception to be used when any particular filter within the url_filter.filtersets.base.FilterSet should be skipped.

Possible reasons for skipping the field:

  • filter lookup config is invalid (e.g. using wrong field name - field is not present in filter set)
  • filter lookup value is invalid (e.g. submitted “a” for integer field)
url_filter.fields module
class url_filter.fields.MultipleValuesField(child=None, min_values=2, max_values=None, many_validators=None, delimiter=u', ', all_valid=True, *args, **kwargs)[source]

Bases: django.forms.fields.CharField

Custom Django field for validating/cleaning multiple values given in a single value separated by a delimiter.

Parameters:
  • child (Field, optional) – Another Django form field which should be used.
  • min_values (int, optional) – Minimum number of values which must be provided. By default at least 2 values are required.
  • max_values (int, optional) – Maximum number of values which can be provided. By default no maximum is enforced.
  • max_validators (list, optional) – Additional validators which should be used to validate all values once split by the delimiter.
  • delimiter (str, optional) – The delimiter by which the value will be split into multiple values. By default , is used.
  • all_valid (bool, optional) – When False, if any specific item does not pass validation it is ignored without failing complete field validation. Default is True which enforces validation for all items.
clean(value)[source]

Custom clean which first validates the value first by using standard CharField and if all passes, it applies similar validations for each value once its split.

many_run_validators(values)[source]

Run each validation from many_validators for the cleaned values.

many_to_python(value)[source]

Method responsible to split the value into multiple values by using the delimiter and cleaning each one as per the child field.

many_validate(values)[source]

Hook for validating all values.

url_filter.filters module
class url_filter.filters.BaseFilter(source=None, *args, **kwargs)[source]

Bases: object

Base class to be used for defining both filters and filtersets.

This class implements the bare-minimum functions which are used across both filters and filtersets however all other functionality must be implemented in subclasses. Additionally by using a single base class, both filters and filtersets inherit from the same base class hence instance checks can be easily done by filteset’s metaclass in order to find all declared filters defined in it.

Parameters:source (str) – Name of the attribute for which which filter applies to within the model of the queryset to be filtered as given to the FilterSet.
parent

FilterSet – Parent FilterSet to which this filter is bound to

name

str – Name of the field as it is defined in parent FilterSet

is_bound

bool – If this filter has been bound to a parent yet

bind(name, parent)[source]

Bind the filter to the filterset.

This method should be used by the parent FilterSet since it allows to specify the parent and name of each filter within the filterset.

components

List of all components (source names) of all parent filtersets.

repr(prefix=u'')[source]

Get the representation of the filter or its subclasses.

Subclasses must overwrite this method.

Note

This class should return unicode text data

Parameters:prefix (str) – All nested filtersets provide useful representation of the complete filterset including all descendants however in that case descendants need to be indented in order for the representation to get structure. This parameter is used to do just that. It specifies the prefix the representation must use before returning any of its representations.
root

This gets the root filterset.

source

Source field/attribute in queryset model to be used for filtering.

This property is helpful when source parameter is not provided when instantiating BaseFilter or its subclasses since it will use the filter name as it is defined in the FilterSet. For example:

>>> from .filtersets import FilterSet
>>> class MyFilterSet(FilterSet):
...     foo = Filter(form_field=forms.CharField())
...     bar = Filter(source='stuff', form_field=forms.CharField())
>>> fs = MyFilterSet()
>>> print(fs.filters['foo'].source)
foo
>>> print(fs.filters['bar'].source)
stuff
class url_filter.filters.CallableFilter(form_field=None, *args, **kwargs)[source]

Bases: url_filter.filters.Filter

Custom filter class meant to be subclassed in order to add support for custom lookups via custom callables

The goal of this filter is to provide:

  • support for custom callbacks (or overwrite existing ones)
  • support different filtering backends

Custom callable functions for lookups and different backends are defined via class methods by using the following method name pattern:

filter_<lookup_name>_for_<backend_name>

Obviously multiple methods can be used to implement functionality for multiple lookups and/or backends. This makes callable filters pretty flexible and ideal for implementing custom reusable filtering filters which follow DRY.

Examples

>>> from django.http import QueryDict
>>> from .filtersets import FilterSet

>>> class MyCallableFilter(CallableFilter):
...     @form_field_for_filter(forms.CharField())
...     def filter_foo_for_django(self, queryset, spec):
...         f = queryset.filter if not spec.is_negated else queryset.exclude
...         return f(foo=spec.value)
...     def filter_foo_for_sqlalchemy(self, queryset, spec):
...         op = operator.eq if not spec.is_negated else operator.ne
...         return queryset.filter(op(Foo.foo, spec.value))

>>> class MyFilterSet(FilterSet):
...     field = MyCallableFilter()

>>> f = MyFilterSet(data=QueryDict('field__foo=bar'), queryset=[])

Note

Unlike base class Filter this filter makes form_field parameter optional. Please note however that when form_field parameter is not provided, all custom filter callables should define their own appropriate form fields by using form_field_for_filter().

get_form_field(lookup)[source]

Get the form field for a particular lookup.

This method attempts to return form field for custom callables as set by form_field_for_filter(). When either custom lookup is not set or its form field is not set, super implementation is used to get the form field. If form field at that point is not found, this method raises AssertionError. That can only happen when form_field parameter is not provided during initialization.

get_spec(config)[source]

Get the FilterSpec for the given LookupConfig with appropriately set FilterSpec.filter_callable when the lookup is a custom lookup

lookups

Get all supported lookups for the filter

This property is identical to the super implementation except it also finds all custom lookups from the class methods and adds them to the set of supported lookups as returned by the super implementation.

class url_filter.filters.Filter(form_field, lookups=None, default_lookup=u'exact', is_default=False, no_lookup=False, *args, **kwargs)[source]

Bases: url_filter.filters.BaseFilter

Class which job is to convert leaf LookupConfig to FilterSpec

Each filter by itself is meant to be used a “filter” (field) in the FilterSet.

Examples

>>> from .filtersets import FilterSet
>>> class MyFilterSet(FilterSet):
...     foo = Filter(forms.CharField())
...     bar = Filter(forms.IntegerField())
Parameters:
  • form_field (Field) – Instance of Django’s forms.Field which will be used to clean the filter value as provided in the queryset. For example if field is IntegerField, this filter will make sure to convert the filtering value to integer before creating a FilterSpec.
  • lookups (list, optional) – List of strings of allowed lookups for this filter. By default all supported lookups are allowed.
  • default_lookup (str, optional) – If the lookup is not provided in the querystring lookup key, this lookup will be used. By default exact lookup is used. For example the default lookup is used when querystring key is user__profile__email which is missing the lookup so exact will be used.
  • is_default (bool, optional) – Boolean specifying if this filter should be used as a default filter in the parent FilterSet. By default it is False. Primarily this is used when querystring lookup key refers to a nested FilterSet however it does not specify which filter to use. For example lookup key user__profile intends to filter something in the user’s profile however it does not specify by which field to filter on. In that case the default filter within profile FilterSet will be used. At most, one default filter should be provided in the FilterSet.
  • no_lookup (bool, optional) – When True, this filter does not allow to explicitly specify lookups in the URL. For example id__gt will not be allowed. This is useful when a given filter should only support a single lookup and that lookup name should not be exposed in the URL. This is of particular use when defining custom callable filters. By default it is False.
form_field

Field – Django form field which is provided in initialization which should be used to validate data as provided in the querystring

default_lookup

str – Default lookup to be used as provided in initialization

is_default

bool – If this filter should be a default filter as provided in initialization

no_lookup

str – If this filter should not support explicit lookups as provided in initialization

clean_value(value, lookup)[source]

Clean the filter value as appropriate for the given lookup.

Parameters:
  • value (str) – Filter value as given in the querystring to be validated and cleaned by using appropriate Django form field
  • lookup (str) – Name of the lookup

See also

get_form_field()

get_form_field(lookup)[source]

Get the form field for a particular lookup.

This method does not blindly return form_field attribute since some lookups require to use different validations. For example if the form_field is CharField but the lookup is isnull, it makes more sense to use BooleanField as form field.

Parameters:lookup (str) – Name of the lookup
Returns:Instantiated form field appropriate for the given lookup.
Return type:Field
get_spec(config)[source]

Get the FilterSpec for the provided config.

Parameters:config (LookupConfig) – Lookup configuration for which to build FilterSpec. The lookup should be a leaf configuration otherwise ValidationError is raised.
Returns:spec constructed from the given configuration.
Return type:FilterSpec
lookups

Cached property for getting lookups this filter supports

The reason why we need as a property is because lookups cant be hardcoded. There are 3 main distinct possibilities which drive which lookups are supported:

  • lookups were explicitly provided in the filter instantiation in which case we use those lookups. For example:

    >>> f = Filter(forms.CharField(), lookups=['exact', 'contains'])
    
  • when filter is already bound to a parent filterset and root filterset has a defined filter_backend we use supported lookups as explicitly defined by the backend. This is necessary since different backends support different sets of lookups.

  • when nether lookups are explicitly provided and filter is not bound yet we have no choice but not support any lookups and so we use empty set as supported lookups

repr(prefix=u'')[source]

Get custom representation of the filter

The representation includes the following information:

  • filter class name
  • source name (same as source) when filter is bound to parent
  • primary form field (same as form_field)
  • which lookups this filter supports
  • default lookup (same as default_lookup)
  • if the filter is a default filter (same as is_default) when filter is bound to parent
  • if this filter does not support explicit lookups (same as no_lookup)
url_filter.filters.form_field_for_filter(form_field)[source]

Decorator for specifying form field for a custom callable filter on the filter callable method

Examples

>>> class MyFilterCallable(CallableFilter):
...     @form_field_for_filter(forms.CharField())
...     def filter_foo_for_django(self, queryset, spec):
...         return queryset
Parameters:form_field (Field) – Django form field which should be used for the decorated custom callable filter
Returns:Function which can be used to decorate a class method
Return type:func
url_filter.utils module
class url_filter.utils.FilterSpec(components, lookup, value, is_negated=False, filter_callable=None)[source]

Bases: object

Class for describing filter specification.

The main job of the FilterSet is to parse the submitted lookups into a list of filter specs. A list of these specs is then used by the filter backend to actually filter given queryset. That’s what FilterSpec provides - a way to portably define filter specification to be used by a filter backend.

The reason why filtering is decoupled from the FilterSet is because this allows to implement filter backends not related to Django.

components

list – A list of strings which are names of the keys/attributes to be used in filtering of the queryset. For example lookup config with key user__profile__email will have components of ``[‘user’, ‘profile’, ‘email’].

lookup

str – Name of the lookup how final key/attribute from components should be compared. For example lookup config with key user__profile__email__contains will have a lookup contains.

value

Value of the filter.

is_negated

bool, optional – Whether this filter should be negated. By default its False.

filter_callable

func, optional – Callable which should be used for filtering this filter spec. This is primaliry meant to be used by CallableFilter.

is_callable

Property for getting whether this filter specification is for a custom filter callable

class url_filter.utils.LookupConfig(key, data)[source]

Bases: object

Lookup configuration which is used by FilterSet to create a FilterSpec.

The main purpose of this config is to allow the use if recursion in FilterSet. Each lookup key (the keys in the querystring) is parsed into a nested one-key dictionary which lookup config stores.

For example the querystring:

?user__profile__email__endswith=gmail.com

is parsed into the following config:

{
    'user': {
        'profile': {
            'email': {
                'endswith': 'gmail.com'
            }
        }
    }
}
key

str – Full lookup key from the querystring. For example user__profile__email__endswith

data

dict, str – Either:

  • nested dictionary where the key is the next key within the lookup chain and value is another LookupConfig
  • the filtering value as provided in the querystring value
Parameters:
  • key (str) – Full lookup key from the querystring.
  • data (dict, str) – A regular vanilla Python dictionary. This class automatically converts nested dictionaries to instances of LookupConfig. Alternatively a filtering value as provided in the querystring.
as_dict()[source]

Converts the nested LookupConfig to a regular dict.

is_key_value()[source]

Check if this LookupConfig is not a nested LookupConfig but instead the value is a non-dict value.

name

If the data is nested LookupConfig, this gets its first lookup key.

value

If the data is nested LookupConfig, this gets its first lookup value which could either be another LookupConfig or actual filtering value.

class url_filter.utils.SubClassDict[source]

Bases: dict

Special-purpose dict with special getter for looking up values by finding matching subclasses.

This is better illustrated in an example:

>>> class Klass(object): pass
>>> class Foo(object): pass
>>> class Bar(Foo): pass
>>> mapping = SubClassDict({
...     Foo: 'foo',
...     Klass: 'klass',
... })
>>> print(mapping.get(Klass))
klass
>>> print(mapping.get(Foo))
foo
>>> print(mapping.get(Bar))
foo
get(k, d=None)[source]

If no value is found by using Python’s default implementation, try to find the value where the key is a base class of the provided search class.

url_filter.utils.dict_pop(key, d)[source]

Pop key from dictionary and return updated dictionary

url_filter.utils.dictify(obj)[source]

Convert any object to a dictionary.

If the given object is already an instance of a dict, it is directly returned. If not, then all the public attributes of the object are returned as a dict.

url_filter.utils.suppress(*args, **kwds)[source]

Suppress given exception type

For example:

>>> with suppress(ValueError):
...     print('test')
...     raise ValueError
test
url_filter.validators module
class url_filter.validators.MaxLengthValidator(limit_value, message=None)[source]

Bases: django.core.validators.MaxLengthValidator

Customer Django max length validator with better-suited error message

clean(x)[source]
code = u'max_length'
compare(a, b)[source]
deconstruct()

Returns a 3-tuple of class import path, positional arguments, and keyword arguments.

message = u''
class url_filter.validators.MinLengthValidator(limit_value, message=None)[source]

Bases: django.core.validators.MinLengthValidator

Customer Django min length validator with better-suited error message

clean(x)[source]
code = u'min_length'
compare(a, b)[source]
deconstruct()

Returns a 3-tuple of class import path, positional arguments, and keyword arguments.

message = u''

Django URL Filter

https://badge.fury.io/py/django-url-filter.svg https://readthedocs.org/projects/django-url-filter/badge/?version=latest https://travis-ci.org/miki725/django-url-filter.svg?branch=master https://coveralls.io/repos/miki725/django-url-filter/badge.svg?branch=master&service=github

Django URL Filter provides a safe way to filter data via human-friendly URLs.

Overview

The main goal of Django URL Filter is to provide an easy URL interface for filtering data. It allows the user to safely filter by model attributes and also allows to specify the lookup type for each filter (very much like Django’s filtering system in ORM).

For example the following will retrieve all items where the id is 5 and title contains "foo":

example.com/listview/?id=5&title__contains=foo

In addition to basic lookup types, Django URL Filter allows to use more sophisticated lookups such as in or year. For example:

example.com/listview/?id__in=1,2,3&created__year=2013

Requirements

  • Python 2.7, 3.x, pypy or pypy3
  • Django 1.8+ (there are plans to support older Django versions)
  • Django REST Framework 2 or 3 (only if you want to use DRF integration)

Installing

Easiest way to install this library is by using pip:

$ pip install django-url-filter

Usage Example

To make example short, it demonstrates Django URL Filter integration with Django REST Framework but it can be used without DRF (see below).

from url_filter.integrations.drf import DjangoFilterBackend


class UserViewSet(ModelViewSet):
    queryset = User.objects.all()
    serializer_class = UserSerializer
    filter_backends = [DjangoFilterBackend]
    filter_fields = ['username', 'email']

Alternatively filterset can be manually created and used directly to filter querysets:

from django.http import QueryDict
from url_filter.filtersets import ModelFilterSet


class UserFilterSet(ModelFilterSet):
    class Meta(object):
        model = User

query = QueryDict('email__contains=gmail&joined__gt=2015-01-01')
fs = UserFilterSet(data=query, queryset=User.objects.all())
filtered_users = fs.filter()

Above will automatically allow the use of all of the Django URL Filter features. Some possibilities:

# get user with id 5
example.com/users/?id=5

# get user with id either 5, 10 or 15
example.com/users/?id__in=5,10,15

# get user with id between 5 and 10
example.com/users/?id__range=5,10

# get user with username "foo"
example.com/users/?username=foo

# get user with username containing case insensitive "foo"
example.com/users/?username__icontains=foo

# get user where username does NOT contain "foo"
example.com/users/?username__icontains!=foo

# get user who joined in 2015 as per user profile
example.com/users/?profile__joined__year=2015

# get user who joined in between 2010 and 2015 as per user profile
example.com/users/?profile__joined__range=2010-01-01,2015-12-31

# get user who joined in after 2010 as per user profile
example.com/users/?profile__joined__gt=2010-01-01

Features

  • Human-friendly URLs

    Filter querystring format looks very similar to syntax for filtering in Django ORM. Even negated filters are supported! Some examples:

    example.com/users/?email__contains=gmail&joined__gt=2015-01-01
    example.com/users/?email__contains!=gmail  # note !
    
  • Related models

    Support related fields so that filtering can be applied to related models. For example:

    example.com/users/?profile__nickname=foo
    
  • Decoupled filtering

    How URLs are parsed and how data is filtered is decoupled. This allows the actual filtering logic to be decoupled from Django hence filtering is possible not only with Django ORM QuerySet but any set of data can be filtered (e.g. SQLAlchemy query objects) assuming corresponding filtering backend is implemented.

  • Usage-agnostic

    This library decouples filtering from any particular usage-pattern. It implements all the basic building blocks for creating filtersets but it does not assume how they will be used. To make the library easy to use, it ships with some integrations with common usage patterns like integration with Django REST Framework. This means that its easy to use in custom applications with custom requirements (which is probably most of the time!)

Indices and tables