Django distributed_task documentation!¶

Contents:

Getting started¶

Step by step introduction.

Overview¶

In between of celery <https://github.com/celery/celery>, distributed_task is extremely lightweight. We’d decided to keep it simple with less of flexibility but straight at the needs. Just define for each method a task-method using the decorator and delay it at run time.

Use case¶

The goal is to prevent “heavy” tasks to be executed between a web request and it’s response.

Examples for those tasks are:

Sending e-mails.
Generation of pdf/csv/... files.
Rendering of images, videos.

Installation¶

To use distributed_task, a Django <https://www.djangoproject.com/> installation is required.

Requirements¶

It’s well tested with following versions:

Version	Python	Django
1.0	2.7, 3.3, 3.4	1.5, 1.6, 1.7

Get the code¶

django_distributed_task package is available on pip:

pip install django_distributed_task

Register app in your Django settings.py¶

After install, register distributed_task to your INSTALLED_APPS:

INSTALLED_APPS = (
    ...
    "distributed_task",
)

And finally sync your database:

./manage.py syncdb distributed_task

Settings¶

Message Broker¶

The default message broker distributed_task.broker.backends.dummy.DummyMessageBroker does not provide any functionality.

The only message broker which is tested and available is RabbitMQ <http://www.rabbitmq.com/>.

Sample setup for RabbitMQ:

DISTRIBUTED_TASK_BROKER = {
    'BROKER': 'distributed_task.broker.backends.amqp.AMQPMessageBroker',
    'OPTIONS': {
        # Your connection data for RabbitMQ
        'HOST': 'localhost',
        'USERNAME': 'guest',
        'PASSWORD': 'guest',
        'PORT': 5672,

        # Your desired queue name (need to change it for multiple installations)
        'QUEUE': 'distributed_task_queue',
    }
}

Usage¶

The default call scheme is:: Task.delay -> Broker -> Worker -> Execution

Tasks¶

distributed_task will check in every installed app (INSTALLED_APPS) for a tasks.py file.

Define your first task¶

Create a tasks.py file in your desired app of choice:

from distributed_task import register_task

@register_task
def my_heavy_task_method():
    pass

Call your task¶

The decorator adds a delay method to your task. You can decide in runtime if you’d like to execute the task delayed or immediately.

Execute delayed in a worker process:

my_heavy_task_method.delay(*args, **kwargs)

Default method execution (bypasses task distribution):

my_heavy_task_method(*args, **kwargs)

Arguments¶

You can pass all args/kwargs to the my_heavy_task_method.delay method as you would call it normally. The serializer is also able to handle Django model instances but not QuerySets.

This works fine:

instance = User.objects.first()

my_heavy_task_method.delay('arg 1', user=instance, some_other_arg=False, some_float=12.5212)

Response / Return values¶

Method return values are not available. Maybe in a further version.

Start the worker¶

Finally, you need to start the worker process::: python manage.py run_worker

Getting Started¶

Installation:

Install