Welcome to Vumi’s documentation!¶

Contents:

Vumi Overview¶

$\tikzstyle{place}=[double copy shadow, shape=rounded rectangle, thick, inner sep=0pt, outer sep=0.5ex, minimum height=2em, minimum width=10em, node distance=10em, ]; \tikzstyle{rabbit}=[->, >=stealth, line width=0.2ex, auto, ]; \tikzstyle{route}=[sloped,midway,above=0.1em]; \tikzstyle{outbound}=[draw=black!50] \tikzstyle{inbound}=[draw=black] \tikzstyle{failure}=[draw=black, decorate, decoration={snake,pre length=1mm,post length=1mm}] \definecolor{darkred}{rgb}{0.5,0,0} \definecolor{darkgreen}{rgb}{0,0.5,0} \definecolor{darkblue}{rgb}{0,0,0.5} \node[place,draw=darkred!50,fill=darkred!20] (failure_worker) {Failure Workers}; \node[place,draw=darkblue!50,fill=darkblue!20] (transport) [below=of failure_worker] {Transports}; \node[place,draw=darkgreen!50,fill=darkgreen!20] (app_worker) [right=of transport] {Application Workers}; \draw[rabbit,inbound] (transport) to node [route] {inbound} (app_worker); \draw[rabbit,inbound,bend right] (transport) to node [route] {event} (app_worker); \draw[rabbit,outbound,bend right] (app_worker) to node [route] {outbound} (transport); \draw[rabbit,failure,bend right] (transport) to node [route] {failure} (failure_worker); \draw[rabbit,outbound,bend right] (failure_worker) to node [route] {outbound} (transport);$

A simple Vumi worker setup

Vumi Tutorial¶

New to Vumi? Well, you came to the right place: read this material to quickly get up and running.

Writing your first Vumi app - Part 1¶

This is the first part in a series of tutorials demonstrating how to develop Vumi apps.

We’ll assume you have a working knowledge of Python, RabbitMQ and VirtualEnv.

Where to get help

If you’re having trouble at any point feel free to drop by #vumi on irc.freenode.net to chat with other Vumi users who might be able to help.

In this part of the tutorial we’ll be creating and testing a simple working environment.

Environment Setup¶

Before we proceed let’s create an isolated working environment using VirtualEnv.

From the command line cd into a directory where you’d like to store your code then run the following command:

$ virtualenv --no-site-packages ve

This will create a ve directory where any libraries you install will go, thus isolating your environment. Once the virtual environment has been created activate it by running source ve/bin/activate.

Note

For this to work VirtualEnv needs to be installed. You can tell it’s installed by executing virtualenv from the command line. If that command runs successfully with no errors VirtualEnv is installed. If not you can install it by executing sudo pip install virtualenv from the command line.

Note

From this point onwards your virtual environment should always be active. The virtualenv is activated by running source ve/bin/activate.

Now that you created and activated the virtual environment install Vumi with the following command:

$ pip install -e git+git://github.com/praekelt/vumi.git@develop#egg=vumi

Note

This will install the development version of Vumi containing the latest-and-greatest features. Although the development branch is kept stable it is not recommended for production environments.

If this is your first Vumi application you need to take care of some initial RabbitMQ setup. Namely you need to add a vumi user and a develop virtual host and grant the required permissions. Vumi includes a script to do this for you which you can execute with the following command:

$ sudo ./ve/src/vumi/utils/rabbitmq.setup.sh

Note

Vumi workers communicate over RabbitMQ and requires it being installed and running. You can tell it’s installed and its current status by executing sudo rabbitmq-server from the command line. If the command is not found you can install RabbitMQ by executing sudo apt-get install rabbitmq-server from the command line (assuming you are on a Debian based distribution).

Testing the Environment¶

Let’s verify this worked. As a test you can create a Telnet worker and an echo application, both of which are included in Vumi.

Philosophy

A complete Vumi instance consists of a transport worker and an application worker which are managed as separate processes. A transport worker is responsible for sending messages to and receiving messages from some communications medium. An application worker processes messages received from a transport worker and generates replies.

Start the Telnet transport worker by executing the following command:

$ twistd -n --pidfile=transportworker.pid vumi_worker --worker-class vumi.transports.telnet.TelnetServerTransport --set-option=transport_name:telnet --set-option=telnet_port:9010

This utilizes Twisted to start a Telnet process listening on port 9010. Specifically it uses Vumi’s builtin TelnetServerTransport to handle communication with Telnet clients. Note that we specify telnet as the transport name when providing --set-option=transport_name:telnet. When starting the application worker as described next the same name should be used, thus connecting the transport worker with the application worker.

Philosophy

A transport worker is responsible for sending messages over and receiving messages from some communication medium. For this example we are using a very simple transport that communicates over Telnet. Other transport mechanisms Vumi supports include SMPP, XMPP, Twitter, IRC, HTTP and a variety of mobile network aggregator specific messaging protocols. In subsequent parts of this tutorial we’ll be using the XMPP transport to communicate over Google Talk.

In a command line session you should now be able to connect to the transport worker via Telnet:

$ telnet localhost 9010

If you keep an eye on the transport worker’s output you should see the following as clients connect:

2012-03-06 12:06:32+0200 [twisted.internet.protocol.ServerFactory] Registering client connected from '127.0.0.1:57995'

Note

At this point only the transport worker is running so Telnet input will not be processed yet. To process the input and generate an echo we need to start the application worker.

In a new command line session start the echo application worker by executing the following command:

$ twistd -n --pidfile=applicationworker.pid vumi_worker --worker-class vumi.demos.words.EchoWorker --set-option=transport_name:telnet

This utilizes Twisted to start a Vumi EchoWorker process connected to the previously created Telnet transport worker.

Philosophy

An application worker is responsible for processing messages received from a transport worker and generating replies - it holds the application logic. For this example we are using an echo worker that will simply echo messages it receives back to the transport worker. In subsequent parts of this tutorial we’ll be utilizing A.I. to generate seemingly intelligent replies.

Now if you enter something in your previously created Telnet session you should immediately receive an echo. The application worker’s output should reflect the activity, for example when entering hallo world:

2012-03-06 12:10:39+0200 [WorkerAMQClient,client] User message: hallo world

That concludes part 1 of this tutorial. In part 2 we’ll be creating a Google Talk chat bot.

Writing your first Vumi app - Part 2¶

This is the second part in a series of tutorials demonstrating how to develop Vumi apps.

If you haven’t done so already you might want to work through part 1 of this tutorial before proceeding.

In this part of the tutorial we’ll be creating a simple chat bot communicating over Google Talk.

More specifically we’ll be utilizing Vumi’s XMPP transport worker to log into a Google Talk account and listen for incoming chat messages. When messages are received an Alice Bot based application worker will determine an appropriate response based on the incoming message. The XMPP transport worker will then send the response. For another Google Talk user chatting with the Vumi connected account it should appear as if she is conversing with another human being.

Note

Remember your virtual environment should be active. Activate it by running running source ve/bin/activate.

XMPP Transport Worker¶

Continuing from part 1 of this tutorial, instead of using the Telnet transport worker we’ll be using Vumi’s built-in XMPP transport worker to communicate over Google Talk.

In order to use the XMPP transport worker you first need to create a configuration file.

To do this, create a transport.yaml file in your current directory and edit it to look like this (replacing "username" and "password" with your specific details):

transport_name: xmpp_transport
username: "username"
password: "password"
status: Playing with Vumi.
host: talk.google.com
port: 5222

Going through that line by line:

transport_name: xmpp_transport - specifies the transport name. This identifies the transport worker for subsequent connection by application workers.

username: "username" - the Google Talk account username to which the transport worker will connect.

password: "password" - the Google Talk account password.

status: Playing with Vumi - causes the Google Talk account’s chat status to change to Playing with Vumi.

host: talk.google.com - The XMPP host to connect to. Google Talk uses talk.google.com.

port: 5222 - The XMPP port to connect to. Google Talk uses 5222.

Note

Vumi utilizes YAML based configuration files to provide configuration parameters to workers, both transport and application. YAML is a human friendly data serialization standard that works quite well for specifying configurations.

Now start the XMPP transport worker with the created configuration by executing the following command:

$ twistd -n --pidfile=transportworker.pid vumi_worker --worker-class vumi.transports.xmpp.XMPPTransport --config=./transport.yaml

SASLNoAcceptableMechanism Exceptions

In the event of this command raising a twisted.words.protocols.jabber.sasl.SASLNoAcceptableMechanism exception you should upgrade your pyOpenSSL package by executing pip install --upgrade pyOpenSSL from the command line.

Note

This is different from the example in part 1 of this tutorial in that we no longer set any configuration options through the command line. Instead all configuration is contained in the specified transport.yaml config file.

This causes a Vumi XMPP transport worker to connect to the configuration specified Google Talk account and listen for messages. You should now be able to start messaging the account from another Google Talk account using any Google Talk client (although no response will be generated until the application worker is instantiated).

Alice Bot Application Worker¶

Continuing from part 1 of this tutorial, instead of using the echo application worker we’ll be creating our own worker to generate seemingly intelligent responses.

Philosophy

Remember application workers are responsible for processing messages received from transport workers and generating replies - it holds the application logic. When developing Vumi applications you’ll mostly be implementing application workers to process messages based on your use case. For the most part you’ll be relying on Vumi’s built-in transport workers to take care of the communications medium. This enables you to forget about the hairy details of the communications medium and instead focus on the fun stuff.

Before we proceed let’s install our dependencies. We’ll be using PyAIML to provide our bot with knowledge. Install it by executing the following command:

$ pip install http://sourceforge.net/projects/pyaiml/files/PyAIML%20%28unstable%29/0.8.6/PyAIML-0.8.6.tar.gz

We also need a brain for our bot. Download a precompiled brain by executing the following command:

$ wget https://github.com/downloads/praekelt/public-eggs/alice.brn

Note

For the sake of simplicity we’re using an existing brain. You can however compile your own brain by downloading the free Alice AIML set and learning it as described in the PyAIML examples. Perhaps you rather want a Fake Captain Kirk.

Now we can move on to creating the application worker. Create a workers.py file in your current directory and edit it to look like this:

import aiml
from vumi.application.base import ApplicationWorker

class AliceApplicationWorker(ApplicationWorker):

    def __init__(self, *args, **kwargs):
        self.bot = aiml.Kernel()
        self.bot.bootstrap(brainFile="alice.brn")
        return super(AliceApplicationWorker, self).__init__(*args, **kwargs)

    def consume_user_message(self, message):
        message_content = message['content']
        message_user = message.user()
        response = self.bot.respond(message_content, message_user)
        self.reply_to(message, response)

The code is straightforward. Application workers are represented by a class that subclasses vumi.application.base.ApplicationWorker. In this example the __init__ method is overridden to initialize our bot’s brain. The heart of application workers though is the consume_user_message method, which is passed messages for processing as they are received by transport workers. The message argument contains details on the received message. In this example the content of the message is retrieved from message['content'], and the Google Talk user sending the message is determined by calling message.user(). A response is then generated for the specific user utilizing the bot by calling self.bot.respond(message_content, message_user). This response is then sent as a reply to the original message by calling self.reply_to(message, response). The transport worker then takes care of sending the response to the correct user over the communications medium.

Philosophy

The application worker has very little knowledge about and does not need to know the specifics of the communications medium. In this example we could just as easily have communicated over SMS or even Twitter without having to change the application worker’s implementation.

Now start the Alice Bot application worker in a new command line session by executing the following command:

$ twistd -n --pidfile=applicationworker.pid vumi_worker --worker-class workers.AliceApplicationWorker --set-option=transport_name:xmpp_transport

Note

Again note how the application worker is connected to the previously defined, already running transport worker by specifying --set-option=transport_name:xmpp_transport.

Now with both the transport worker and application worker running you should be able to send a chat message to the Google Talk account configured in transport.yaml and receive a seemingly intelligent response generated by our Alice Bot.

Coming soon¶

The tutorial ends here for the time being. Future installments of the tutorial will cover:

Advanced applications.
Scaling and deploying.

In the meantime, you might want to check out some other docs.

ScaleConf Workshop - General Introduction¶

Note

These instructions were written for the first Vumi workshop on the 21st of April 2013, right after the ScaleConf conference in Cape Town.

Spotted an error? Please feel free to contribute to the documentation.

What is Vumi?¶

Vumi is a scalable, multi channel messaging platform. It has been designed to allow large scale mobile messaging in the majority world. It is actively being developed by the Praekelt Foundation and other contributors. It is available as Open Source software under the BSD license.

What were the design goals?¶

The Praekelt Foundation has a lot of experience building mobile messaging campaigns in the areas such as mobile health, education and democracy. Unfortunately, a lot of this experience comes from having built systems that caused problems in terms of scale and/or maintenance.

Key learnings from these mistakes led to a number of guiding principles in the design of Vumi, such as:

The campaign application logic should be decoupled from how it communicates with the end user.
The campaign application and the means of communication with the end-user should each be re-usable in a different context.
The system should be able to scale by adding more commodity machines, i.e. it should scale horizontally.

The above mentioned guiding principles resulted in a number of core concepts that make up a Vumi application.

A Vumi Message¶

A Vumi Message is the means of communication inside Vumi. Esentially a Vumi Message is just a bit of JSON that contains information on where a message was received from, who it was addressed to, what the message contents were and some extra metadata to allow it to be routed from and end-user to an application and back again.

Transports¶

Transports provide the communication channel to end users by integrating into various services such as chat systems, mobile network operators or possibly even traditional voice phone lines.

Transports are tasked with translating an inbound request into a standardized Vumi Message and vice-versa.

A simple example would be an SMS, which when received is converted into a bit of JSON that looks something like this:

{
    "message_id": "message1",
    "to_addr": "1234",
    "from_addr": "27761234567",
    "content": "This is an incoming SMS!",
    "transport_name": "smpp_transport",
    "transport_type": "sms",
    "transport_metadata": {
        // this is a dictionary containing
        // transport specific data
    }
}

Applications¶

Applications are tasked with either generating messages to be sent to or acting on the messages received from end users via the transports.

As a general rule the Applications should not care about which transport the message was received from, it merely acts on the message contents and provides a suitable reply.

A reply message looks something like this:

{
    "message_id": "message2",
    "in_reply_to": "message1",
    "to_addr": "27761234567",
    "from_addr": "1234",
    "content": "Thanks! We've received your SMS!",
    "transport_name": "smpp_transport",
    "transport_type": "sms",
    "helper_metadata": {
        // this is a dictionary containing
        // application specific data
    }
}

Dispatchers¶

Dispatchers are an optional means of connecting Transports and Applications. They allow for more complicated routing between the two.

A simple scenario is an application that receives from a USSD transport but requires the option of also replying via an SMS transport. A dispatcher would allow one to contruct this.

Dispatchers do this by inspecting the messages exchanged between the Transport and the Application and then deciding where it needs to go.

+----------------+
| SMS Transport  |<----+   +------------+    +-------------+
+----------------+     +-->|            |    |             |
                           | Dispatcher |<-->| Application |
+----------------+     +-->|            |    |             |
| USSD Transport |<----+   +------------+    +-------------+
+----------------+

How does it work?¶

All of these different components are built using the Python programming language using Twisted, an event driven networking library.

The messages between the different components are exchanged and routed using RabbitMQ a high performance AMQP message broker.

For data storage Redis is used for data that are generally temporary but and may potentially be lost. Riak is used for things that need strong availability guarantees.

A sample use case of Redis would be to store session state whereas Riak would be used to store all messages sent and received indefinitely.

Supervisord is used to manage all the different processes and provide any easy commandline tool to start and stop them.

Let’s get started!¶

As part of the workshop we will provide you with a South African USSD code and an SMS longcode. In the next section we’ll help you get Vumi running on your local machine so you can start developing your first application!

Forwarding SMSs from an SMPP bind to a URL¶

A simple use case for Vumi is to aggregate incoming SMSs and forward them via HTTP POST to a URL.

In this use case we are going to:

Use a SMSC simulator for local development.

Configure Vumi accept all incoming and outgoing messages on an SMPP bind.

Setup a worker that forwards all incoming messages to a URL via HTTP POST.

Setup Supervisord to manage all the different processes.

Note

Vumi relies for a large part on AMQP for its routing capabilities and some basic understanding is assumed. Have a look at http://blog.springsource.com/2010/06/14/understanding-amqp-the-protocol-used-by-rabbitmq/ for a more detailed explanation of AMQP.

Installing the SMSC simulator¶

Go to the ./utils directory in the Vumi repository and run the bash script called install_smpp_simulator.sh. This will install the SMSC simulator from http://seleniumsoftware.com on your local machine. This simulator does exactly the same as a normal SMSC would do with the exception that it doesn’t actually relay the messages to mobile networks.:

$ cd ./utils
$ ./install_smpp_simulator.sh

This will have installed the application in the ./utils/smppsim/SMPPSim directory.

By default the SMPP simulator tries to open port 88 for it’s HTTP console, since you often need administrative rights to open ports lower than 1024 let’s change that to 8080 instead.

Line 60 of ./utils/smppsim/SMPPSim/conf/smppsim.props says:

HTTP_PORT=88

Change this to:

HTTP_PORT=8080

Another change we need to make is on line 83:

ESME_TO_ESME=TRUE

Needs to be changed to, FALSE:

ESME_TO_ESME=FALSE

Having this set to True sometimes causes the SMSC and Vumi to bounce messages back and forth without stopping.

Note

The simulator is a Java application and we’re assuming you have Java installed correctly.

Configuring Vumi¶

Vumi applications are made up of at least two components, the Transport which deals with in & outbound messages and the Application which acts on the messages received and potentially generates replies.

SMPP Transport¶

Vumi’s SMPP Transport can be configured by a YAML file, ./config/example_smpp.yaml. For this example, this is what our SMPP configuration looks like:

transport_name: smpp_transport
system_id: smppclient1  # username
password: password      # password
host: localhost         # the host to connect to
port: 2775              # the port to connect to

The SMPP Transport publishes inbound messages in Vumi’s common message format and accepts the same format for outbound messages.

Here is a sample message:

{
    "message_id": "message1",
    "to_addr": "1234",
    "from_addr": "27761234567",
    "content": "This is an incoming SMS!",
    "transport_name": "smpp_transport",
    "transport_type": "sms",
    "transport_metadata": {
        // this is a dictionary containing
        // transport specific data
    }
}

HTTP Relay Application¶

Vumi ships with a simple application which forwards all messages it receives as JSON to a given URL with the option of using HTTP Basic Authentication when doing so. This application is also configured using the YAML file:

Setting up the webserver that responds to the HTTP request that the HTTPRelayApplication makes is left as an exercise for the reader. The HTTPRelayApplication has the ability to automatically respond to incoming messages based on the HTTP response received.

To do this:

The resource must return with a status of 200

The resource must set an HTTP Header X-Vumi-HTTPRelay-Reply and it must be set to true (case insensitive)

Any content that is returned in the body of the response is sent back as a message. If you want to limit this to 140 characters for use with SMS then that is the HTTP resource’s responsibility.

Supervisord!¶

Let’s use Supervisord to ensure all the different parts keep running. Here is the configuration file supervisord.example.conf:

[inet_http_server]         ; inet (TCP) server disabled by default
port=127.0.0.1:9010        ; (ip_address:port specifier, *:port for all iface)

[supervisord]
pidfile=./tmp/pids/supervisord.pid ; (supervisord pidfile;default supervisord.pid)

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=http://127.0.0.1:9010 ; use an http:// url to specify an inet socket

[program:transport]
command=twistd -n
    --pidfile=./tmp/pids/%(program_name)s.pid
    vumi_worker
    --worker-class=vumi.transports.smpp.SmppTransport
    --config=./config/example_smpp.yaml
stdout_logfile=./logs/%(program_name)s_%(process_num)s.log
stderr_logfile=./logs/%(program_name)s_%(process_num)s.err

[program:application]
command=twistd -n
    --pidfile=./tmp/pids/%(program_name)s.pid
    vumi_worker
    --worker-class=vumi.application.http_relay.HTTPRelayApplication
    --config=./config/example_http_relay.yaml
autorestart=true
stdout_logfile=./logs/%(program_name)s_%(process_num)s.log
stderr_logfile=./logs/%(program_name)s_%(process_num)s.err

[program:smsc]
command=java
    -Djava.net.preferIPv4Stack=true
    -Djava.util.logging.config.file=conf/logging.properties
    -jar smppsim.jar
    conf/smppsim.props
autorestart=true
directory=./utils/smppsim/SMPPSim/
stdout_logfile=./logs/%(program_name)s_%(process_num)s.log
stderr_logfile=./logs/%(program_name)s_%(process_num)s.err

Ensure you’re in your python virtualenv and start it with the following command:

$ supervisord -c etc/supervisord.example.conf

You’ll be able to see the HTTP management console at http://localhost:9010/ or at the command line with:

$ supervisorctl -c etc/supervisord.example.conf

Let’s give it a try:¶

Go to http://localhost:8080 and send an SMS to Vumi via “Inject an MO message”.
Type a message, it doesn’t matter what destination_addr you chose, all incoming messages will be routed using the SMPP Transport’s transport_name to the application subscribed to those messages. The HTTPRelayApplication will HTTP POST to the URL provided.

Applications¶

Vumi applications implement application logic or call out to external services that implement such logic. Usually you will implement your own application workers but Vumi does provide a base application worker class and a few generic application workers.

Base class for applications¶

A base class you should extend when writing applications.

Application¶

class vumi.application.base.ApplicationConfig(config_data, static=False)¶

Base config definition for applications.

You should subclass this and add application-specific fields.

Configuration options:

Parameters:	amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance. transport_name (str) – The name this application instance will use to create its queues. send_to (dict) – ‘send_to’ configuration dict.

class vumi.application.base.ApplicationWorker(options, config=None)¶

Base class for an application worker.

Handles vumi.message.TransportUserMessage and vumi.message.TransportEvent messages.

Application workers may send outgoing messages using reply_to() (for replies to incoming messages) or send_to() (for messages that are not replies).

send_to() can take either an endpoint parameter to specify the endpoint to send on (and optionally add additional message data from application configuration).

ALLOWED_ENDPOINTS lists the endpoints this application is allowed to send messages to using the send_to() method. If it is set to None, any endpoint is allowed.

Messages sent via send_to() pass optional additional data from configuration to the TransportUserMessage constructor, based on the endpoint parameter passed to send_to. This usually contains information useful for routing the message.

An example send_to() configuration might look like:

- send_to:
  - default:
    transport_name: sms_transport

NOTE: If you are using non-endpoint routing, ‘transport_name’ must be defined for each send_to section since dispatchers rely on this for routing outbound messages.

The available set of endpoints defaults to just the single endpoint named default. If applications wish to define their own set of available endpoints they should override ALLOWED_ENDPOINTS. Setting ALLOWED_ENDPOINTS to None allows the application to send on arbitrary endpoint names.

CONFIG_CLASS¶: alias of ApplicationConfig

static check_endpoint(allowed_endpoints, endpoint)¶

Check that endpoint is in the list of allowed endpoints.

Parameters:	allowed_endpoints (list) – List (or set) of allowed endpoints. If `allowed_endpoints` is `None`, all endpoints are allowed. endpoint (str) – Endpoint to check. The special value `None` is equivalent to `default`.

close_session(message)¶

Close a session.

The .reply_to() method should not be called when the session is closed.

consume_ack(event)¶: Handle an ack message.

consume_delivery_report(event)¶: Handle a delivery report.

consume_nack(event)¶: Handle a nack message

consume_user_message(message)¶: Respond to user message.

dispatch_event(event)¶: Dispatch to event_type specific handlers.

dispatch_user_message(message)¶: Dispatch user messages to handler.

new_session(message)¶

Respond to a new session.

Defaults to calling consume_user_message.

setup_application()¶

All application specific setup should happen in here.

Subclasses should override this method to perform extra setup.

setup_worker()¶

Set up basic application worker stuff.

You shouldn’t have to override this in subclasses.

teardown_application()¶: Clean-up of setup done in setup_application should happen here.

HTTP Relay¶

Calls out to an external HTTP API that implements application logic and provides a similar API for application logic to call when sending messages.

HTTP Relay¶

class vumi.application.http_relay.HTTPRelayConfig(config_data, static=False)¶

Configuration options:

Parameters:

amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance.
transport_name (str) – The name this application instance will use to create its queues.
send_to (dict) – ‘send_to’ configuration dict.
url (URL) – URL to submit incoming message to.
event_url (URL) – URL to submit incoming events to. (Defaults to the same as ‘url’).
http_method (str) – HTTP method for submitting messages.
auth_method (str) – HTTP authentication method.
username (str) – Username for HTTP authentication.
password (str) – Password for HTTP authentication.

class vumi.application.http_relay.HTTPRelayApplication(options, config=None)¶

RapidSMS Relay¶

Calls out to an application implemented in RapidSMS.

RapidSMS Relay¶

class vumi.application.rapidsms_relay.RapidSMSRelayConfig(config_data, static=False)¶

RapidSMS relay configuration.

Configuration options:

Parameters:

amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance.
transport_name (str) – The name this application instance will use to create its queues.
send_to (dict) – ‘send_to’ configuration dict.
web_path (str) – Path to listen for outbound messages from RapidSMS on.
web_port (int) – Port to listen for outbound messages from RapidSMS on.
redis_manager (dict) – Redis manager configuration (only required if allow_replies is true)
allow_replies (bool) – Whether to support replies via the in_reply_to argument from RapidSMS.
vumi_username (str) – Username required when calling web_path (default: no authentication)
vumi_password (str) – Password required when calling web_path
vumi_auth_method (str) – Authentication method required when calling web_path.The ‘basic’ method is currently the only available method
vumi_reply_timeout (int) – Number of seconds to keep original messages in redis so that replies may be sent via in_reply_to.
allowed_endpoints (list) – List of allowed endpoints to send from.
rapidsms_url (URL) – URL of the rapidsms http backend.
rapidsms_username (str) – Username to use for the rapidsms_url (default: no authentication)
rapidsms_password (str) – Password to use for the rapidsms_url
rapidsms_auth_method (str) – Authentication method to use with rapidsms_url. The ‘basic’ method is currently the only available method.
rapidsms_http_method (str) – HTTP request method to use for the rapidsms_url

class vumi.application.rapidsms_relay.RapidSMSRelay(options, config=None)¶: Application that relays messages to RapidSMS.

Sandbox¶

Runs custom application logic in a sandbox.

Sandbox application workers¶

Sandbox¶

class vumi.application.sandbox.SandboxConfig(config_data, static=False)¶

Configuration options:

Parameters:

amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance.
transport_name (str) – The name this application instance will use to create its queues.
send_to (dict) – ‘send_to’ configuration dict.
sandbox (dict) – Dictionary of resources to provide to the sandbox. Keys are the names of resources (as seen inside the sandbox). Values are dictionaries which must contain a cls key that gives the full name of the class that provides the resource. Other keys are additional configuration for that resource.
executable (str) – Full path to the executable to run in the sandbox.
args (list) – List of arguments to pass to the executable (not including the path of the executable itself).
path (str) – Current working directory to run the executable in.
env (dict) – Custom environment variables for the sandboxed process.
timeout (int) – Length of time the subprocess is given to process a message.
recv_limit (int) – Maximum number of bytes that will be read from a sandboxed process’ stdout and stderr combined.
rlimits (dict) – Dictionary of resource limits to be applied to sandboxed processes. Defaults are fairly restricted. Keys maybe names or values of the RLIMIT constants in Python resource module. Values should be appropriate integers.
logging_resource (str) – Name of the logging resource to use to report errors detected in sandboxed code (e.g. lines written to stderr, unexpected process termination). Set to null to disable and report these directly using Twisted logging instead.
sandbox_id (str) – This is set based on individual messages.

class vumi.application.sandbox.Sandbox(options, config=None)¶: Sandbox application worker.

Javascript Sandbox¶

class vumi.application.sandbox.JsSandboxConfig(config_data, static=False)¶

JavaScript sandbox configuration.

Configuration options:

Parameters:

amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance.
transport_name (str) – The name this application instance will use to create its queues.
send_to (dict) – ‘send_to’ configuration dict.
sandbox (dict) – Dictionary of resources to provide to the sandbox. Keys are the names of resources (as seen inside the sandbox). Values are dictionaries which must contain a cls key that gives the full name of the class that provides the resource. Other keys are additional configuration for that resource.
executable (str) – Full path to the executable to run in the sandbox.
args (list) – List of arguments to pass to the executable (not including the path of the executable itself).
path (str) – Current working directory to run the executable in.
env (dict) – Custom environment variables for the sandboxed process.
timeout (int) – Length of time the subprocess is given to process a message.
recv_limit (int) – Maximum number of bytes that will be read from a sandboxed process’ stdout and stderr combined.
rlimits (dict) – Dictionary of resource limits to be applied to sandboxed processes. Defaults are fairly restricted. Keys maybe names or values of the RLIMIT constants in Python resource module. Values should be appropriate integers.
sandbox_id (str) – This is set based on individual messages.
javascript (str) – JavaScript code to run.
app_context (str) – Custom context to execute JS with.
logging_resource (str) – Name of the logging resource to use to report errors detected in sandboxed code (e.g. lines written to stderr, unexpected process termination). Set to null to disable and report these directly using Twisted logging instead.

class vumi.application.sandbox.JsSandbox(options, config=None)¶

Configuration options:

As for Sandbox except:

executable defaults to searching for a node.js binary.
args defaults to the JS sandbox script in the vumi.application module.
An instance of JsSandboxResource is added to the sandbox resources under the name js if no js resource exists.
An instance of LoggingResource is added to the sandbox resources under the name log if no log resource exists.
logging_resource is set to log if it is not set.
An extra ‘javascript’ parameter specifies the javascript to execute.
An extra optional ‘app_context’ parameter specifying a custom context for the ‘javascript’ application to execute with.

Example ‘javascript’ that logs information via the sandbox API (provided as ‘this’ to ‘on_inbound_message’) and checks that logging was successful:

api.on_inbound_message = function(command) {
    this.log_info("From command: inbound-message", function (reply) {
        this.log_info("Log successful: " + reply.success);
        this.done();
    });
}

Example ‘app_context’ that makes the Node.js ‘path’ module available under the name ‘path’ in the context that the sandboxed javascript executes in:

{path: require('path')}

Javascript File Sandbox¶

class vumi.application.sandbox.JsFileSandbox(options, config=None)¶

class CONFIG_CLASS(config_data, static=False)¶

Configuration options:

Parameters:

amqp_prefetch_count (int) – The number of messages fetched concurrently from each AMQP queue by each worker instance.
transport_name (str) – The name this application instance will use to create its queues.
send_to (dict) – ‘send_to’ configuration dict.
sandbox (dict) – Dictionary of resources to provide to the sandbox. Keys are the names of resources (as seen inside the sandbox). Values are dictionaries which must contain a cls key that gives the full name of the class that provides the resource. Other keys are additional configuration for that resource.
executable (str) – Full path to the executable to run in the sandbox.
args (list) – List of arguments to pass to the executable (not including the path of the executable itself).
path (str) – Current working directory to run the executable in.
env (dict) – Custom environment variables for the sandboxed process.
timeout (int) – Length of time the subprocess is given to process a message.
recv_limit (int) – Maximum number of bytes that will be read from a sandboxed process’ stdout and stderr combined.
rlimits (dict) – Dictionary of resource limits to be applied to sandboxed processes. Defaults are fairly restricted. Keys maybe names or values of the RLIMIT constants in Python resource module. Values should be appropriate integers.
logging_resource (str) – Name of the logging resource to use to report errors detected in sandboxed code (e.g. lines written to stderr, unexpected process termination). Set to null to disable and report these directly using Twisted logging instead.
sandbox_id (str) – This is set based on individual messages.
javascript_file (str) – The file containting the Javascript to run
app_context (str) – Custom context to execute JS with.

Sandbox resources¶

RedisResource¶

class vumi.application.sandbox.RedisResource(name, app_worker, config)¶

Resource that provides access to a simple key-value store.

Configuration options:

Parameters:

redis_manager (dict) – Redis manager configuration options.
keys_per_user_soft (int) – Maximum number of keys each user may make use of in redis before usage warnings are logged. (default: 80% of hard limit).
keys_per_user_hard (int) – Maximum number of keys each user may make use of in redis (default: 100). Falls back to keys_per_user.
keys_per_user (int) – Synonym for keys_per_user_hard. Deprecated.

handle_delete(*args, **kwargs)¶

Delete a key.

Command fields:

key: The key to delete.

Reply fields:

success: true if the operation was successful, otherwise false.

Example:

api.request(
    'kv.delete',
    {key: 'foo'},
    function(reply) {
        api.log_info('Value deleted: ' +
                     reply.success);
    }
);

handle_get(*args, **kwargs)¶

Retrieve the value of a key.

Command fields:

key: The key whose value should be retrieved.

Reply fields:

success: true if the operation was successful, otherwise false.
value: The value retrieved.

Example:

api.request(
    'kv.get',
    {key: 'foo'},
    function(reply) {
        api.log_info(
            'Value retrieved: ' +
            JSON.stringify(reply.value));
    }
);

handle_incr(*args, **kwargs)¶

Atomically increment the value of an integer key.

The current value of the key must be an integer. If the key does not exist, it is set to zero.

Command fields:

key: The key to delete.
amount: The integer amount to increment the key by. Defaults to 1.

Reply fields:

success: true if the operation was successful, otherwise false.
value: The new value of the key.

Example:

api.request(
    'kv.incr',
    {key: 'foo',
     amount: 3},
    function(reply) {
        api.log_info('New value: ' +
                     reply.value);
    }
);

handle_set(*args, **kwargs)¶

Set the value of a key.

Command fields:

key: The key whose value should be set.
value: The value to store. May be any JSON serializable object.
seconds: Lifetime of the key in seconds. The default null indicates that the key should not expire.

Reply fields:

success: true if the operation was successful, otherwise false.

Example:

api.request(
    'kv.set',
    {key: 'foo',
     value: {x: '42'}},
    function(reply) { api.log_info('Value store: ' +
                                   reply.success); });

OutboundResource¶

class vumi.application.sandbox.OutboundResource(name, app_worker, config)¶

Resource that provides the ability to send outbound messages.

Includes support for replying to the sender of the current message, replying to the group the current message was from and sending messages that aren’t replies.

JsSandboxResource¶

class vumi.application.sandbox.JsSandboxResource(name, app_worker, config)¶

Resource that initializes a Javascript sandbox.

Typically used alongside vumi/applicaiton/sandboxer.js which is a simple node.js based Javascript sandbox.

Requires the worker to have a javascript_for_api method.

LoggingResource¶

class vumi.application.sandbox.LoggingResource(name, app_worker, config)¶

Resource that allows a sandbox to log messages via Twisted’s logging framework.

handle_critical(api, command)¶

Logs a message at the CRITICAL log level.

See handle_log() for details.

handle_debug(api, command)¶

Logs a message at the DEBUG log level.

See handle_log() for details.

handle_error(api, command)¶

Logs a message at the ERROR log level.

See handle_log() for details.

handle_info(api, command)¶

Logs a message at the INFO log level.

See handle_log() for details.

handle_log(*args, **kwargs)¶

Log a message at the specified severity level.

The other log commands are identical except that level need not be specified. Using the log-level specific commands is preferred.

Command fields:

level: The severity level to log at. Must be an integer log level. Default severity is the INFO log level.
msg: The message to log.

Reply fields:

success: true if the operation was successful, otherwise false.

Example:

api.request(
    'log.log',
    {level: 20,
     msg: 'Abandon ship!'},
    function(reply) {
        api.log_info('New value: ' +
                     reply.value);
    }
);

handle_warning(api, command)¶

Logs a message at the WARNING log level.

See handle_log() for details.

log(api, msg, level)¶

Logs a message via vumi.log (i.e. Twisted logging).

Sub-class should override this if they wish to log messages elsewhere. The api parameter is provided for use by such sub-classes.

The log method should always return a deferred.

HttpClientResource¶

class vumi.application.sandbox.HttpClientResource(name, app_worker, config)¶

Resource that allows making HTTP calls to outside services.

All command on this resource share a common set of command and response fields:

Command fields:

url: The URL to request
verify_options: A list of options to verify when doing

an HTTPS request. Possible string values are VERIFY_NONE, VERIFY_PEER, VERIFY_CLIENT_ONCE and VERIFY_FAIL_IF_NO_PEER_CERT. Specifying multiple values results in passing along a reduced OR value (e.g. VERIFY_PEER | VERIFY_FAIL_IF_NO_PEER_CERT)
headers: A dictionary of keys for the header name and a list

of values to provide as header values.
data: The payload to submit as part of the request.

files: A dictionary, submitted as multipart/form-data

in the request:

[{
    "field name": {
        "file_name": "the file name",
        "content_type": "content-type",
        "data": "data to submit, encoded as base64",
    }
}, ...]

The data field in the dictionary will be base64 decoded before the HTTP request is made.

Success reply fields:

success: Set to true
body: The response body
code: The HTTP response code

Failure reply fields:

success: set to false
reason: Reason for the failure

Example:

api.request(
    'http.get',
    {url: 'http://foo/'},
    function(reply) { api.log_info(reply.body); });

agent_class¶: alias of Agent

handle_delete(api, command)¶

Make an HTTP DELETE request.