Tornado-sent events

https://travis-ci.org/mivade/tornadose.svg?branch=master

An implementation of the publish/subscribe pattern for the Tornado web server.

Installation

Tornadose is on PyPI:

$ pip install tornadose

This will grab the latest official release. Alternatively, or for development, you can clone the repository and install it manually:

$ git clone https://github.com/mivade/tornadose.git
$ cd tornadose
$ pip install -e .

Usage

A simple example of using server-sent events (a.k.a. EventSource):

import random
from tornado.ioloop import IOLoop, PeriodicCallback
from tornado.web import Application
from tornadose.handlers import EventSource
from tornadose.stores import DataStore

store = DataStore()

app = Application(
    [(r'/', EventSource, {'store': store})],
    debug=True)
app.listen(9000)

loop = IOLoop.instance()
PeriodicCallback(lambda: store.submit(random.random()), 1000).start()
loop.start()

To monitor the stream with curl:

$ curl http://localhost:9000

or with HTTPie:

$ http -S get localhost:9000

Additional demos can be found in the demos directory.

Contributing

Contributions, complaints, criticisms, and whatever else are welcome. The source code and issue tracker can be found on GitHub.

See also

Some other implementations of server-sent events with Tornado include:

License

Tornadose is freely available under the terms of the MIT license. See LICENSE for details.

Contents

Data storage and publishing

In order to publish data to listeners, Tornadose utilizes a data store concept in which subscribers listen to a data store to receive updates.

class tornadose.stores.BaseStore(*args, **kwargs)[source]

Base class for all data store types.

At a minimum, derived classes should implement submit and publish methods.

deregister(subscriber)[source]

Stop publishing to a subscriber.

initialize(*args, **kwargs)[source]

Hook for doing custom initialization. Child classes should implement this method instead of overwriting __init__.

publish()[source]

Push messages to all listeners. This method must be implemented by child classes. A recommended way to implement this method is as a looping coroutine which yields until new data is available via the submit() method.

register(subscriber)[source]

Register a new subscriber. This method should be invoked by listeners to start receiving messages.

submit(message)[source]

Add a new message to be pushed to subscribers. This method must be implemented by child classes.

This method exists to store new data. To actually publish the data, implement the publish method.

class tornadose.stores.DataStore(*args, **kwargs)[source]

Generic object for producing data to feed to clients.

To use this, simply instantiate and update the data property whenever new data is available. When creating a new EventSource handler, specify the DataStore instance so that the EventSource can listen for updates.

class tornadose.stores.QueueStore(*args, **kwargs)[source]

Publish data via queues.

This class is meant to be used in cases where subscribers should not miss any data. Compared to the DataStore class, new messages to be broadcast to clients are put in a queue to be processed in order.

class tornadose.stores.RedisStore(*args, **kwargs)[source]

Publish data via a Redis backend.

This data store works in a similar manner as DataStore. The primary advantage is that external programs can be used to publish data to be consumed by clients.

The channel keyword argument specifies which Redis channel to publish to and defaults to tornadose.

All remaining keyword arguments are passed directly to the redis.StrictRedis constructor. See redis-py‘s documentation for detais.

New messages are read in a background thread via a concurrent.futures.ThreadPoolExecutor. This requires either Python >= 3.2 or the backported futures module to be installed.

Raises:ConnectionError – when the Redis host is not pingable

Request handlers

Tornadose defines handlers for using the EventSource interface or WebSockets. For other handlers, the BaseHandler class is provided.

class tornadose.handlers.BaseHandler(application, request, **kwargs)[source]

Bases: tornado.web.RequestHandler

Base handler for subscribers. To be compatible with data stores defined in tornadose.stores, custom handlers should inherit this class and implement the publish() method.

initialize(store)[source]

Common initialization of handlers happens here. If additional initialization is required, this method must either be called with super or the child class must assign the store attribute and register itself with the store.

publish()[source]

Push a message to the subscriber. This method must be implemented by child classes.

submit(message)[source]

Submit a new message to be published.

class tornadose.handlers.EventSource(application, request, **kwargs)[source]

Bases: tornadose.handlers.BaseHandler

Handler for server-sent events a.k.a. EventSource.

The EventSource interface has a few advantages over websockets:

  • It is a normal HTTP connection and so can be more easily monitored than websockets using tools like curl or HTTPie.
  • Browsers generally try to reestablish a lost connection automatically.
  • The publish/subscribe pattern is better suited to some applications than the full duplex model of websockets.
publish(message)[source]

Pushes data to a listener.

class tornadose.handlers.WebSocketSubscriber(application, request, **kwargs)[source]

Bases: tornadose.handlers.BaseHandler, tornado.websocket.WebSocketHandler

A Websocket-based subscription handler.

open()[source]

Register with the publisher.

publish(message)[source]

Push a new message to the client. The data will be available as a JSON object with the key data.

Changelog

Version 0.5

Upcoming

  • Removed ready event from DataStore.
  • Stopped testing on Python 2.6. As long as Tornado supports Python 2.6, tornadose will most likely still work, but for the ease of testing requirements, official support is now removed.

Version 0.4.1

2016-06-30

  • Point release which includes a source distribution on PyPI. This avoids problems if Tornadose is a requirement listed in a setup.py file since setuptools doesn’t do wheels.

Version 0.4.0

2016-05-28

  • Added a Redis-backed data store. This allows for cross-application publishing since anything can publish to the channel the store is listening to.

Version 0.3.0

2015-12-08

  • Improve performance by always using a Queue for message handling.

Version 0.2.2

2015-10-21

  • Fix bug that printed out all messages sent with websocket subscribers which was originally present for debugging purposes.

Version 0.2.1

2015-10-17

  • Subscription handlers automatically get registered with stores. This simplifies creating custom handlers.

Version 0.2.0

2015-10-11

  • Reworks stores and handlers (backwards incompatible!).
  • Adds a new queue-based QueueStore store.
  • Implements a websocket-based subscriber to supplement EventSource.
  • Begins to add unit testing.

Version 0.1.2

2015-09-20

  • Defines an EventSource request handler and a DataStore object for using server-sent events with Tornado.