MyTardis

Documentation Status Semaphore build status Codacy Badge Coveralls Badge

MyTardis is a multi-institutional collaborative venture that facilitates the archiving and sharing of data and metadata collected at major facilities such as the Australian Synchrotron and ANSTO and within Institutions.

An example of the benefit of a system such as MyTardis in the protein crystallography community is that while the model coordinates and (less often) the structure factors (processed experimental data) are stored in the community Protein Data Bank (PDB) the raw diffraction data is often not available. There are several reasons why this is important, which can be summarised as:

  • The availability of raw data is extremely useful for the development of improved methods of image analysis and data processing.
  • Fostering the archival of raw data at an institutional level is one the best ways of ensuring that this data is not lost (laboratory archives are typically volatile).

Find out more

Project homepage http://mytardis.org

The source code is hosted at https://github.com/mytardis/mytardis

Documentation at http://mytardis.readthedocs.org includes

  • User documentation
  • Administrator documentation
  • Developer documentation

Releases

The default branch on GitHub is develop. This is the cutting edge development version. Please DO NOT use this in production, as it may have bugs that eat your data.

The master branch is the current stable release with all the latest bugfixes included. It will move to newer versions automatically. Follow this branch if you want to stay up to date in a production environment.

Each version has its own branch named by version number. At the time of writing this is 3.7. Follow this branch for your production installation and perform version upgradres manually.

Each bugfix or set of fixes bumps the minor version and each new release is tagged, eg. 3.7.2. Use tagged releases if you are paranoid about changes to the code you have not tested yourself.

To follow development, please see the contributing section below.

Reporting Bugs

Bug reports and feature requests can be made via our public issue tracker.

Contributing

New contributors are always welcome, however all developers should review the pull-request checklist before making pull requests.

For any wishes, comments, praise etc. either open a GitHub issue or contact us.

Active developers are also welcome to join our Slack team.

Contact details can be found on mytardis.org.

Documentation

User Guide

These screenshots refer to an older version. While the workflow has remained largely the same, the look and feel is slightly different.

Browsing & Downloading Data

  1. Click ‘Log In’.
_images/Picture-14.png
  1. Enter your username and password. NOTE: Usually one is able to use their institutional username and password combination.
_images/Picture-15.png
  1. Click ‘Data’ on the top right menu.
_images/Picture-16.png
  1. A list of experiments the user has access to is listed. Click the title of an experiment to view its information.
_images/Picture-17.png
  1. The ‘Description’ view shows basic information about the experiment such as Authors, Title and the total dataset size.
_images/Picture-18.png
  1. Click the ‘Datasets’ tab to view all datasets contained within an experiment.
_images/Picture-19.png
  1. Download all experiment data by clicking ‘Download Entire Experiment’, or click ‘Show’ to view individual file information.
_images/Picture-20.png
  1. Files with special dataset metadata have Show Metadata next to them. Click on this link to view associated metadata.
  2. Click on the red filename link to download individual files.
_images/Picture-21.png
  1. Click on the checkboxes next to files and then the ‘Download Selected Files’ button to download a combination of files.
_images/Picture-22.png

Granting Experiment Access

  1. Click ‘Log In’.
_images/Picture-14.png
  1. Enter your username and password. NOTE: Usually one is able to use their institutional username and password combination.
_images/Picture-15.png
  1. Click ‘Data’ on the top right menu.
_images/Picture-16.png
  1. Under the ‘Controls’ menu on the right, click Control Panel
  2. To control user access to individual experiments, click Show under an experiment.
_images/Picture-24.png
  1. A list of users with access appears. To remove a user, click x
_images/Picture-25.png
  1. To add a user, start typing their name in the Add User box. Click submit when done.
_images/Picture-26.png

NOTE: Those using their insitution’s account (authcate, etc) need to login first before others can give them access to experiments.

NOTE: Users granted access to an experiment can’t grant other users access.

See also:
  • searching
  • tabs

Configuration and Administration

Installation

The sections through to Extended Configuration below provide a Quick Start guide for getting a basic MyTardis installation up and running. The following section provides additional information on advanced configuration and add-on capabilities of MyTardis.

Prerequisites

Ubuntu 14.04

Run the script:

sudo bash install-ubuntu-requirements.sh

It will install required packages with this command:

apt-get update
apt-get install git ipython libldap2-dev libsasl2-dev \
  libssl-dev libxml2-dev libxslt1-dev python-anyjson \
  python-billiard python-bs4 python-crypto python-dateutil \
  python-dev python-flexmock python-html5lib \
  python-pip python-psycopg2 python-pystache \
  python-virtualenv python-wand python-yaml virtualenvwrapper \
  zlib1g-dev libfreetype6-dev libjpeg-dev
Redhat/CentOS 7

Redhat based distributions are not actively supported. Here is a starting point, but you may require additional packages:

sudo yum install epel-release
sudo yum install cyrus-sasl-ldap cyrus-sasl-devel openldap-devel \
libxslt libxslt-devel libxslt-python git gcc graphviz-devel \
python-virtualenv python-virtualenvwrapper python-pip php-devel php-pear \
ImageMagick ImageMagick-devel libevent-devel compat-libevent14-devel

Note that at least one of libevent-devel and compat-libevent14-devel needs to be successfully installed as they are alternatives of the same package for different distributions.

Download

To get the most recent stable release:

git clone -b master https://github.com/mytardis/mytardis.git
cd mytardis

This clones the repository as read-only.

Or, to get the current development branch:

git clone -b develop git://github.com/mytardis/mytardis.git
cd mytardis

Quick configuration

It is recommended that you use a virtualenv. The list of packages above includes the virtualenvwrapper toolkit. Set up your environment with these commands:

Ubuntu 14.04:

source /usr/share/virtualenvwrapper/virtualenvwrapper.sh

Redhat/CentOS 7:

source /usr/bin/virtualenvwrapper.sh

Then create the mytardis virtual environment

mkvirtualenv --system-site-packages mytardis
pip install -U pip

Note: the next time you want to work with this virtualenv, run the appropriate source command and then use the command: workon mytardis

MyTardis dependencies are then installed with pip:

pip install -U -r requirements.txt

For Redhat/CentOS 7, also run:

pip install -U -r requirements-centos.txt

Configuring MyTardis is done through a standard Django settings.py file. MyTardis comes with a sample configuration file at tardis/default_settings.py. You can import this as the basis of your own config file - options defined here will override the relevant options in default_settings.py.

Create a new file tardis/settings.py containing the following:

from default_settings import *

# Add site specific changes here.

# Turn on django debug mode.
DEBUG = True

# Use the built-in SQLite database for testing.
# The database needs to be named something other than "tardis" to avoid
# a conflict with a directory of the same name.
DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'
DATABASES['default']['NAME'] = 'tardis_db'

In addition you will need to create a new SECRET_KEY for your installation. This is important for security reasons.

A convenient method is to run the following command in your mytardis installation location:

python -c "import os; from random import choice; key_line = '%sSECRET_KEY=\"%s\"  # generated from build.sh\n' % ('from tardis.default_settings import * \n\n' if not os.path.isfile('tardis/settings.py') else '', ''.join([choice('abcdefghijklmnopqrstuvwxyz0123456789\\!@#$%^&*(-_=+)') for i in range(50)])); f=open('tardis/settings.py', 'a+'); f.write(key_line); f.close()"

This is the minimum set of changes required to successfully run the server in development mode. You can make any other site-specific changes as necessary.

Initialisation

Create and configure the database:

python mytardis.py migrate

This avoids creating a superuser before the MyTardis specific UserProfile table has been created. More information about the migrate commands can be found at admin.

Next, create a superuser:

python mytardis.py createmysuperuser

MyTardis can now be executed in its simplest form using:

python mytardis.py runserver

This will start the Django web server at http://localhost:8000/.

Extended configuration

See below for some extra configuration options that are specific to MyTardis.

An automatically generated documentation of the settings can be found in pydoc/tardis.

Essential Production Settings

These settings are essential if you want to run MyTardis in production mode (DEBUG = False).

SECRET_KEY

This key needs to be unique per installation and, as the name implies, be kept secret.

A new one can be conveniently generated with the command:

echo "SECRET_KEY='`python mytardis.py generate_secret_key`'" >> tardis/settings.py

However, the more complex command shown above needs to be used at installation time.

ALLOWED_HOSTS

ALLOWED_HOSTS is a list of hostnames and/or IP addresses under which the server is accessible. If this is not set you will get a 500 Error for any request.

Database
tardis.default_settings.DATABASE_ENGINE

The database server engine that will be used to store the MyTardis metadata, possible values are postgresql_psycopg2, postgresql, mysql, sqlite3 or oracle.

tardis.default_settings.DATABASE_NAME

The name of the database to used to store the data, this is the path to the database if you are using the SQLite storage engine.

tardis.default_settings.DATABASE_USER

The user name used to authenticate to the database. If you are using SQLite this field is not used.

tardis.default_settings.DATABASE_PASSWORD

The password used to authenticate to the database. If you are using SQLite this field is not used.

tardis.default_settings.DATABASE_HOST

The host name of the machine hosting the database service. If this is empty then localhost will be used. If you are using SQLite then this field is ignored.

tardis.default_settings.DATABASE_PORT

The port the database is running on. If this is empty then the default port for the database engine will be used. If you are using SQLite then this field is ignored.

LDAP

For further information see LDAP authentication

Repository
tardis.default_settings.DEFAULT_STORAGE_BASE_DIR

The path to the default MyTardis storage location. This is where files will be stored to if you do not provide any other location explicitly through ``StorageBox``es.

tardis.default_settings.STAGING_PATH

The path to the staging path. This is where new files to be included in datasets will be sourced. This functionality, however, is not available in current MyTardis versions.

tardis.default_settings.REQUIRE_DATAFILE_CHECKSUMS

If True, a Datafile requires an MD5 or SHA-512 checksum from the time it is first recorded in the MyTardis database. This enables a model-level constraint check each time a Datafile record is saved. Defaults to True. Datafile record is saved.

tardis.default_settings.REQUIRE_DATAFILE_SIZES

If True, a Datafile require a size from the time it is first recorded in the MyTardis database. This enables a model-level constraint check each time a Datafile record is saved. Defaults to True.

tardis.default_settings.REQUIRE_VALIDATION_ON_INGESTION

If True, ingestion of a Datafile is only permitted if the Datafile matches its supplied size and/or checksums. Defaults to True.

Access Rights & Licensing
Licences

By default, no licences are loaded. A user can make metadata public without assigning a licence, but they cannot allow public access to their data.

Creative Commons licences (for Australia) are available in tardis/tardis_portal/fixtures/cc_licenses.json. You can load them with

python mytardis.py loaddata.

You can use the admin interface to add other licences. Please ensure allows_distribution is set to the correct value to ensure the licence appears in conjunction with suitable public access types.

Filters
tardis.default_settings.POST_SAVE_FILTERS

This contains a list of post save filters that are execute when a new data file is created.

The POST_SAVE_FILTERS variable is specified like:

POST_SAVE_FILTERS = [
    ("tardis.tardis_portal.filters.exif.EXIFFilter", ["EXIF", "http://exif.schema"]),
    ]

For further details please see the Filter Setup section.

Archive Organizations
tardis.default_settings.DEFAULT_ARCHIVE_FORMATS.

This is a prioritized list of download archive formats to be used in contexts where only one choice is offered to the user; e.g. the “download selected” buttons. (The list allows for using different archive formats depending on the user’s platform.)

tardis.default_settings.DEFAULT_PATH_MAPPER.

This gives the default archive “organization” to be used. Organizations are defined via the next attribute.

tardis.default_settings.DOWNLOAD_PATH_MAPPERS.

This is a hash that maps archive organization names to Datafile filename mapper functions. These functions are reponsible for generating the archive pathnames used for files written to “tar” and “zip” archives by the downloads module.

The DOWNLOAD_PATH_MAPPERS variable is specified like:

DOWNLOAD_PATH_MAPPERS = {
    'test': ('tardis.apps.example.ExampleMapper',),
    'test2': ('tardis.apps.example.ExampleMapper', {'foo': 1})
}

The key for each entry is the logical name for the organization, and the value is a tuple consisting of the function’s pathname and a set of optional keyword arguments to be passed to the function. At runtime, the function is called with each Datafile as a positional argument, and an additional ‘rootdir’ keyword argument. The function should compute and return a (unique) pathname based on the Datafile and associated objects. If the function returns None, this tells the archive builder to leave out the file.

By default, the archive builder uses the built-in “deep-storage” mapper which gives pathnames that try to use directory information to rebuild a file tree.

Storage Locations ( StorageBox es)

A MyTardis instance can be configured to support multiple locations ( StorageBox es) for storing data files. Each location holds copies ( DataFileObject s) of DataFile s that are recorded in the MyTardis database.

The StorageBox architecture is compatible with the Django File Storage API. This makes it relatively easy to support a number of different storage backends such as cloud storage or SFTP servers. Please refer to the StorageBox documentation for more detailed information.

Additional Tabs

Additional and custom tabs may be configured in MyTardis on a per-installation basis. The tabs are implemented as separate Django applications with a single view (index), listed in the TARDIS_APPS configuration item and either linked to, or installed in the TARDIS_APP_ROOT directory, by default tardis/apps.

Documentation on the additional tabs is available from tabs.

Additional Views

Custom views may be configured in MyTardis on a per-installation basis. The tabs are implemented as separate Django applications with a single view function listed in the *_VIEWS configuration item and added to the INSTALLED_APPS list.

Refer to the views documentation for further information.

Site Customisations

Some settings that allow customised messages and styles.

PUBLICATION_INTRODUCTION = """
<p><strong>... introduction and publication agreement ...</strong></p>
"""
SITE_STYLES = ''  # should be CSS

# if either GA setting is empty, GA is disabled
GOOGLE_ANALYTICS_ID = ''  # whatever Google provides
GOOGLE_ANALYTICS_HOST = ''  # the host registered with Google

# these refer to any template finder findable location, e.g. APPDIR/templates/...
CUSTOM_ABOUT_SECTION_TEMPLATE = 'tardis_portal/about_include.html'
CUSTOM_USER_GUIDE = 'user_guide/index.html'

Deployment

Collecting Static Files

For performance reasons you should avoid static files being served via the application, and instead serve them directly through the webserver.

To collect all the static files to a single directory:

python mytardis.py collectstatic
tardis.default_settings.STATIC_ROOT

This contains the location to deposit static content for serving.

tardis.default_settings.STATIC_URL

The path static content will be served from. (eg. /static or http://mytardis-resources.example.com/)

Serving with Nginx + Gunicorn

In this configuration, Nginx serves static files and proxies application requests to a Gunicorn server:

   HTTP  +-----------+       +-----------------+
+------->|   Nginx   +------>| Gunicorn Server |
         +-----------+       +-----------------+
           0.0.0.0:80         127.0.0.1:8000

Nginx should then be configured to send requests to the server. Here is an example configuration (SSL part from Mozilla SSL configurator). Please amend for your own needs and understand the settings before deploying it.:

upstream mytardis {
    server unix:/var/run/gunicorn/mytardis/socket;
    server 127.0.0.1:8000 backup;
}
server {
    listen 80 default_server;
    server_name demo.mytardis.org;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 default_server ssl;
    server_name demo.mytardis.org;

    # certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
    ssl_certificate /path/to/signed_cert_plus_intermediates;
    ssl_certificate_key /path/to/private_key;
    ssl_session_timeout 5m;
    ssl_session_cache shared:SSL:50m;

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /path/to/dhparam.pem;

    # intermediate configuration. tweak to your needs.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;

    # HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
    add_header Strict-Transport-Security max-age=15768000;

    # OCSP Stapling ---
    # fetch OCSP records from URL in ssl_certificate and cache them
    ssl_stapling on;
    ssl_stapling_verify on;

    ## verify chain of trust of OCSP response using Root CA and Intermediate certs
    ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates;

    resolver <IP DNS resolver>;

    client_max_body_size 4G;
    keepalive_timeout 5;

    gzip off;  # security reasons
    gzip_proxied any;
    # MyTardis generates uncompressed archives, so compress them in transit
    gzip_types application/x-javascript text/css;
    gzip_min_length 1024;
    gzip_vary on;

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Protocol $scheme;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://mytardis;
        # this is to solve centos 6 error:
        # upstream prematurely closed
        client_max_body_size 4G;
        client_body_buffer_size 8192k;
        proxy_connect_timeout 2000;
        proxy_send_timeout 2000;
        proxy_read_timeout 2000;
    }

    location /static/ {
        expires 7d;
        alias /srv/static_files/;
    }
}

Don’t forget to create the static files directory and give it appropriate permissions. The location is set in the settings.py file.

# Collect static files to ``settings.STATIC_ROOT``
python mytardis.py collectstatic
# Allow Nginx read permissions
setfacl -R -m user:nginx:rx static_dir
Serving with Apache HTTPD + mod_wsgi

We do not support the use of Apache. If you need this and want to support this use case, we welcome your contribution of any relevant documentation.

Authentication Methods

Users

MyTardis supports several sources of authentication and identity, referred to as user providers.

In the settings.py user providers are activated by specifying them within the USER_PROVIDERS variable:

USER_PROVIDERS = ('tardis.tardis_portal.auth.localdb_auth.DjangoUserProvider',)

Groups

MyTardis also supports several sources for group membership information, referred to as group providers.

In the settings.py group providers are activated by specifying them within the GROUP_PROVIDERS variable:

GROUP_PROVIDERS = ('tardis.tardis_portal.auth.localdb_auth.DjangoGroupProvider',
                   'tardis.tardis_portal.auth.vbl_auth.VblGroupProvider',)

Included Auth Plugins

HTTP Basic Endpoint Authentication

tardis.tardis_portal.auth.httpbasicendpoint_auth

HTTP Basic Endpoint authentication uses access to a HTTP resource (or endpoint) to determine if authentication should be allowed.

To use HTTP Basic Endpoint authentication, you’ll need a HTTP resource protected by HTTP Basic Auth which is accessible from the MyTardis server.

In the settings.py add the following to AUTH_PROVIDERS with an appropriate name. eg.

AUTH_PROVIDERS = (
    ('acls', 'acls', 'tardis.tardis_portal.auth.httpbasicendpoint_auth.HttpBasicEndpointAuth'),
)

On each request, MyTardis will attempt to use basic authentication with the provided credentials to access the HTTP resource. If it fails, access is denied.

Endpoint to use in HTTP Basic Endpoint Auth. eg.

HTTP_BASIC_AUTH_ENDPOINT = 'https://test.example/endpoint'
LDAP Authentication

tardis.tardis_portal.auth.ldap_auth

To enable LDAP you’ll need to specify which components of the LDAP authentication backend are enabled. In the settings.py add the following to either Authentication, User Provider, Group Provider slugs.:

'tardis.tardis_portal.auth.ldap_auth.ldap_auth'

This is a wrapper function that allows initialisation of the LDAP provider using settings.py values.

ldap_auth() Function

The following are configuration settings that are used when initialising the LDAP backend.

tardis.default_settings.LDAP_TLS

Enable TLS connections.

tardis.default_settings.LDAP_URL

Set the URL of the LDAP server, e.g. ldap://localhost:389/

tardis.default_settings.LDAP_USER_LOGIN_ATTR

Set the login attribute of the users, usually this will be either cn or uid

tardis.default_settings.LDAP_USER_ATTR_MAP

The LDAP user attribute map is used to map internal identifiers like first_name, last_name and email to their LDAP equivalents e.g. {“givenName”: “first_name”, “sn”: “last_name”, “mail”: “email”}

tardis.default_settings.LDAP_GROUP_ID_ATTR

This is where you specify the group identifier from LDAP, usually it will be cn.

tardis.default_settings.LDAP_GROUP_ATTR_MAP

This map is used to map internal identifiers like display e.g. {“description”: “display”}

tardis.default_settings.LDAP_BASE

Sets the search base of the LDAP queries dc=example, dc=com

tardis.default_settings.LDAP_USER_BASE

Sets the suffix to append to the user RDN (e.g. uid=jsmith) to construct the DN e.g. “ou=People, ” + LDAP_BASE

tardis.default_settings.LDAP_GROUP_BASE

Sets the suffix to append to the group RDN (e.g. cn=group1) to construct the DN e.g. “ou=Group, ” + LDAP_BASE

Temporary Token Authentication

tardis.tardis_portal.auth.token_auth

To use token authentication, you’ll need to specify the following settings.py

‘tardis.tardis_portal.auth.token_auth.TokenGroupProvider’,

TOKEN_EXPIRY_DAYS = 30

TOKEN_LENGTH = 30

TOKEN_USERNAME = ‘tokenuser’

and create a user with

bin/django createtokenuser

Cleaning up

bin/django cleanuptokens

It is recommended that you schedule regular purging of expired tokens. Set a cronjob to run bin/django cleanuptokens

Expiry

Token auth works by hijacking the group provider system.

MyTardis groups are calculated and cached when a user logs in.

This means that if a session is active, and a token becomes in valid (either through deletion or expiry) that access will still be granted. To mitigate this, when a token user logs in, an explicit expiry is set on their session - the earlier of 4am the next day, or the session expiry date (the end of the day)

This forces the user to attempt to log in again, and be denied access.

  • tardis.tardis_portal.auth.ip_auth

The tardis.tardis_portal.auth package module contains the authentication specific code.

Filter Setup

filters

Filters are called once an object has been saved to the database. They build on the Django Signals Infrastructure.

In the settings.py file filters are activated by specifying them within the POST_SAVE_FILTERS variable, for example

POST_SAVE_FILTERS = [
    ("tardis.tardis_portal.filters.exif.EXIFFilter",
     ["EXIF", "http://exif.schema"]),
]

The format they are specified in is

(<filter class path>, [args], {kwargs})

Where args and kwargs are both optional.

Database

Initialising

When creating a new database the syncdb command will need to be called to initialise the schema and insert the initial data fixtures.

Usage

python mytardis.py syncdb --noinput --all

Migrating

Some of the upgrades to MyTardis will require that the database schema be upgraded to match the internal data model. This tool migrates data from old database schemas to the current one. It detects which version of the database you are currently running and will automatically migrate to the current version. If you decided not to use South for migrating the data models and have removed it from the list of INSTALLED_APPS in the settings.py file, you could skip this step and go on initiating an administrator account with the command createmysuperuser , which is described in more detail below.

In certain cases it is also necessary to update the permissions table.

Usage

Intial migration after syncdb --all:

python mytardis.py migrate --fake

Then:

python mytardis.py migrate

If the model changes require it, run:

python mytardis.py update_permissions

creating superuser

After success of database initialization or migration, please use a command line utility called createmysuperuser to create an administrator account using the admin site which is hooked to the URL /admin/.

Usage

python mytardis.py createmysuperuser

Backup

The backupdb command allows to backup and to restore of the MyTardis database. The command uses the corresponding database tool to facilitate this task. Currently implemented are PostgreSQL and MySQL. In case of backup, a directory called backups is created (if it does not exists) in the working directory. In case of restore, the database for storing the tablespace must already exist before loading the backup file into the database.

Usage

python mytardis.py backupdb

-r FILE, --restore=FILE
-v VERBOSITY, --verbosity=VERBOSITY
--settings=SETTINGS
--pythonpath=PYTHONPATH
--traceback
--version
-h, --help

Schema and Parameter Sets

MyTardis stores metadata as Parameters, which are grouped in to Parameter Sets, which are defined by a Schema.

Managing Schema

Schema are managed through the Django administrative interface. The administrative interface is normally accesible from a link similar to:

http://domain.com:8000/admin/

Selecting “Schemas” in the adminstrative interface will display a list of the installed schemas. Clicking on a schema displays the editor for that schema.

Schema definitions are the combination of two tables, Schema and ParameterName.

The Schema fields are:

Namespace

The namespace uniquely identifies the schema. When exporting an experiment as a METS file the namespace is used as an XML Namespace, and thus must follow the XML standard, i.e. in the form of a URL.

The MyTardis naming convention is:

http://domain.com/localidentifiers/schemaname/version
Name
The display name of the schema.
Type
Experiment, Dataset or Datafile.
Subtype
Used to group and identify schema for forms based searching.

The ParameterName fields are:

Schema
The namespace of the schema which this parameter belongs to.
Name
The identifier used to ingest parameters.
Full Name
The display name of the parameter.
Units
The display name of the units for numerical values.
Data Type

One of:

  • Numeric
  • String
  • Longstring
  • URL
  • Filename
  • Datetime
  • Link

Strings use the input field widget for editing, while longstrings use a textarea widget.

Immutable
If true, no user editing of the parameter is allowed, regardless of access.
Comparison Type
The type of search to be performed with Forms Based Search. Not used by Advanced Search.
Is Searchable
Flag whether this parameter is searchable.
Choices
If defined, a drop down list of values is displayed.
Order
The display order of the parameters within the schema. Sorting is by Order, then alphabetically.

Storage Options and Architecture in MyTardis

Database layout for storage

The storage for each DataFile is configured individually. A “way to store data” is called StorageBox. Each file has one or many related DataFileObjects, which link a DataFile with a StorageBox. A DataFile can have several copies stored in different StorageBoxes via several DataFileObjects.

StorageBoxes

StorageBoxes contain all the information needed to store a file except an id unique to the file and storagebox.

Each StorageBox points to a class that implements the Django storage API via a python module path as string.

Optional instatiation parameters for each StorageBox can be stored in StorageBoxOptions. These are used as parameters to the storage class set in the django_storage_class attribute of a StorageBox

These parameters are string types by default. However, by setting the optional parameter value_type to 'pickle', any picklable object can be stored here and hence used for instantiation of the storage class.

Optional classification and other metadata can be stored in StorageBoxAttributes.

A special case is where someone registers a file and wants to put it into location themselves but needs to be given the place to put it (via the API). Such situation can only be resolved with StorageBoxes that implement the “build_save_location” function. Such StorageBoxes need to have a StorageBoxAttribute with key “staging” and value “True”.

DataFiles

DataFiles are the logical representation of the file. They contain information about the name, size, checksum, dates etc.

DataFileObjects

DataFileObjects point to the DataFile they belong to and the StorageBox they reside in. They also have an identifier that the StorageBox uses to find the actual file. DataFileObjects also have a date, and a state-flag.

Available backends

Django storage API compatible backends are available for example at https://github.com/jschneier/django-storages

We have tested the following backends with MyTardis:

  • File on disk or any other system mounted storage
  • SFTP
  • SWIFT Object Storage
Documented backends
S3 compatible storage

S3 and S3 compatible storage backends are supported by the django-storages package.

We added some modifications that allow database driven configuration. Until these changes are accepted by the maintainer, use this fork and branch: https://github.com/monash-merc/django-storages/tree/mytardis-branch

Install it with:

pip install -e git+https://github.com/monash-merc/django-storages.git@mytardis-branch#egg=django-storages
Configuration

The following options can be set as StorageBoxOption on your S3 storage box or system wide in your Django settings (a default of * labels mandatory settings):

StorageBoxOption Django Setting Default
access_key AWS_S3_ACCESS_KEY_ID AWS_ACCESS_KEY_ID *
secret_key AWS_S3_SECRET_ACCESS_KEY AWS_SECRET_ACCESS_KEY *
file_overwrite AWS_S3_FILE_OVERWRITE True
headers AWS_HEADERS {}
bucket_name AWS_STORAGE_BUCKET_NAME *
auto_create_bucket AWS_AUTO_CREATE_BUCKET False
default_acl AWS_DEFAULT_ACL public-read
bucket_acl AWS_BUCKET_ACL default_acl
querystring_auth AWS_QUERYSTRING_AUTH True
querystring_expire AWS_QUERYSTRING_EXPIRE 3600
reduced_redundancy AWS_REDUCED_REDUNDANCY False
location AWS_LOCATION ''
encryption AWS_S3_ENCRYPTION False
custom_domain AWS_S3_CUSTOM_DOMAIN None
calling_format AWS_S3_CALLING_FORMAT SubdomainCallingFormat()
secure_urls AWS_S3_SECURE_URLS True
file_name_charset AWS_S3_FILE_NAME_CHARSET utf-8
gzip AWS_IS_GZIPPED False
preload_metadata AWS_PRELOAD_METADATA False
gzip_content_types GZIP_CONTENT_TYPES ('text/css', 'text/javascript', 'application/javascript', 'application/x-javascript', 'image/svg+xml')
url_protocol AWS_S3_URL_PROTOCOL http:
host AWS_S3_HOST S3Connection.DefaultHost
use_ssl AWS_S3_USE_SSL True
port AWS_S3_PORT None
proxy AWS_S3_PROXY_HOST None
proxy_port AWS_S3_PROXY_PORT None
The max amount of memory a returned file can take up before being rolled over into a temporary file on disk. Default is 0: Do not roll over.
max_memory_size AWS_S3_MAX_MEMORY_SIZE 0

Appendix: Conversion of ‘Replicas’

Replicas used to be the method of file storage abstraction in MyTardis versions 3.x. The StorageBoxes replace this. For pain-free upgrading, a conversion has been included with the database migrations that works as follows:

All ‘Locations’ that are local are converted to default (folder on disk) StorageBoxes. All ‘Locations’ that are not local are converted to dummy ‘link only’ StorageBoxes with the corresponding name. These can be upgraded manually to a StorageBox with an appropriate backend after the migration has taken place.

Max size is set to size of disk, hence for multiple locations on the same disk this number provides no protection. This also should be set to reasonable values manually after the migration.

Each ‘Replica’ of a file will be converted to a DataFileObject pointing to the relevant StorageBox.

All files are manually reverified, so that unverified entries can be checked for errors.

Facility Overview

Introduction

The Facility Overview included with MyTardis allows facility administrators to monitor the output of connected instruments, highlighting unverified files, dataset size, file counts and file listings.

Setup

In order for datasets to appear in the Facility Overview, each dataset must be associated with an Instrument, which is itself associated with a Facility. The Facility object will reference a facility administrator’s group, members of which may view the Facility Overview.

   +-------------------+   +------------------------+   +----------+
   | Facility Managers |-->| Facility Manager Group |-->| Facility |
   +-------------------+   +------------------------+   +----------+
|------------------------------------------------------------|
|
|  +------------+   +----------+   +-----------+
|->| Instrument |-->| Datasets |-->| Datafiles |
   +------------+   +----------+   +-----------+
                          ^
                          |  +-------------+
                          |--| Experiments |
                             +-------------+

The facility managers, facility manager groups, facilities and instruments may be configured via the django admin interface. MyData, the desktop uploader client for the MyTardis server, can be configured to assign the appropriate instrument to uploaded datasets at the point of ingestion.

It should be noted that the dataset visibility within the facility overview is limited to dataset and datafile listings only. Access to the experiment and dataset views, as well as raw data, is still controlled via the ACL framework.

Usage

Members of the facility manager groups for one or more facilities will see the “Facility Overview” menu item in the MyTardis web portal. After opening the facility overview, a list of recently ingested datasets will be displayed from the facility being managed. If a user manages multiple facilities, a blue drop-down selector will also appear on the right-hand side of the page. As the facility overview is designed to give a snapshot of recently uploaded datasets, older data is not immediately accessible; MyTardis’ search feature is better suited to this.

In addition to simply listing the most recent datasets, the datasets can be grouped by instrument or by owner, and filtered by username, experiment name and instrument name. Note that while filters are active, it may appear as though no new pages are loaded by clicking “Load more”, since the additional datasets fetched from the server might not match the active filters.

Development

Architecture

This page describes the architecture of MyTardis.

MyTardis is built on the Django web framework, which itself is built on Python, thus MyTardis follows the architectural model of Django.

Component Architecture

Web Server
MyTardis is typically deployed with the standard Nginx + Gunicorn + Django + Python stack.
RDBMS
Ideally, use PostgreSQL. MySQL or MariaDB may be usable but it is not as well tested.
Data Storage
Local disk, network mounted disks as well as S3 and SFTP are supported.
Ingestion
Ingestion is normally site specific. A desktop application is available (MyData).

Functional Architecture

MyTardis follows the standard Django architecture.

The major functional modules in MyTardis are:

API
A RESTful API provides anonymous and authenticated access to most of the stored data.
Auth
Authentication and Authorisation.
Download
Data download functionality.
Filters
Processing of metadata as data is uploaded, e.g. extract EXIF metadata from JPEG files.
Management
Additional commands for the Django CLI. The backup utility is implemented as a Django command.
Migrations
Database migration code.
Publish
RIF-CS metadata publication.
Search
Provides a searchable and authorisation enabled index for quickly finding data.
SFTP server
Provides read access to data in MyTardis via the SFTP protocol

Information on the individual modules is available from the Module Index.

Scalability Model

At the component / module level, performance and Scalability within MyTardis is achieved by:

  • Allowing long lived or compute intensive operations to be hosted on separate machines from the main web server.
    • E.g. by running multiple web servers and sending long lived operations, such as ingestion, to a server reserved for such operations.
  • Performance and Scalability of the database is achieved by a combination of 1) optimising the SQL requests issued by MyTardis, and 2) database specific scalability, please refer to the appropriate database documentation.
  • Performance and Scalability of the web server is provided through the normal mechanisms for nginx / wsgi hosted python applications, e,g,:
    • Increasing individual Server capability
      • Individual Server performance / utilization may be managed by controlling the number of python (django) processes active at any one time.
    • Deploying multiple web servers
      • Multiple web servers may be deployed using standard mechanisms, e.g. load balancers. State (session) information is distributed using Django’s standard session model.
  • The Data Archive is a normal file system, e.g. NFS, SAMBA, etc., with performance and scalability dependent on the implementation and deployment.
  • Extraction and formatting of metadata for ingestion is up to the client and may be distributed across any number of machines.
SQL Scalability Notes

The datafileparameter table is expected to be the single largest table by an order of magnitude, in the hundreds of millions of records (at the Australian Synchrotron).

For Postgres, this will probably be addressed by table partioning. In this case using a date field to partition is often recommended. Dates may be introduced as part of the support for anotations, raw data, versioned data and derived data.

Persistance and Data Architecture

Django provides an Object-Relational-Model that maps Django Models to the underlying relational database.

The Django Models are defined in tardis.tardis_portal.models.

Security Model

Authentication

Access to data catalogued by MyTardis may be either public, i.e. anonymous access allowed, or private, requiring authentication. The Authentication Framework is documented in ref-authframework.

Authorisation

MyTardis includes an extensible authorisation engine, documented in ref-authframework and authorisation.

Class Diagram

Unless the application has a very rich business model (which MyTardis does not), a class diagram doesn’t convey much information in Django applications (it ends up being a flat list of classes). To understand the context in which the MyTardis classes are created, please refer to the Django documentation, http://docs.djangoproject.com/ on the responsibilities of models, views, managers and templates.

Source Code

This section describes the top level layout of the source code in the GitHub repository.

  • docs
    • MyTardis User and Administrative documentation
  • tardis
    • apps
      • Contains all the optional functionality and installation specific
        functionality. Examples include the optional ANDS Register (ands_register) and Related Info (related_info) tabs, and the ANSTO (mecat-ansto) and Australian Synchrotron (mecat-as) installation specific modules.
      • Some applications are part of the main code base, others can be added
        at installation time.
    • search
      • Search related code
    • tardis_portal
      • This is the main django application that contains most of the core MyTardis functionality

Authorisation Framework

Django Authorisation

Django has a built-in authorisation/permission mechanism that is in use by default. It is enabled in MyTardis in default_settings.py together with the custom object level permission framework described below.

AUTHENTICATION_BACKENDS = (
    'django.contrib.auth.backends.ModelBackend',
    'tardis.tardis_portal.auth.authorisation.ACLAwareBackend',
)

The Django default permissions are automatically available for each Model. The verbs are add, change, delete, and they can be queried on the user object as follows:

user.has_perm('tardis_portal.add_experiment')
user.has_perm('tardis_portal.add_dataset')
user.has_perm('tardis_portal.change_experiment')
user.has_perm('tardis_portal.delete_datasetparameterset')

There is a function in tardis.tardis_portal.auth.authservice called _set_user_from_dict that adds the following permissions for each new user created using custom methods:

'add_experiment'
'change_experiment'
'change_group'
'change_userauthentication'
'change_experimentacl'

These permissions apply in general and are augmented by ACLs

Object Level Permissions and Access Control Lists

The main purpose of the ACL system is to manage per experiment permissions. The architecture allows for future expansion to more find grained permission management. However, at this stage only the Experiment level is supported by the user interface.

Permissions are applied with a few predefined roles:

read
read permission allows individuals and groups access to view an experiment.
write
write permissions cover addition of new datasets and datafiles and also deletion of datafile.
delete
delete permission allows deletion of datasets and experiments.

Roles are applied through the web using the Control Panel and can be applied to either users or groups.

To make an experiment public requires an explicit publish action.

The ACL permissions can be queried on the user object just like standard permissions, however, with the addition of the object in question:

user.has_perm('tardis_acls.change_experiment', experiment)

Verbs currently available are change, view, delete, owns, share.

The translation of ACLs to has_perm verbs is defined in a function in tardis.tardis_portal.auth.authorisation.

To allow for querying on any object related to experiments, extra logic was added to some of the models. To support the logic, in addition to ACLs, has_perm calls model functions named _has_VERB_perm, which allows model-specific permission logic.

The current policy is that if those functions return True or False then that result is returned without further checking. If they return an object, permissions will be checked for this object thereby allowing delegation.

RESTful API for MyTardis

The data and metadata stored in MyTardis instances is made accessible through a RESTful API.

Not all functionality has been exposed via the API at this time. This documentation reflects what is available and tested.

The API version v1 is built on the Tastypie framework.

The RESTful API can also be explored via the automatically generated Swagger documentation at http://mytardis-example.com/api/v1/swagger/.

See swagger.io for details on the Swagger standard.

API accessible models

  • Experiment
  • ExperimentParameterSet
  • ExperimentParameter
  • Dataset
  • DatasetParameterSet
  • DatasetParameter
  • DataFile
  • DatafileParameterSet
  • DatafileParameter
  • StorageBox
  • StorageBoxOption
  • StorageBoxAttribute
  • Schema
  • ParameterName
  • User
  • Group
  • Facility
  • Instrument
  • ObjectACL

Authentication

Currently implemented are Basic Auth, to be used via HTTPS only, and SessionAuth which queries Django sessions.

Due to our desire to provide information to users without any login, eg. for public data, the Basic Auth mechanism is slightly non-standard.

The standard sends an anonymous request, awaits a WWW-Authenticate header, then sends authentication credentials. Instead, this API sends public data for anonymous requests.

Using curl or the requests library this poses no problem. However, using urllib2 or web browser without a Django session is not going to work out of the box.

Here is a snippet (found here: http://stackoverflow.com/questions/4628610/does-urllib2-support-preemptive-authentication-authentication) that makes urllib2 work, should you want to use this library:

class PreemptiveBasicAuthHandler(urllib2.BaseHandler):

        def __init__(self, password_mgr=None):
                if password_mgr is None:
                    password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
                self.passwd = password_mgr
                self.add_password = self.passwd.add_password

        def http_request(self, req):
                uri = req.get_full_url()
                user, pw = self.passwd.find_user_password(None, uri)
                if pw is None:
                    return req

                raw = "%s:%s" % (user, pw)
                auth = 'Basic %s' % base64.b64encode(raw).strip()
                req.add_unredirected_header('Authorization', auth)
                return req

auth_handler = PreemptiveBasicAuthHandler()
auth_handler.add_password(realm=None,
                          uri=url,
                          user='mytardis',
                          passwd='mytardis')
opener = urllib2.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib2.install_opener(opener)

Querying the database (GET)

All endpoints support querying lists and individual records via GET requests. Some support more complex queries via GET parameters as well.

Creating objects, adding files (POST)

The creation of Experiments, Datasets and Dataset_Files via POSTs with the option to include metadata/parametersets has been implemented and tested.

The following examples demonstrate how to go about it.

In all except the file attachment case the POST data should be a JSON string, the Content-Type header needs to be set to application/json and the Accept header as well. Other response formats may be made available in the future.

In all cases the URI of the created object is returned in the Location header of the response.

Experiments

Example JSON input

{
  "title": "API-created Experiment #1",
  "description": "Wow, all automatic!",
  "institution_name": "Monash University",
  "parameter_sets": [
    {
      "schema": "http://institution.com/my/schema",
      "parameters": [
         {
           "name": "important_parameter_1",
           "value": "Test16"
         },
         {
           "name": "important_parameter_3",
           "value": "57.136"
         }
      ]
    },
    {
      "schema": "http://company.com/some/other/schema",
      "parameters": [
         {
           "name": "meaningful_name",
           "value": "Test17"
         },
         {
           "name": "meaningless_name",
           "value": "1234"
         }
      ]
    }
  ]
}

This creates an experiment with two parametersets with two parameters each.

Alternative to Schema namespaces and Parameter names, you can also specify the URIs to each. Until the querying of Schemas and Parameters is documented this is discouraged.

Datasets

Example JSON input:

{
  "description": "API-created Dataset",
  "experiments": [
    "/api/v1/experiment/1/",
    "/api/v1/experiment/2/"
  ],
  "immutable": false,
  "parameter_sets": [
    {
      "parameters": [
        {
          "name": "obscure-instrument-setting-52",
          "value": "awesome dataset api POST"
        },
        {
          "name": "temperature",
          "value": "301"
        }
      ],
      "schema": "http://datasets.com/need/schemas/too"
    },
    {
      "parameters": [
        {
          "name": "someotherparameter",
          "value": "some other value"
        }
      ],
      "schema": "http://better-datasets.com/offers/better/schemas"
    }
  ]
}
DataFiles

There are three ways to add a file to MyTardis via the API.

Via multipart form POST

This works for single files at the moment.

The key is to send a multipart-form instead of ‘application/json’. This can be accomplished with the requests library as shown in the following example.

To use requests you need to install it first, eg. pip install requests.

Also, for this to work, the POST data needs to be sent with the JSON string called 'json_data' and the file called 'attached_file'.

Example JSON input:

{
    "dataset": "/api/v1/dataset/1/",
    "filename": "mytestfile.txt",
    "md5sum": "c858d6319609d6db3c091b09783c479c",
    "size": "12",
    "mimetype": "text/plain",
    "parameter_sets": [{
        "schema": "http://datafileshop.com/fileinfo/v1",
        "parameters": [{
            "name": "fileparameter1",
            "value": "123"
        },
        {
            "name": "fileparameter2",
            "value": "1234"
        }]
    }]
}

Example requests script:

import requests
from requests.auth import HTTPBasicAuth

url = "http://localhost:8000/api/v1/dataset_file/"
headers = {'Accept': 'application/json'}
response = requests.post(url, data={"json_data": data}, headers=headers,
                         files={'attached_file': open(filename, 'rb')},
                         auth=HTTPBasicAuth(username, password)
                         )
Via staging location

Another way to add a file is to create the database entry first without providing a storage location. This will return back a location on the server that you are assumed to have access to. Once the file appears there, for example when you copy it there, it will be moved to its permanent storage location managed by MyTardis.

The full file path that you should copy/move the file to is returned as the content of the response.

Example JSON input:

{
    "dataset": "/api/v1/dataset/1/",
    "filename": "mytestfile.txt",
    "md5sum": "c858d6319609d6db3c091b09783c479c",
    "size": "12",
    "mimetype": "text/plain",
    "parameter_sets": [{
        "schema": "http://datafileshop.com/fileinfo/v1",
        "parameters": [{
            "name": "fileparameter1",
            "value": "123"
        },
        {
            "name": "fileparameter2",
            "value": "1234"
        }]
    }]
}
Via shared permanent storage location

This method assumes that there exists a storage that is shared between MyTardis and you. The registered file will remain in this location.

For this to work you need to get a Location (internal MyTardis settings) name to submit with your metadata.

Examples JSON:

{
   "dataset": "/api/v1/dataset/1/",
   "filename": "mytestfile.txt",
   "md5sum": "c858d6319609d6db3c091b09783c479c",
   "size": "12",
   "mimetype": "text/plain",
   "replicas": [{
       "url": "mytestfile.txt",
       "location": "local",
       "protocol": "file"
   }],
   "parameter_sets": [{
       "schema": "http://datafileshop.com/fileinfo/2",
       "parameters": [{
           "name": "fileparameter1",
           "value": "123"
       },
       {
           "name": "fileparameter2",
           "value": "123"
       }]
   }]
}
urllib2 POST example script

Replace MODEL with one of the available model names in lower case. data is the JSON as a string.

import urllib2
url = "http://localhost:8000/api/v1/MODEL/"
headers = {'Accept': 'application/json',
           'Content-Type': 'application/json'}
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm="django-tastypie",
                          uri=url,
                          user=username,
                          passwd=password)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
myrequest = urllib2.Request(url=url, data=data,
                            headers=headers)
myrequest.get_method = lambda: 'POST'
output = "error"
output = urllib2.urlopen(myrequest)
print output.headers["Location"]

Tests

Running the Test Suite

Run this command to run the unit tests:

./test.py

If you want to speciy any options or specific tests to run, the test argument is required first:

./test.py test –some-argument

You can choose to run the test suite with different options (e.g. with coverage, with different verbosity, etc.). To see the full list of options, run the same command with the –help flag.

Running Individual Unit Tests

The unit tests reside in the tardis/tardis_portal/tests directory. To run the unit tests individually, you can use this command:

./test.py test <test_module_name_here>

Note that the test module name argument should be the relative file path with ”.” as folder separator. For example, if you want to run the test “test_authentication.py”, then your command to execute this test would be:

./test.py test tardis.tardis_portal.tests.test_authentication

Other options

You can choose to include different options when running the unit tests (e.g. run in verbose mode, print out a traceback, etc.). Run the test or django test command with –help flag to see the the full list of options:

./test.py test --help

Running BDD tests

To run BDD tests with the default settings and a headless browser, call:

./test.py behave

To use another browser, e.g. Chrome, follow the steps to install “chromedriver” in http://splinter.readthedocs.io/en/latest/drivers/chrome.html and then call:

./test.py behave -D browser=chrome

tardis

tardis package

Subpackages
tardis.analytics package
Submodules
tardis.analytics.apps module
class tardis.analytics.apps.AnalyticsConfig(app_name, app_module)

Bases: django.apps.config.AppConfig

name = 'tardis.analytics'
verbose_name = 'Analytics framework'
tardis.analytics.ga module

Google analyitics tracking

tardis.analytics.ga.track_download(label, session_id, ip, user, total_size=None, num_files=None, ua=None)
tardis.analytics.ga.track_login(label, session_id, ip, user)
tardis.analytics.ga.track_logout(label, session_id, ip, user)
tardis.analytics.tracker module

Generic tracking and analytics interface Supports Google Analytics through ga.py, others may follow

class tardis.analytics.tracker.IteratorTracker(iterator, tracker_data=None)

Bases: object

wraps file iterator to track successful and incomplete downloads

close()
next()
tardis.analytics.tracker.service = <module 'tardis.analytics.ga' from '/home/docs/checkouts/readthedocs.org/user_builds/mytardis/checkouts/develop/tardis/analytics/ga.pyc'>

This can become a setting to other service in the future

Module contents
tardis.apps package
Subpackages
tardis.apps.anzsrc_codes package
Subpackages
tardis.apps.anzsrc_codes.tests package
Submodules
tardis.apps.anzsrc_codes.tests.test_oaipmh module
class tardis.apps.anzsrc_codes.tests.test_oaipmh.RifCSTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
testExistsInOaipmh()
tardis.apps.anzsrc_codes.tests.tests module
class tardis.apps.anzsrc_codes.tests.tests.TabTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
testAccessWithReadPerms()
testAccessWithoutReadPerms()
Module contents
Submodules
tardis.apps.anzsrc_codes.forms module
class tardis.apps.anzsrc_codes.forms.FoRCodeForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False)

Bases: django.forms.forms.Form

base_fields = OrderedDict([('uri', <django.forms.fields.URLField object>), ('code', <django.forms.fields.CharField object>), ('name', <django.forms.fields.CharField object>)])
declared_fields = OrderedDict([('uri', <django.forms.fields.URLField object>), ('code', <django.forms.fields.CharField object>), ('name', <django.forms.fields.CharField object>)])
media
tardis.apps.anzsrc_codes.models module
tardis.apps.anzsrc_codes.settings module
tardis.apps.anzsrc_codes.urls module
tardis.apps.anzsrc_codes.views module
Module contents
tardis.apps.deep_storage_download_mapper package
Submodules
tardis.apps.deep_storage_download_mapper.mapper module

File mapper that works for files stored in deep directory structures. It recreates the structure as stored in the datafile directory

tardis.apps.deep_storage_download_mapper.mapper.deep_storage_mapper(obj, rootdir=None)

If rootdir is None, just return a filesystem-safe representation of the object, e.g. “DatasetDescription_123” or “strange %2F filename.txt”

For now, only DataFiles are supported when rootdir is not None.

Parameters:
  • obj (DataFile, Dataset or Experiment) – The model instance (DataFile, Dataset or Experiment) to generate a path for.
  • rootdir (basestring) – The top-level directory name, or None
Returns:

Filesystem-safe path for the object in the archive or SFTP view.

Return type:

basestring

Raises:
tardis.apps.deep_storage_download_mapper.urls module
Module contents
tardis.apps.dl_mapper_df_dir_only package
Submodules
tardis.apps.dl_mapper_df_dir_only.mapper module

File mapper that works for files stored in deep directory structures. It recreates the structure as stored in the datafile directory

tardis.apps.dl_mapper_df_dir_only.mapper.df_dir_only(datafile, rootdir)
tardis.apps.dl_mapper_df_dir_only.urls module
Module contents
tardis.apps.equipment package
Subpackages
tardis.apps.equipment.migrations package
Submodules
tardis.apps.equipment.migrations.0001_initial module
class tardis.apps.equipment.migrations.0001_initial.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0002_auto_20150528_1128')]
operations = [<CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key', <django.db.models.fields.CharField>), (u'description', <django.db.models.fields.TextField>), (u'make', <django.db.models.fields.CharField>), (u'model', <django.db.models.fields.CharField>), (u'type', <django.db.models.fields.CharField>), (u'serial', <django.db.models.fields.CharField>), (u'comm', <django.db.models.fields.DateField>), (u'decomm', <django.db.models.fields.DateField>), (u'url', <django.db.models.fields.URLField>), (u'dataset', <django.db.models.fields.related.ManyToManyField>)], name=u'Equipment'>]
Module contents
tardis.apps.equipment.south_migrations package
Submodules
tardis.apps.equipment.south_migrations.0001_initial module
Module contents
Submodules
tardis.apps.equipment.admin module
tardis.apps.equipment.forms module
class tardis.apps.equipment.forms.EquipmentSearchForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False)

Bases: django.forms.forms.Form

base_fields = OrderedDict([('key', <django.forms.fields.CharField object>), ('description', <django.forms.fields.CharField object>), ('make', <django.forms.fields.CharField object>), ('model', <django.forms.fields.CharField object>), ('type', <django.forms.fields.CharField object>), ('serial', <django.forms.fields.CharField object>)])
declared_fields = OrderedDict([('key', <django.forms.fields.CharField object>), ('description', <django.forms.fields.CharField object>), ('make', <django.forms.fields.CharField object>), ('model', <django.forms.fields.CharField object>), ('type', <django.forms.fields.CharField object>), ('serial', <django.forms.fields.CharField object>)])
media
tardis.apps.equipment.models module
class tardis.apps.equipment.models.Equipment(id, key, description, make, model, type, serial, comm, decomm, url)

Bases: django.db.models.base.Model

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Equipment.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Equipment.dataset
Equipment.objects = <django.db.models.manager.Manager object>
tardis.apps.equipment.tests module

tests.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.apps.equipment.tests.EquipmentTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

fixtures = ['initial_data.json']
setUp()
testEquipmentDetail()
testEquipmentList()
testSearchEquipmentForm()
testSearchEquipmentResult()
tardis.apps.equipment.tests.suite()
tardis.apps.equipment.urls module
tardis.apps.equipment.views module
tardis.apps.equipment.views.index(request)
tardis.apps.equipment.views.search(request)
tardis.apps.equipment.views.view_id(request, object_id)
tardis.apps.equipment.views.view_key(request, object_key)
Module contents
tardis.apps.filepicker package
Submodules
tardis.apps.filepicker.filepicker_settings module
tardis.apps.filepicker.urls module
tardis.apps.filepicker.utils module

taken from https://raw.github.com/Filepicker/django-filepicker/master/django_filepicker/utils.py on 11 Apr 2013

class tardis.apps.filepicker.utils.FilepickerFile(url)

Bases: object

cleanup()

Removes any downloaded objects and closes open files.

filepicker_url_regex = <_sre.SRE_Pattern object>
get_file()

Downloads the file from filepicker.io and returns a Django File wrapper object

tardis.apps.filepicker.views module

Filepicker.io button view and upload handler

Module contents
tardis.apps.mx_views package
Submodules
tardis.apps.mx_views.urls module
tardis.apps.mx_views.views module
Module contents
tardis.apps.oaipmh package
Subpackages
tardis.apps.oaipmh.provider package
Submodules
tardis.apps.oaipmh.provider.base module
class tardis.apps.oaipmh.provider.base.BaseProvider(site)

Bases: oaipmh.interfaces.IOAI, object

A base provider which roughly implements the PyOAI interface for OAI-PMH servers.

Extend this if you’re writing your own provider for a new type or a different metadata format.

getRecord(metadataPrefix, identifier)

Get a record for a metadataPrefix and identifier.

Parameters:
  • metadataPrefix (string) – identifies metadata set to retrieve
  • identifier (string) –
    • repository-unique identifier of record
Raises:
  • oaipmh.error.CannotDisseminateFormatError – if metadataPrefix is unknown or not supported by identifier.
  • oaipmh.error.IdDoesNotExistError – if identifier is unknown or illegal.
Returns:

a header, metadata, about tuple describing the record.

identify()
listIdentifiers(metadataPrefix, set=None, from_=None, until=None)

Get a list of header information on records.

Parameters:
  • metadataPrefix (string) – identifies metadata set to retrieve
  • set (string) – set identifier; only return headers in set
  • from (datetime) – only retrieve headers from from_ date forward (in naive UTC)
  • until (datetime) – only retrieve headers with dates up to and including until date (in naive UTC)
Raises:
  • error.CannotDisseminateFormatError – if metadataPrefix is not supported by the repository.
  • error.NoSetHierarchyError – if the repository does not support sets.
Returns:

an iterable of headers.

listMetadataFormats(identifier=None)

List metadata formats supported by repository or record.

Parameters:

identifier (string) – identify record for which we want to know all supported metadata formats. If absent, list all metadata formats supported by repository.

Raises:
  • error.IdDoesNotExistError – if record with identifier does not exist.
  • error.NoMetadataFormatsError – if no formats are available for the indicated record.
Returns:

an iterable of metadataPrefix, schema, metadataNamespace tuples (each entry in the tuple is a string).

listRecords(metadataPrefix, set=None, from_=None, until=None)

Get a list of header, metadata and about information on records.

Parameters:
  • metadataPrefix (string) – identifies metadata set to retrieve
  • set (string) – set identifier; only return records in set
  • from (datetime) – only retrieve records from from_ date forward (in naive UTC)
  • until (datetime) – only retrieve records with dates up to and including until date (in naive UTC)
Raises:
  • oaipmh.error.CannotDisseminateFormatError – if metadataPrefix is not supported by the repository.
  • oaipmh.error.NoSetHierarchyError – if the repository does not support sets.
Returns:

an iterable of header, metadata, about tuples.

listSets()

Get a list of sets in the repository.

Raises:error.NoSetHierarchyError – if the repository does not support sets.
Returns:an iterable of setSpec, setName tuples (strings).
writeMetadata(element, metadata)

Create XML elements under the given element, using the provided metadata.

Should avoid doing any model-lookups, as they should be done when creating the metadata.

Parameters:
  • element (lxml.etree.Element) – element to put all content under (as SubElements)
  • metadata (oaipmh.common.Metadata) – metadata to turn into XML
tardis.apps.oaipmh.provider.experiment module
Module contents
tardis.apps.oaipmh.tests package
Subpackages
tardis.apps.oaipmh.tests.provider package
Submodules
tardis.apps.oaipmh.tests.provider.test_base module
class tardis.apps.oaipmh.tests.provider.test_base.BaseProviderTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testGetRecord()

Default behaviour should be to not handle the identifier.

testIdentify()

There can be only one provider that responds. By default, don’t.

testListIdentifiers()

By default a provider cannot handle the given metadata prefix.

testListMetadataFormats()

By default a provider handles no metadata formats.

testListRecords()

By default a provider cannot handle the given metadata prefix.

testListSets()

By default a provider does not implement sets.

tardis.apps.oaipmh.tests.provider.test_experiment module
Module contents
Submodules
tardis.apps.oaipmh.tests.test_oai module
class tardis.apps.oaipmh.tests.test_oai.EndpointTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testGetRecord()
testIdentify()
testListIdentifiers()
testListMetadataFormats()
testListRecords()
Module contents
Submodules
tardis.apps.oaipmh.server module
class tardis.apps.oaipmh.server.ProxyingMetadataRegistry(providers)

Bases: oaipmh.metadata.MetadataRegistry

A registry that only writes, and does so by proxying to Providers.

hasReader(metadata_prefix)
hasWriter(metadata_prefix)
readMetadata(metadata_prefix, element)
registerReader(metadata_prefix, reader)
registerWriter(metadata_prefix, writer)
writeMetadata(metadata_prefix, element, metadata)

Write metadata as XML.

element - ElementTree element to write under metadata - metadata object to write

class tardis.apps.oaipmh.server.ProxyingServer(providers)

Bases: oaipmh.interfaces.IOAI

getRecord(metadataPrefix, identifier)

Get a record for a metadataPrefix and identifier.

Raises:
  • oaipmh.error.CannotDisseminateFormatError – if no provider returns a result, but at least one provider responds with oaipmh.error.CannotDisseminateFormatError (meaning the identifier exists)
  • oaipmh.error.IdDoesNotExistError – if all providers fail with oaipmh.error.IdDoesNotExistError
Returns:

first successful provider response

Return type:

response

identify()

Retrieve information about the repository.

Returns:an oaipmh.common.Identify object describing the repository.
Return type:oaipmh.common.Identify
listIdentifiers(metadataPrefix, **kwargs)

Lists identifiers from all providers as a single set.

Raises:
  • error.CannotDisseminateFormatError – if metadataPrefix is not supported by the repository.
  • error.NoSetHierarchyError – if a set is provided, as the repository does not support sets.
Returns:

a set.Set of headers.

Return type:

set

listMetadataFormats(**kwargs)

List metadata formats from all providers in a single set.

Raises:
  • error.IdDoesNotExistError – if record with identifier does not exist.
  • error.NoMetadataFormatsError – if no formats are available for the indicated record, but it does exist.
Returns:

a frozenset of metadataPrefix, schema, metadataNamespace tuples (each entry in the tuple is a string).

Return type:

frozenset

listRecords(metadataPrefix, **kwargs)

Lists records from all providers as a single set.

Raises:
  • error.CannotDisseminateFormatError – if metadataPrefix is not supported by the repository.
  • error.NoSetHierarchyError – if a set is provided, as the repository does not support sets.
Returns:

a set.Set of header, metadata, about tuples.

Return type:

set

listSets()

List sets.

Raises:oaipmh.error.NoSetHierarchyError – because set hierarchies are currrently not implemented
tardis.apps.oaipmh.server.get_server(current_site)
tardis.apps.oaipmh.urls module
tardis.apps.oaipmh.views module
tardis.apps.oaipmh.views.endpoint(request)
Module contents
tardis.apps.publication_forms package
Subpackages
tardis.apps.publication_forms.tests package
Submodules
tardis.apps.publication_forms.tests.test_email_templates module
class tardis.apps.publication_forms.tests.test_email_templates.EmailTemplates(methodName='runTest')

Bases: django.test.testcases.TestCase

test_pub_approved_template()
test_pub_awaiting_approval_template()
test_pub_rejected_template()
test_pub_released_template()
test_pub_requires_authorisation_template()
test_pub_reverted_to_draft_template()
Module contents
Submodules
tardis.apps.publication_forms.default_settings module

default settings for publication app override in main settings

tardis.apps.publication_forms.doi module
class tardis.apps.publication_forms.doi.DOI(doi=None)
activate(doi=None)
deactivate(doi=None)
mint(experiment_id, uri, publisher='Monash University')
tardis.apps.publication_forms.email_text module
tardis.apps.publication_forms.email_text.email_pub_approved(pub_title, pub_url, doi=None, message=None)
tardis.apps.publication_forms.email_text.email_pub_awaiting_approval(pub_title)
tardis.apps.publication_forms.email_text.email_pub_rejected(pub_title, message=None)
tardis.apps.publication_forms.email_text.email_pub_released(pub_title, doi=None)
tardis.apps.publication_forms.email_text.email_pub_requires_authorisation(user_name, pub_url, approvals_url)
tardis.apps.publication_forms.email_text.email_pub_reverted_to_draft(pub_title, message=None)
tardis.apps.publication_forms.email_text.interpolate_template(template_name, **kwargs)
tardis.apps.publication_forms.pub_form_config module

Set up publication form schemas

class tardis.apps.publication_forms.pub_form_config.PubFormConfig
do_setup(force=False)
tardis.apps.publication_forms.tasks module
tardis.apps.publication_forms.tasks.get_release_date(publication)
tardis.apps.publication_forms.tasks.has_pdb_embargo(publication)
tardis.apps.publication_forms.tasks.populate_pdb_pub_records(*args, **kwargs)
tardis.apps.publication_forms.tasks.process_embargos(*args, **kwargs)
tardis.apps.publication_forms.urls module
tardis.apps.publication_forms.utils module
class tardis.apps.publication_forms.utils.CifHelper(cif_url)

Bases: object

as_list(obj)
get_cif_file_object()
class tardis.apps.publication_forms.utils.PDBCifHelper(pdb_id)

Bases: tardis.apps.publication_forms.utils.CifHelper

get_citations()
get_free_r_value()
get_obs_r_value()
get_pdb_id()
get_pdb_title()
get_pdb_url()
get_resolution()
get_sequence_info()
get_spacegroup()
get_unit_cell()
tardis.apps.publication_forms.utils.check_pdb_status(pdb_id)
tardis.apps.publication_forms.utils.get_pub_admin_email_addresses()
tardis.apps.publication_forms.utils.get_site_admin_email()
tardis.apps.publication_forms.utils.get_unreleased_pdb_info(pdb_id)
tardis.apps.publication_forms.utils.send_mail_to_authors(publication, subject, message, fail_silently=False)
tardis.apps.publication_forms.views module
tardis.apps.publication_forms.views.approval_ajax(request)
tardis.apps.publication_forms.views.approval_view(request, *args, **kwargs)
tardis.apps.publication_forms.views.approve_publication(*args, **kwargs)
tardis.apps.publication_forms.views.clear_publication_metadata(publication)
tardis.apps.publication_forms.views.create_draft_publication(user, publication_title, publication_description)
tardis.apps.publication_forms.views.fetch_experiments_and_datasets(request, *args, **kwargs)
tardis.apps.publication_forms.views.get_draft_publication(user, publication_id)
tardis.apps.publication_forms.views.get_licenses()
tardis.apps.publication_forms.views.get_publications_awaiting_approval()
tardis.apps.publication_forms.views.index(request, *args, **kwargs)
tardis.apps.publication_forms.views.map_form_to_schemas(extraInfo, publication)
tardis.apps.publication_forms.views.pdb_helper(request, pdb_id)
tardis.apps.publication_forms.views.process_form(request, *args, **kwargs)
tardis.apps.publication_forms.views.reject_publication(publication, message=None)
tardis.apps.publication_forms.views.remove_draft_status(publication)
tardis.apps.publication_forms.views.require_publication_admin(f)
tardis.apps.publication_forms.views.revert_publication_to_draft(publication, message=None)
tardis.apps.publication_forms.views.select_forms(datasets)
tardis.apps.publication_forms.views.set_embargo_release_date(publication, release_date)
tardis.apps.publication_forms.views.set_publication_authors(author_list, publication)
tardis.apps.publication_forms.views.synchrotron_search_epn(publication)
Module contents
tardis.apps.push_to package
Subpackages
tardis.apps.push_to.migrations package
Submodules
tardis.apps.push_to.migrations.0001_initial module
class tardis.apps.push_to.migrations.0001_initial.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'auth', u'__first__'), (u'auth', u'0006_require_contenttypes_0002')]
operations = [<CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key_type', <django.db.models.fields.CharField>), (u'public_key', <django.db.models.fields.TextField>), (u'private_key', <django.db.models.fields.TextField>), (u'remote_user', <django.db.models.fields.CharField>), (u'password', <django.db.models.fields.CharField>)], options={u'abstract': False}, name=u'Credential'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'nickname', <django.db.models.fields.CharField>), (u'oauth_authorize_url', <django.db.models.fields.CharField>), (u'oauth_token_url', <django.db.models.fields.CharField>), (u'oauth_check_token_url', <django.db.models.fields.CharField>), (u'oauth_client_id', <django.db.models.fields.CharField>), (u'oauth_client_secret', <django.db.models.fields.CharField>), (u'cert_signing_url', <django.db.models.fields.CharField>), (u'allow_for_all', <django.db.models.fields.BooleanField>), (u'allowed_groups', <django.db.models.fields.related.ManyToManyField>)], options={u'verbose_name': u'OAuth2 SSH cert signing service', u'verbose_name_plural': u'OAuth2 SSH cert signing services'}, name=u'OAuthSSHCertSigningService'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key_type', <django.db.models.fields.CharField>), (u'public_key', <django.db.models.fields.TextField>), (u'private_key', <django.db.models.fields.TextField>), (u'nickname', <django.db.models.fields.CharField>), (u'logo_img', <django.db.models.fields.CharField>), (u'host_name', <django.db.models.fields.CharField>), (u'port', <django.db.models.fields.IntegerField>), (u'administrator', <django.db.models.fields.related.ForeignKey>)], options={u'abstract': False}, name=u'RemoteHost'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'allowed_remote_hosts', model_name=u'oauthsshcertsigningservice'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'allowed_users', model_name=u'oauthsshcertsigningservice'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'remote_hosts', model_name=u'credential'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'user', model_name=u'credential'>]
tardis.apps.push_to.migrations.0002_auto_20160518_1953 module
class tardis.apps.push_to.migrations.0002_auto_20160518_1953.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'push_to', u'0001_initial')]
operations = [<AlterField field=<django.db.models.fields.CharField>, name=u'key_type', model_name=u'credential'>, <AlterField field=<django.db.models.fields.CharField>, name=u'key_type', model_name=u'remotehost'>]
Module contents
tardis.apps.push_to.tests package
Submodules
tardis.apps.push_to.tests.test_models module
class tardis.apps.push_to.tests.test_models.ModelsTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
test_credential_generation()
test_get_allowed_signing_services()
test_key_type_detection()
test_keys_from_private_key_only()
test_pkey_to_credential()
Module contents
Submodules
tardis.apps.push_to.apps module
class tardis.apps.push_to.apps.PushToConfig(app_name, app_module)

Bases: tardis.app_config.AbstractTardisAppConfig

name = 'tardis.apps.push_to'
verbose_name = 'Push To'
tardis.apps.push_to.exceptions module
exception tardis.apps.push_to.exceptions.NoSuitableCredential

Bases: exceptions.Exception

This exception is thrown when attempting to find a suitable credential for a remote host but none is found

tardis.apps.push_to.models module
class tardis.apps.push_to.models.Credential(*args, **kwargs)

Bases: tardis.apps.push_to.models.KeyPair

A credential that may contain a password and/or key. The auth method chosen depends on the credentials available, allowed auth methods, and priorities defined by the SSH client.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Credential.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

static Credential.generate_keypair_credential(tardis_user, remote_user, remote_hosts, bit_length=2048)

Generates and saves an RSA key pair credential. Credentials returned by this method are intended to be registered on remote systems before being used. @type tardis_user: User @type remote_user: str @type bit_length: int @type remote_hosts: list[RemoteHost] :return: the generated credential :rtype: object

Credential.get_client_for_host(remote_host)

Attempts to establish a connection with the remote_host using this credential object. The remote_host may be any host, but only those in the remote_hosts field are expected to work. @type remote_host: .RemoteHost :return: a connected SSH client :rtype: SSHClient

static Credential.get_suitable_credential(tardis_user, remote_host, remote_user=None)
Credential.objects = <django.db.models.manager.Manager object>
Credential.remote_hosts
Credential.user
Credential.verify_remote_access(remote_host=None)

@type remote_host: RemoteHost

class tardis.apps.push_to.models.CredentialAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

form

alias of CredentialForm

media
class tardis.apps.push_to.models.CredentialForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of Credential

widgets = {'password': <django.forms.widgets.PasswordInput object>}
CredentialForm.base_fields = OrderedDict([('key_type', <django.forms.fields.CharField object>), ('public_key', <django.forms.fields.CharField object>), ('private_key', <django.forms.fields.CharField object>), ('user', <django.forms.models.ModelChoiceField object>), ('remote_hosts', <django.forms.models.ModelMultipleChoiceField object>), ('remote_user', <django.forms.fields.CharField object>), ('password', <django.forms.fields.CharField object>)])
CredentialForm.declared_fields = OrderedDict()
CredentialForm.media
class tardis.apps.push_to.models.DBHostKeyPolicy

Bases: paramiko.client.MissingHostKeyPolicy

Host key verification policy based on the host key stored in the database.

missing_host_key(client, hostname, key)

@type key: PKey

class tardis.apps.push_to.models.KeyPair(*args, **kwargs)

Bases: django.db.models.base.Model

A key pair

class Meta
abstract = False
KeyPair.key
Returns:a subclass of PKey of the appropriate key type
Return type:PKey
Raises:ValidationError
KeyPair.save(*args, **kwargs)
class tardis.apps.push_to.models.OAuthSSHCertSigningService(*args, **kwargs)

Bases: django.db.models.base.Model

Connection parameters for an OAuth2 SSH certificate signing service. Supports certificate signing server available here: https://github.com/monash-merc/ssh-authz

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception OAuthSSHCertSigningService.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

OAuthSSHCertSigningService.allowed_groups
OAuthSSHCertSigningService.allowed_remote_hosts
OAuthSSHCertSigningService.allowed_users
static OAuthSSHCertSigningService.get_available_signing_services(user)

Gets all SSH cert signing services available for a given user :param User user: User :return: allowed signing services :rtype: User

static OAuthSSHCertSigningService.get_oauth_service(user, service_id)

@type user: User @type service_id: int

OAuthSSHCertSigningService.objects = <django.db.models.manager.Manager object>
class tardis.apps.push_to.models.RemoteHost(*args, **kwargs)

Bases: tardis.apps.push_to.models.KeyPair

A remote host that may be connected to via SSH

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception RemoteHost.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

RemoteHost.administrator
RemoteHost.credential_set
RemoteHost.oauthsshcertsigningservice_set
RemoteHost.objects = <django.db.models.manager.Manager object>
class tardis.apps.push_to.models.RemoteHostAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

Hides the private key field, which is not necessary for host keys

fields = ['nickname', 'administrator', 'host_name', 'port', 'key_type', 'public_key', 'logo_img']
media
tardis.apps.push_to.oauth_tokens module
tardis.apps.push_to.oauth_tokens.get_token(request, oauth_service)

Returns the OAuth2 token from the current session :param Request request: django session object :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Returns:
Return type:string
tardis.apps.push_to.oauth_tokens.get_token_data(oauth_service, token)

Gets the OAuth2 user attributes using the supplied token :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Parameters:token (basestring) – an OAuth2 token
Returns:a json object of user attributes
Return type:dict
tardis.apps.push_to.oauth_tokens.set_token(request, oauth_service, token)

Stores the OAuth2 token in the current session :param Request request: django request object :param OAuthSSHCertSigningService oauth_service: an

OAuthSSHCertSigningService object
Parameters:token (basestring) – the OAuth2 token
tardis.apps.push_to.ssh_authz module
tardis.apps.push_to.ssh_authz.sign_certificate(credential, token, url)

An interface to the OAuth2 SSH certificate signing service @type credential: models.Credential

tardis.apps.push_to.tasks module
tardis.apps.push_to.urls module
tardis.apps.push_to.utils module
tardis.apps.push_to.utils.bytes_available(ssh_client, path)
tardis.apps.push_to.utils.can_copy(ssh_client, object_type, object_id, path)
tardis.apps.push_to.utils.get_default_push_location(sftp_client)
tardis.apps.push_to.utils.get_object_size(type, id)
tardis.apps.push_to.utils.is_directory(sftp_client, path)
tardis.apps.push_to.utils.list_subdirectories(sftp_client, path, show_hidden=False)
tardis.apps.push_to.utils.shell_escape(s)
tardis.apps.push_to.views module
Module contents
tardis.apps.slideshow_view package
Submodules
tardis.apps.slideshow_view.urls module
tardis.apps.slideshow_view.views module
Module contents
Module contents
tardis.default_settings package
Submodules
tardis.default_settings.admins module
tardis.default_settings.analytics module
tardis.default_settings.apps module
tardis.default_settings.auth module
tardis.default_settings.auth.AUTOGENERATE_API_KEY = False

Generate a tastypie API key with user post_save (tardis/tardis_portal/models/hooks.py)

tardis.default_settings.auth.REGISTRATION_OPEN = True

Enable/disable the self-registration link and form in the UI. Note - this does not actually disable the URL endpoints for registration. You must also remove the registration app from INSTALLED_APPS to disable registration.

tardis.default_settings.caches module
tardis.default_settings.caches.CACHES = {'default': {'LOCATION': 'default_cache', 'BACKEND': 'django.core.cache.backends.db.DatabaseCache'}, 'celery-locks': {'LOCATION': 'celery_lock_cache', 'BACKEND': 'django.core.cache.backends.db.DatabaseCache'}}

change the CACHES setting to memcached if you prefer. Requires additional dependencies.

tardis.default_settings.celery module
tardis.default_settings.celery.BROKER_URL = 'django://'

use django:, add kombu.transport.django to INSTALLED_APPS or use redis: install redis separately and add the following to a custom buildout.cfg:

django-celery-with-redis redis hiredis
tardis.default_settings.custom_views module
tardis.default_settings.custom_views.DATASET_VIEWS = []

Dataset view overrides (‘contextual views’) are specified as tuples mapping a Schema namespace to a class-based view (or view function). See: https://mytardis.readthedocs.org/en/develop/contextual_views.html#dataset-and-experiment-views

eg:

DATASET_VIEWS = [
    ('http://example.org/schemas/dataset/my_awesome_schema',
     'tardis.apps.my_awesome_app.views.CustomDatasetViewSubclass'),
]
tardis.default_settings.custom_views.EXPERIMENT_VIEWS = []

Experiment view overrides (‘contextual views’) are specified as tuples mapping a Schema namespace to a class-based view (or view function). See: https://mytardis.readthedocs.org/en/develop/contextual_views.html#dataset-and-experiment-views

eg:

EXPERIMENT_VIEWS = [
    ('http://example.org/schemas/expt/my_awesome_schema',
     'tardis.apps.my_awesome_app.views.CustomExptViewSubclass'),
]
tardis.default_settings.custom_views.INDEX_VIEWS = {}

A custom index page override is defined in as dictionary mapping a class-based view (or view function) to a Django Site, specified by SITE_ID (an integer) or the domain name of the incoming request. See: https://mytardis.readthedocs.org/en/develop/contextual_views.html#custom-index-view

eg:

INDEX_VIEWS = {
    1: 'tardis.apps.my_custom_app.views.MyCustomIndexSubclass',
    'store.example.com': 'tardis.apps.myapp.AnotherCustomIndexSubclass'
}
tardis.default_settings.database module
tardis.default_settings.debug module
tardis.default_settings.debug.ALLOWED_HOSTS = ['*']

For security reasons this needs to be set to your hostname and/or IP address in production.

tardis.default_settings.debug.DEBUG = True

Set to false for production use

tardis.default_settings.downloads module
tardis.default_settings.downloads.DEFAULT_ARCHIVE_FORMATS = ['tar']

Site’s preferred archive types, with the most preferred first other available option: ‘tgz’. Add to list if desired

tardis.default_settings.email module
tardis.default_settings.filters module
tardis.default_settings.frontend module
tardis.default_settings.frontend.BLEACH_ALLOWED_ATTRIBUTES = {'a': ['href', 'title'], 'acronym': ['title'], 'abbr': ['title']}

These are the default bleach values and shown here as an example.

tardis.default_settings.frontend.BLEACH_ALLOWED_TAGS = ['a', 'abbr', 'acronym', 'b', 'blockquote', 'code', 'em', 'i', 'li', 'ol', 'strong', 'ul']

These are the default bleach values and shown here as an example.

tardis.default_settings.i18n module
tardis.default_settings.localisation module
tardis.default_settings.logging module
tardis.default_settings.middlewares module
tardis.default_settings.publication module
tardis.default_settings.search module
tardis.default_settings.sftp module
tardis.default_settings.sftp.SFTP_HOST_KEY = '-----BEGIN RSA PRIVATE KEY-----\nMIICXgIBAAKCAIEAl7sAF0x2O/HwLhG68b1uG8KHSOTqe3Cdlj5i/1RhO7E2BJ4B\n3jhKYDYtupRnMFbpu7fb21A24w3Y3W5gXzywBxR6dP2HgiSDVecoDg2uSYPjnlDk\nHrRuviSBG3XpJ/awn1DObxRIvJP4/sCqcMY8Ro/3qfmid5WmMpdCZ3EBeC0CAwEA\nAQKCAIBSGefUs5UOnr190C49/GiGMN6PPP78SFWdJKjgzEHI0P0PxofwPLlSEj7w\nRLkJWR4kazpWE7N/bNC6EK2pGueMN9Ag2GxdIRC5r1y8pdYbAkuFFwq9Tqa6j5B0\nGkkwEhrcFNBGx8UfzHESXe/uE16F+e8l6xBMcXLMJVo9Xjui6QJBAL9MsJEx93iO\nzwjoRpSNzWyZFhiHbcGJ0NahWzc3wASRU6L9M3JZ1VkabRuWwKNuEzEHNK8cLbRl\nTyH0mceWXcsCQQDLDEuWcOeoDteEpNhVJFkXJJfwZ4Rlxu42MDsQQ/paJCjt2ONU\nWBn/P6iYDTvxrt/8+CtLfYc+QQkrTnKn3cLnAkEAk3ixXR0h46Rj4j/9uSOfyyow\nqHQunlZ50hvNz8GAm4TU7v82m96449nFZtFObC69SLx/VsboTPsUh96idgRrBQJA\nQBfGeFt1VGAy+YTLYLzTfnGnoFQcv7+2i9ZXnn/Gs9N8M+/lekdBFYgzoKN0y4pG\n2+Q+Tlr2aNlAmrHtkT13+wJAJVgZATPI5X3UO0Wdf24f/w9+OY+QxKGl86tTQXzE\n4bwvYtUGufMIHiNeWP66i6fYCucXCMYtx6Xgu2hpdZZpFw==\n-----END RSA PRIVATE KEY-----\n'

public, useless key, debugging use only

tardis.default_settings.sftp.SFTP_USERNAME_ATTRIBUTE = 'email'

The attribute from the User model (‘email’ or ‘username’) used to generate the SFTP login example on the sftp_access help page.

tardis.default_settings.sharing module
tardis.default_settings.site_customisations module
tardis.default_settings.site_customisations.SITE_TITLE = 'MyTardis'

customise the title of your site

tardis.default_settings.site_customisations.SPONSORED_TEXT = None

add text to the footer to acknowledge someone

tardis.default_settings.staging module
tardis.default_settings.static_files module
tardis.default_settings.static_files.get_admin_media_path()
tardis.default_settings.storage module
tardis.default_settings.storage.METADATA_STORE_PATH = '/home/docs/checkouts/readthedocs.org/user_builds/mytardis/checkouts/develop/var/store'

storage path for image paths stored in parameters. Better to set to another location if possible

tardis.default_settings.templates module
tardis.default_settings.uploads module
tardis.default_settings.uploads.UPLOAD_METHOD = False

Old version: UPLOAD_METHOD = “uploadify”. This can be changed to an app that provides an upload_button function, eg. “tardis.apps.filepicker.views.upload_button” to use a fancy commercial uploader. To use filepicker, please also get an API key at http://filepicker.io

tardis.default_settings.urls module
Module contents
tardis.default_settings.get_git_version()
tardis.search package
Submodules
tardis.search.forms module
class tardis.search.forms.GroupedSearchForm(*args, **kwargs)

Bases: haystack.forms.ModelSearchForm

base_fields = OrderedDict([('q', <django.forms.fields.CharField object>)])
declared_fields = OrderedDict([('q', <django.forms.fields.CharField object>)])
media
search(*args, **kwargs)
tardis.search.search_indexes module

search indexes for single search

class tardis.search.search_indexes.DataFileIndex

Bases: haystack.indexes.SearchIndex, haystack.constants.Indexable

datafile_filename = <haystack.fields.CharField object>
dataset_id_stored = <haystack.fields.IntegerField object>
experiment_id_stored = <haystack.fields.MultiValueField object>
fields = {'text': <haystack.fields.CharField object>, 'dataset_id_stored': <haystack.fields.IntegerField object>, 'experiment_id_stored': <haystack.fields.MultiValueField object>, 'datafile_filename': <haystack.fields.CharField object>}
get_model()
objects = <haystack.manager.SearchIndexManager object>
prepare_experiment_id_stored(obj)
prepare_text(obj)
text = <haystack.fields.CharField object>
class tardis.search.search_indexes.DatasetIndex

Bases: haystack.indexes.SearchIndex, haystack.constants.Indexable

dataset_description = <haystack.fields.CharField object>
dataset_id_stored = <haystack.fields.IntegerField object>
experiment_id_stored = <haystack.fields.MultiValueField object>
fields = {'dataset_description': <haystack.fields.CharField object>, 'text': <haystack.fields.CharField object>, 'dataset_id_stored': <haystack.fields.IntegerField object>, 'experiment_id_stored': <haystack.fields.MultiValueField object>}
get_model()
objects = <haystack.manager.SearchIndexManager object>
prepare_experiment_id_stored(obj)
prepare_text(obj)
text = <haystack.fields.CharField object>
class tardis.search.search_indexes.ExperimentIndex

Bases: haystack.indexes.SearchIndex, haystack.constants.Indexable

experiment_author = <haystack.fields.MultiValueField object>
experiment_created_time = <haystack.fields.DateTimeField object>
experiment_creator = <haystack.fields.CharField object>
experiment_description = <haystack.fields.CharField object>
experiment_end_time = <haystack.fields.DateTimeField object>
experiment_id_stored = <haystack.fields.IntegerField object>
experiment_institution_name = <haystack.fields.CharField object>
experiment_start_time = <haystack.fields.DateTimeField object>
experiment_title = <haystack.fields.CharField object>
experiment_update_time = <haystack.fields.DateTimeField object>
fields = {'experiment_update_time': <haystack.fields.DateTimeField object>, 'experiment_description': <haystack.fields.CharField object>, 'text': <haystack.fields.CharField object>, 'experiment_id_stored': <haystack.fields.IntegerField object>, 'experiment_created_time': <haystack.fields.DateTimeField object>, 'experiment_start_time': <haystack.fields.DateTimeField object>, 'experiment_institution_name': <haystack.fields.CharField object>, 'experiment_creator': <haystack.fields.CharField object>, 'experiment_end_time': <haystack.fields.DateTimeField object>, 'experiment_author': <haystack.fields.MultiValueField object>, 'experiment_title': <haystack.fields.CharField object>}
get_model()
index_queryset(using=None)

Used when the entire index for model is updated.

objects = <haystack.manager.SearchIndexManager object>
prepare_experimentauthor(obj)
prepare_text(obj)
text = <haystack.fields.CharField object>
tardis.search.urls module
tardis.search.utils module
class tardis.search.utils.SearchQueryString(query_string)

Bases: object

Class to manage switching between space separated search queries and ‘+’ separated search queries (for addition to urls

TODO This would probably be better handled with filters

query_string()
url_safe_query()
tardis.search.views module
Module contents
tardis.tardis_portal package
Subpackages
tardis.tardis_portal.auth package
Submodules
tardis.tardis_portal.auth.authentication module

A module containing helper methods for the manage_auth_methods function in views.py.

tardis.tardis_portal.auth.authentication.add_auth_method(request)

Add a new authentication method to request.user’s existing list of authentication methods. This method will ask for a confirmation if the user wants to merge two accounts if the authentication method he provided already exists as a method for another user.

Parameters:request (Request) – the HTTP request object
Returns:The HttpResponse which contains request.user’s new list of authentication methods
Return type:HttpResponse
tardis.tardis_portal.auth.authentication.edit_auth_method(request)

Change the local DB (Django) password for request.user.

tardis.tardis_portal.auth.authentication.list_auth_methods(request)

Generate a list of authentication methods that request.user uses to authenticate to the system and send it back in a HttpResponse.

Parameters:request (Request) – the HTTP request object
Returns:The HttpResponse which contains request.user’s list of authentication methods
Return type:HttpResponse
tardis.tardis_portal.auth.authentication.merge_auth_method(request)

Merge the account that the user is logged in as and the account that he provided in the Authentication Form. Merging accounts involve relinking the UserAuthentication table entries, transferring ObjectACL entries to the merged account, changing the Group memberships and deleting the unneeded account.

Parameters:request (Request) – the HTTP request object
Returns:The HttpResponse which contains request.user’s new list of authentication methods
Return type:HttpResponse
tardis.tardis_portal.auth.authentication.remove_auth_method(request)

Removes the non-local DB auth method from the UserAuthentication model.

Parameters:request (Request) – the HTTP request object
Returns:The HttpResponse which contains request.user’s new list of authentication methods
Return type:HttpResponse
tardis.tardis_portal.auth.authorisation module

Object-level authorisation backend

class tardis.tardis_portal.auth.authorisation.ACLAwareBackend

Bases: object

app_label = 'tardis_acls'
authenticate(username, password)

do not use this backend for authentication

get_perm_bool(verb)

relates ACLs to permissions

has_perm(user_obj, perm, obj=None)

main method, calls other methods based on permission type queried

supports_anonymous_user = True
supports_object_permissions = True
tardis.tardis_portal.auth.authservice module

models.py

class tardis.tardis_portal.auth.authservice.AuthService(settings=<django.conf.LazySettings object>)

The AuthService provides an interface for querying the auth(n|z) framework within MyTardis. The auth service works by reading the class path to plugins from the settings file.

Parameters:settings (django.conf.settings) – the settings object that contains the list of user and group plugins.
authenticate(authMethod, **credentials)

Try and authenticate the user using the auth type he/she specified to use and if authentication didn’t work using that

Parameters:
  • authMethod (string) – the shortname of the auth method.
  • credentials (kwargs) – the credentials as expected by the auth plugin
Returns:

authenticated User or None

Return type:

User or None

getGroups(user)
Parameters:user (User) – User
Returns:a list of tuples containing pluginname and group id
Return type:list
getGroupsForEntity(entity)

Return a list of the groups an entity belongs to

Parameters:entity (string) – the entity to earch for, user or group.
Returns:groups
Return type:Group

The groups will be reurned as a list similar to:

[{'name': 'Group 456', 'id': '2'},
{'name': 'Group 123', 'id': '1'}]
getUser(authMethod, user_id, force_user_create=False)

Return a user model based on the given auth method and user id.

This function is responsible for creating the user within the Django DB and returning the resulting user model.

getUsernameByEmail(authMethod, email)

Return a username given the auth method and email address of a user.

get_or_create_user(user_obj_or_dict, authMethod=None)

refactored out for external use by AAF and possibly others

searchEntities(filter)

Return a list of users and/or groups

searchGroups(**kw)

basestring id: the value of the id to search for basestring name: the value of the displayname to search for int max_results: the maximum number of elements to return basestring sort_by: the attribute the users should be sorted on basestring plugin: restrict the search to the specific group

provider

returns a list of users and/or groups rtype list

searchUsers(filter)

Return a list of users and/or groups

tardis.tardis_portal.auth.decorators module
tardis.tardis_portal.auth.decorators.datafile_access_required(f)
tardis.tardis_portal.auth.decorators.dataset_access_required(f)
tardis.tardis_portal.auth.decorators.dataset_download_required(f)
tardis.tardis_portal.auth.decorators.dataset_write_permissions_required(f)
tardis.tardis_portal.auth.decorators.delete_permissions_required(f)
tardis.tardis_portal.auth.decorators.experiment_access_required(f)
tardis.tardis_portal.auth.decorators.experiment_download_required(f)
tardis.tardis_portal.auth.decorators.experiment_ownership_required(f)

A decorator for Django views that validates if a user is an owner of an experiment or ‘superuser’ prior to further processing the request. Unauthenticated requests are redirected to the login page. If the user making the request satisfies none of these criteria, an error response is returned.

Parameters:f (types.FunctionType) – A Django view function
Returns:A Django view function
Return type:types.FunctionType
tardis.tardis_portal.auth.decorators.get_accessible_datafiles_for_user(request)
tardis.tardis_portal.auth.decorators.get_accessible_experiments(request)
tardis.tardis_portal.auth.decorators.get_accessible_experiments_for_dataset(request, dataset_id)
tardis.tardis_portal.auth.decorators.get_owned_experiments(request)
tardis.tardis_portal.auth.decorators.get_shared_experiments(request)
tardis.tardis_portal.auth.decorators.group_ownership_required(f)

A decorator for Django views that validates if a user is a group admin or ‘superuser’ prior to further processing the request. Unauthenticated requests are redirected to the login page. If the user making the request satisfies none of these criteria, an error response is returned.

Parameters:f (types.FunctionType) – A Django view function
Returns:A Django view function
Return type:types.FunctionType
tardis.tardis_portal.auth.decorators.has_datafile_access(request, datafile_id)
tardis.tardis_portal.auth.decorators.has_datafile_download_access(request, datafile_id)
tardis.tardis_portal.auth.decorators.has_dataset_access(request, dataset_id)
tardis.tardis_portal.auth.decorators.has_dataset_download_access(request, dataset_id)
tardis.tardis_portal.auth.decorators.has_dataset_ownership(request, dataset_id)
tardis.tardis_portal.auth.decorators.has_dataset_write(request, dataset_id)
tardis.tardis_portal.auth.decorators.has_delete_permissions(request, experiment_id)
tardis.tardis_portal.auth.decorators.has_experiment_access(request, experiment_id)
tardis.tardis_portal.auth.decorators.has_experiment_download_access(request, experiment_id)
tardis.tardis_portal.auth.decorators.has_experiment_ownership(request, experiment_id)
tardis.tardis_portal.auth.decorators.has_experiment_write(request, experiment_id)
tardis.tardis_portal.auth.decorators.has_read_or_owner_ACL(request, experiment_id)

Check whether the user has read access to the experiment - this means either they have been granted read access, or that they are the owner.

NOTE: This does not check whether the experiment is public or not, which means even when the experiment is public, this method does not automatically returns true.

As such, this method should NOT be used to check whether the user has general read permission.

tardis.tardis_portal.auth.decorators.has_write_permissions(request, experiment_id)
tardis.tardis_portal.auth.decorators.is_group_admin(request, *args, **kwargs)
tardis.tardis_portal.auth.decorators.upload_auth(f)
tardis.tardis_portal.auth.decorators.write_permissions_required(f)
tardis.tardis_portal.auth.fix_circular module
tardis.tardis_portal.auth.fix_circular.getGroups(user)
tardis.tardis_portal.auth.httpbasicendpoint_auth module

Created on Dec 15, 2011

@author: uqtdettr

class tardis.tardis_portal.auth.httpbasicendpoint_auth.HttpBasicEndpointAuth(openerDirector=<urllib2.OpenerDirector instance>, endpoint=None)

Bases: tardis.tardis_portal.auth.interfaces.AuthProvider

This class provides authentication against a HTTP resource protected by HTTP Basic authentication. Access is granted based on the user credentials being valid against that resource.

class SimplePasswordMgr

Bases: urllib2.HTTPPasswordMgr

Simple password manager which provides the same credentials, no matter the realm or the uri.

add_password(realm, uri, username, password)
clear()
find_user_password(realm, authuri)
HttpBasicEndpointAuth.authenticate(request)

Authenticate a user, expecting the user will be using form-based auth and the username and password will be passed in url-encoded form POST variables.

Parameters:request (django.http.HttpRequest) – a HTTP Request instance
Returns:authenticated user
Return type:User
HttpBasicEndpointAuth.get_user(user_id)
tardis.tardis_portal.auth.interfaces module
class tardis.tardis_portal.auth.interfaces.AuthProvider
authenticate(request)

from a request authenticate try to authenticate the user. return a user dict if successful.

getUsernameByEmail(email)

returns the username (format string) from the auth domain

Implementing this function is optional- it is needed for resolving experiment owner email addresses to usernames during ingestion.

get_user(user_id)
class tardis.tardis_portal.auth.interfaces.GroupProvider
getGroupById(id)

return the group associated with the id

getGroups(user)

return an iteration of the available groups.

getGroupsForEntity(id)

return a list of groups associated with a particular entity id

searchGroups(**filter)

return a list of groups that match the filter

class tardis.tardis_portal.auth.interfaces.UserProvider
getUserById(id)

return the user dictionary in the format of:

{"id": 123,
"first_name": "John",
"last_name": "Smith",
"email": "john@example.com"}
getUsernameByEmail(email)

returns the username (format string) from the auth domain needed for resolving experiment owners during ingestion

searchUsers(**filter)

return a list of user descriptions from the auth domain.

each user is in the format of:

{"id": 123,
"first_name": "John",
"last_name": "Smith",
"email": "john@example.com"}
tardis.tardis_portal.auth.ldap_auth module
tardis.tardis_portal.auth.localdb_auth module

Local DB Authentication module.

class tardis.tardis_portal.auth.localdb_auth.DjangoAuthBackend

Bases: tardis.tardis_portal.auth.interfaces.AuthProvider

Authenticate against Django’s Model Backend.

authenticate(request)

authenticate a user, this expect the user will be using form based auth and the username and password will be passed in as POST variables.

Parameters:request (django.http.HttpRequest) – a HTTP Request instance
Returns:authenticated User
Return type:User
get_user(user_id)
class tardis.tardis_portal.auth.localdb_auth.DjangoGroupProvider

Bases: tardis.tardis_portal.auth.interfaces.GroupProvider

getGroupById(id)

return the group associated with the id:

{"id": 123,

“display”: “Group Name”,}

getGroups(user)

return an iteration of the available groups.

name = u'django_group'
searchGroups(**filter)
class tardis.tardis_portal.auth.localdb_auth.DjangoUserProvider

Bases: tardis.tardis_portal.auth.interfaces.UserProvider

getUserById(id)

return the user dictionary in the format of:

{"id": 123,
"first_name": "John",
"last_name": "Smith",
"email": "john@example.com"}
name = u'django_user'
tardis.tardis_portal.auth.token_auth module

token authentication module

class tardis.tardis_portal.auth.token_auth.TokenAuthMiddleware

Bases: object

adds tokens to the user object and the session from a GET query

process_request(request)
class tardis.tardis_portal.auth.token_auth.TokenGroupProvider

Bases: tardis.tardis_portal.auth.interfaces.GroupProvider

Transforms tokens into auth groups

getGroups(user)
name = u'token_group'
searchGroups(**kwargs)

return nothing because these are not groups in the standard sense

tardis.tardis_portal.auth.utils module

Created on 15/03/2011

@author: gerson

tardis.tardis_portal.auth.utils.configure_user(user)

Configure a user account that has just been created by adding the user to the default groups and creating a UserProfile.

Parameters:user (User) – the User instance for the newly created account
Returns:User profile for user
Return type:UserProfile
tardis.tardis_portal.auth.utils.create_user(auth_method, user_id, email='')
tardis.tardis_portal.auth.utils.get_or_create_user(auth_method, user_id, email='')
Module contents
tardis.tardis_portal.filters package
Submodules
tardis.tardis_portal.filters.diffractionimage module

diffractionimage.py

class tardis.tardis_portal.filters.diffractionimage.DiffractionImageFilter(name, schema, diffdump_path, diff2jpeg_path, tagsToFind=[], tagsToExclude=[])

Bases: object

This filter runs the CCP4 diffdump binary on a diffraction image and collects its output into the trddatafile schema

If a white list is specified then it takes precidence and all other tags will be ignored.

param name: the short name of the schema. type name: string param schema: the name of the schema to load the EXIF data into. type schema: string param tagsToFind: a list of the tags to include. type tagsToFind: list of strings param tagsToExclude: a list of the tags to exclude. type tagsToExclude: list of strings

getDiffractionImageMetadata(filename)

Return a dictionary of the metadata.

getDiffractionPreviewImage(filename)

Return a base64 encoded preview image.

getParameters(schema, metadata)

Return a list of the paramaters that will be saved.

getSchema()

Return the schema object that the paramaterset will use.

output_beamcenter(term, value)
output_detectordistance(term, value)
output_exposuretime(term, value)
output_imagesize(term, value)
output_metadata(term, value)
output_oscillation(term, value)
output_pixelsize(term, value)
output_twotheta(term, value)
output_wavelength(term, value)
parse_output(output)
parse_term(line)
parse_value(line)
run_diff2jpeg(filename)
run_diffdump(file_path)
saveDiffractionImageMetadata(instance, schema, metadata)

Save all the metadata to a DataFiles paramamter set.

split_oscillation(terms, value)
split_output(terms, value, strip)
tardis.tardis_portal.filters.diffractionimage.make_filter(name='', schema='', tagsToFind=[], tagsToExclude=[])

This filter runs the CCP4 diffdump binary on a diffraction image and collects its output into the trddatafile schema

If a white list is specified then it takes precidence and all other tags will be ignored.

param name: the short name of the schema. type name: string param schema: the name of the schema to load the EXIF data into. type schema: string param tagsToFind: a list of the tags to include. type tagsToFind: list of strings param tagsToExclude: a list of the tags to exclude. type tagsToExclude: list of strings

tardis.tardis_portal.filters.jeolsem module
class tardis.tardis_portal.filters.jeolsem.JEOLSEMFilter

Bases: object

This filter collects metadata from JEOL SEM text files.

param name: the short name of the schema. type name: string param schema: the name of the schema to load the EXIF data into. type schema: string

ATTR_PREFIXES = ('$CM_', '$$SM_')
SCHEMA = 'http://www.jeol.com/#jeol-sem-schema'
get_file_contents(datafile)
get_metadata(schema, filedata)
is_already_processed(datafile)
is_jeol_sem_metadata(filedata)
is_text_file(datafile)
save_metadata(datafile, schema, metadata)
Module contents

__init__.py

class tardis.tardis_portal.filters.FilterInitMiddleware(filters=None)

Bases: object

tardis.tardis_portal.management package
Subpackages
tardis.tardis_portal.management.commands package
Submodules
tardis.tardis_portal.management.commands.backupdb module

Command for backup database

class tardis.tardis_portal.management.commands.backupdb.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

backup()
do_mysql_backup(outfile)
do_mysql_restore(infile)
do_postgresql_backup(outfile)
do_postgresql_restore(infile)
handle(*args, **options)
help = 'Backup database. Only Mysql and Postgresql engines are implemented'
option_list = (<Option at 0x7fabbcb9c488: -r/--restore>, <Option at 0x7fabbcb9c3f8: -d/--database>)
restore(infile)
tardis.tardis_portal.management.commands.checkhashes module

Management utility to create a token user

exception tardis.tardis_portal.management.commands.checkhashes.AlgorithmNotSupported

Bases: exceptions.Exception

class tardis.tardis_portal.management.commands.checkhashes.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

handle(*args, **options)
help = 'Used to check file hashes against actual files.'
exception tardis.tardis_portal.management.commands.checkhashes.NoHashAvailable

Bases: exceptions.Exception

exception tardis.tardis_portal.management.commands.checkhashes.ValidationFailed

Bases: exceptions.Exception

tardis.tardis_portal.management.commands.checkhashes.validate_digest(algorithm, datafile)
tardis.tardis_portal.management.commands.chownexperiment module

Management command to change the owner of some experiments

class tardis.tardis_portal.management.commands.chownexperiment.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

args = '<user> <experiment id>...'
handle(*args, **options)
help = 'Change the owner of the experiments.'
option_list = (<Option at 0x7fabbd2b15f0: -n/--dryRun>, <Option at 0x7fabbca9bf80: -u/--forUser>, <Option at 0x7fabbca9be18: -a/--all>)
tardis.tardis_portal.management.commands.cleanuptokens module

Management utility to clean up tokens

class tardis.tardis_portal.management.commands.cleanuptokens.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

handle(*args, **options)
help = 'Deletes unused tokens and optionally their ACLs'
option_list = (<Option at 0x7fabbca82f38: --keep-acls>,)
tardis.tardis_portal.management.commands.createmysuperuser module

Management utility to create superusers.

class tardis.tardis_portal.management.commands.createmysuperuser.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

handle(*args, **options)
help = 'Used to create a MyTardis superuser.'
option_list = (<Option at 0x7fabbca87bd8: --username>, <Option at 0x7fabbcb88a70: --email>, <Option at 0x7fabbcb88bd8: --noinput>)
tardis.tardis_portal.management.commands.createmysuperuser.is_valid_email(value)
tardis.tardis_portal.management.commands.createtokenuser module

Management utility to create a token user

class tardis.tardis_portal.management.commands.createtokenuser.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

handle(*args, **options)
help = 'Used to create a MyTardis tokenuser.'
option_list = (<Option at 0x7fabbc99c1b8: --username>,)
tardis.tardis_portal.management.commands.createuser module

Management utility to create regular users.

class tardis.tardis_portal.management.commands.createuser.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

handle(*args, **options)
help = 'Used to create a MyTardis user.'
option_list = (<Option at 0x7fabbcaafcb0: --username>, <Option at 0x7fabbcaafd88: --email>, <Option at 0x7fabbcaafd40: --noinput>)
tardis.tardis_portal.management.commands.createuser.is_valid_email(value)
tardis.tardis_portal.management.commands.dumpschemas module

Command for dumping soft schema definitions

class tardis.tardis_portal.management.commands.dumpschemas.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

args = '[namespace...]'
handle(*args, **options)
help = 'Dump soft schema definitions. No namespace = dump all schemas'
option_list = (<Option at 0x7fabbcaa29e0: --format>, <Option at 0x7fabbcafcd88: --database>)
tardis.tardis_portal.management.commands.loadschemas module

Command for loading soft schema definitions

class tardis.tardis_portal.management.commands.loadschemas.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

args = 'schema [schema ...]'
handle(*args, **options)
help = 'Load soft schema definitions'
option_list = (<Option at 0x7fabbd30b440: --database>, <Option at 0x7fabbd30bb48: --replace>)
tardis.tardis_portal.management.commands.rmexperiment module

Management command to delete the specified experiment and its associated datasets, datafiles and parameters.

The operation is atomic, either the entire experiment is deleted, or nothing.

rmexperiment was introduced due to the Oracle DISTINCT workaround causing sql delete cascading to fail. The current implementation of rmexperiment still relies on some cascading.

class tardis.tardis_portal.management.commands.rmexperiment.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

args = '<MyTardis Exp ID>'
handle(*args, **options)
help = 'Delete the supplied MyTardis Experiment ID'
option_list = (<Option at 0x7fabbd480998: --list>, <Option at 0x7fabbd457cb0: --confirmed>)
tardis.tardis_portal.management.commands.runfilters module

Management command to (re-)run the ingestion filters by hand.

class tardis.tardis_portal.management.commands.runfilters.Command(stdout=None, stderr=None, no_color=False)

Bases: django.core.management.base.BaseCommand

args = '[<filter-no>] ...'
handle(*args, **options)
help = "Run selected ingestion filters on all Datafiles.\nNote that a typical ingestion filter sets a 'flag' parameter in its\nDatafile's parameter set to avoid adding multiple copies of the ingested\nmetadata parameters. This command cannot override that flag to force\nmetadata to be reingested."
instantiateFilters(filterIds)
listFilters()
option_list = (<Option at 0x7fabbc9bb908: -n/--dryRun>, <Option at 0x7fabbc9bbb00: -l/--list>, <Option at 0x7fabbc9bbb48: -a/--all>)
runFilters(filters, dryRun=False)
tardis.tardis_portal.management.commands.sftpd module
Module contents
Module contents
tardis.tardis_portal.migrations package
Submodules
tardis.tardis_portal.migrations.0001_initial module
class tardis.tardis_portal.migrations.0001_initial.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'contenttypes', u'0002_remove_content_type_name'), (u'auth', u'__first__'), (u'auth', u'0006_require_contenttypes_0002')]
operations = [<CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'filename', <django.db.models.fields.CharField>), (u'directory', <django.db.models.fields.TextField>), (u'size', <django.db.models.fields.CharField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'modification_time', <django.db.models.fields.DateTimeField>), (u'mimetype', <django.db.models.fields.CharField>), (u'md5sum', <django.db.models.fields.CharField>), (u'sha512sum', <django.db.models.fields.CharField>), (u'deleted', <django.db.models.fields.BooleanField>), (u'deleted_time', <django.db.models.fields.DateTimeField>), (u'version', <django.db.models.fields.IntegerField>)], options={u'ordering': [u'filename']}, name=u'DataFile'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'uri', <django.db.models.fields.TextField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'verified', <django.db.models.fields.BooleanField>), (u'last_verified_time', <django.db.models.fields.DateTimeField>), (u'datafile', <django.db.models.fields.related.ForeignKey>)], name=u'DataFileObject'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'DatafileParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'datafile', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'DatafileParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'description', <django.db.models.fields.TextField>), (u'directory', <django.db.models.fields.TextField>), (u'immutable', <django.db.models.fields.BooleanField>)], options={u'ordering': [u'-id']}, name=u'Dataset'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'DatasetParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'dataset', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'DatasetParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'url', <django.db.models.fields.URLField>), (u'approved', <django.db.models.fields.BooleanField>), (u'title', <django.db.models.fields.CharField>), (u'institution_name', <django.db.models.fields.CharField>), (u'description', <django.db.models.fields.TextField>), (u'start_time', <django.db.models.fields.DateTimeField>), (u'end_time', <django.db.models.fields.DateTimeField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'update_time', <django.db.models.fields.DateTimeField>), (u'handle', <django.db.models.fields.TextField>), (u'locked', <django.db.models.fields.BooleanField>), (u'public_access', <django.db.models.fields.PositiveSmallIntegerField>), (u'created_by', <django.db.models.fields.related.ForeignKey>)], name=u'Experiment'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'author', <django.db.models.fields.CharField>), (u'institution', <django.db.models.fields.CharField>), (u'email', <django.db.models.fields.CharField>), (u'order', <django.db.models.fields.PositiveIntegerField>), (u'url', <django.db.models.fields.URLField>), (u'experiment', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'order']}, name=u'ExperimentAuthor'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'ExperimentParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'experiment', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'ExperimentParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'manager_group', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'Facilities'}, name=u'Facility'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>)], name=u'FreeTextSearchField'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'group', <django.db.models.fields.related.ForeignKey>), (u'user', <django.db.models.fields.related.ForeignKey>)], name=u'GroupAdmin'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'facility', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'Instruments'}, name=u'Instrument'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'InstrumentParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'instrument', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'InstrumentParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'jti', <django.db.models.fields.CharField>), (u'created_time', <django.db.models.fields.DateTimeField>)], name=u'JTI'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'url', <django.db.models.fields.URLField>), (u'internal_description', <django.db.models.fields.TextField>), (u'image_url', <django.db.models.fields.URLField>), (u'allows_distribution', <django.db.models.fields.BooleanField>), (u'is_active', <django.db.models.fields.BooleanField>)], name=u'License'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'pluginId', <django.db.models.fields.CharField>), (u'entityId', <django.db.models.fields.CharField>), (u'object_id', <django.db.models.fields.PositiveIntegerField>), (u'canRead', <django.db.models.fields.BooleanField>), (u'canWrite', <django.db.models.fields.BooleanField>), (u'canDelete', <django.db.models.fields.BooleanField>), (u'isOwner', <django.db.models.fields.BooleanField>), (u'effectiveDate', <django.db.models.fields.DateField>), (u'expiryDate', <django.db.models.fields.DateField>), (u'aclOwnershipType', <django.db.models.fields.IntegerField>), (u'content_type', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'content_type', u'object_id'], u'verbose_name': u'Object ACL'}, name=u'ObjectACL'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'full_name', <django.db.models.fields.CharField>), (u'units', <django.db.models.fields.CharField>), (u'data_type', <django.db.models.fields.IntegerField>), (u'immutable', <django.db.models.fields.BooleanField>), (u'comparison_type', <django.db.models.fields.IntegerField>), (u'is_searchable', <django.db.models.fields.BooleanField>), (u'choices', <django.db.models.fields.CharField>), (u'order', <django.db.models.fields.PositiveIntegerField>)], options={u'ordering': (u'order', u'name')}, name=u'ParameterName'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'namespace', <django.db.models.fields.URLField>), (u'name', <django.db.models.fields.CharField>), (u'type', <django.db.models.fields.IntegerField>), (u'subtype', <django.db.models.fields.CharField>), (u'immutable', <django.db.models.fields.BooleanField>), (u'hidden', <django.db.models.fields.BooleanField>)], name=u'Schema'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'django_storage_class', <django.db.models.fields.TextField>), (u'max_size', <django.db.models.fields.BigIntegerField>), (u'status', <django.db.models.fields.CharField>), (u'name', <django.db.models.fields.TextField>), (u'description', <django.db.models.fields.TextField>), (u'master_box', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'storage boxes'}, name=u'StorageBox'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key', <django.db.models.fields.TextField>), (u'value', <django.db.models.fields.TextField>), (u'storage_box', <django.db.models.fields.related.ForeignKey>)], name=u'StorageBoxAttribute'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key', <django.db.models.fields.TextField>), (u'value', <django.db.models.fields.TextField>), (u'storage_box', <django.db.models.fields.related.ForeignKey>)], name=u'StorageBoxOption'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'token', <django.db.models.fields.CharField>), (u'expiry_date', <django.db.models.fields.DateField>), (u'experiment', <django.db.models.fields.related.ForeignKey>), (u'user', <django.db.models.fields.related.ForeignKey>)], name=u'Token'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'username', <django.db.models.fields.CharField>), (u'authenticationMethod', <django.db.models.fields.CharField>)], name=u'UserAuthentication'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'isDjangoAccount', <django.db.models.fields.BooleanField>), (u'rapidConnectEduPersonTargetedID', <django.db.models.fields.CharField>), (u'user', <django.db.models.fields.related.ForeignKey>)], name=u'UserProfile'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'userProfile', model_name=u'userauthentication'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'parametername'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'instrumentparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'instrumentparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'instrumentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'instrumentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameter_name', model_name=u'freetextsearchfield'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'experimentparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'experimentparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'experimentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'experimentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'license', model_name=u'experiment'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'datasetparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'datasetparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'datasetparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'datasetparameter'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'experiments', model_name=u'dataset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'instrument', model_name=u'dataset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'datafileparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'datafileparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'datafileparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'datafileparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'storage_box', model_name=u'datafileobject'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'dataset', model_name=u'datafile'>, <AlterUniqueTogether unique_together=set([(u'schema', u'name')]), name=u'parametername'>, <AlterUniqueTogether unique_together=set([(u'experiment', u'author')]), name=u'experimentauthor'>, <AlterUniqueTogether unique_together=set([(u'datafile', u'storage_box')]), name=u'datafileobject'>, <AlterUniqueTogether unique_together=set([(u'dataset', u'directory', u'filename', u'version')]), name=u'datafile'>]
tardis.tardis_portal.migrations.0001_squashed_0011_auto_20160505_1643 module
class tardis.tardis_portal.migrations.0001_squashed_0011_auto_20160505_1643.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'contenttypes', u'0002_remove_content_type_name'), (u'auth', u'__first__'), (u'auth', u'0006_require_contenttypes_0002')]
operations = [<CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'filename', <django.db.models.fields.CharField>), (u'directory', <django.db.models.fields.CharField>), (u'size', <django.db.models.fields.CharField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'modification_time', <django.db.models.fields.DateTimeField>), (u'mimetype', <django.db.models.fields.CharField>), (u'md5sum', <django.db.models.fields.CharField>), (u'sha512sum', <django.db.models.fields.CharField>), (u'deleted', <django.db.models.fields.BooleanField>), (u'deleted_time', <django.db.models.fields.DateTimeField>), (u'version', <django.db.models.fields.IntegerField>)], options={u'ordering': [u'filename']}, name=u'DataFile'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'uri', <django.db.models.fields.TextField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'verified', <django.db.models.fields.BooleanField>), (u'last_verified_time', <django.db.models.fields.DateTimeField>), (u'datafile', <django.db.models.fields.related.ForeignKey>)], name=u'DataFileObject'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'DatafileParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'datafile', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'DatafileParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'description', <django.db.models.fields.TextField>), (u'directory', <django.db.models.fields.CharField>), (u'immutable', <django.db.models.fields.BooleanField>)], options={u'ordering': [u'-id']}, name=u'Dataset'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'DatasetParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'dataset', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'DatasetParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'url', <django.db.models.fields.URLField>), (u'approved', <django.db.models.fields.BooleanField>), (u'title', <django.db.models.fields.CharField>), (u'institution_name', <django.db.models.fields.CharField>), (u'description', <django.db.models.fields.TextField>), (u'start_time', <django.db.models.fields.DateTimeField>), (u'end_time', <django.db.models.fields.DateTimeField>), (u'created_time', <django.db.models.fields.DateTimeField>), (u'update_time', <django.db.models.fields.DateTimeField>), (u'handle', <django.db.models.fields.TextField>), (u'locked', <django.db.models.fields.BooleanField>), (u'public_access', <django.db.models.fields.PositiveSmallIntegerField>), (u'created_by', <django.db.models.fields.related.ForeignKey>)], name=u'Experiment'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'author', <django.db.models.fields.CharField>), (u'institution', <django.db.models.fields.CharField>), (u'email', <django.db.models.fields.CharField>), (u'order', <django.db.models.fields.PositiveIntegerField>), (u'url', <django.db.models.fields.URLField>), (u'experiment', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'order']}, name=u'ExperimentAuthor'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'ExperimentParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'experiment', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'ExperimentParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'manager_group', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'Facilities'}, name=u'Facility'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>)], name=u'FreeTextSearchField'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'group', <django.db.models.fields.related.ForeignKey>), (u'user', <django.db.models.fields.related.ForeignKey>)], name=u'GroupAdmin'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'facility', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'Instruments'}, name=u'Instrument'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'string_value', <django.db.models.fields.TextField>), (u'numerical_value', <django.db.models.fields.FloatField>), (u'datetime_value', <django.db.models.fields.DateTimeField>), (u'link_id', <django.db.models.fields.PositiveIntegerField>), (u'link_ct', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'name'], u'abstract': False}, name=u'InstrumentParameter'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'instrument', <django.db.models.fields.related.ForeignKey>)], bases=(<class 'django.db.models.base.Model'>, <class 'tardis.tardis_portal.models.parameters.ParameterSetManagerMixin'>), options={u'ordering': [u'id'], u'abstract': False}, name=u'InstrumentParameterSet'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'jti', <django.db.models.fields.CharField>), (u'created_time', <django.db.models.fields.DateTimeField>)], name=u'JTI'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'url', <django.db.models.fields.URLField>), (u'internal_description', <django.db.models.fields.TextField>), (u'image_url', <django.db.models.fields.URLField>), (u'allows_distribution', <django.db.models.fields.BooleanField>), (u'is_active', <django.db.models.fields.BooleanField>)], name=u'License'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'pluginId', <django.db.models.fields.CharField>), (u'entityId', <django.db.models.fields.CharField>), (u'object_id', <django.db.models.fields.PositiveIntegerField>), (u'canRead', <django.db.models.fields.BooleanField>), (u'canWrite', <django.db.models.fields.BooleanField>), (u'canDelete', <django.db.models.fields.BooleanField>), (u'isOwner', <django.db.models.fields.BooleanField>), (u'effectiveDate', <django.db.models.fields.DateField>), (u'expiryDate', <django.db.models.fields.DateField>), (u'aclOwnershipType', <django.db.models.fields.IntegerField>), (u'content_type', <django.db.models.fields.related.ForeignKey>)], options={u'ordering': [u'content_type', u'object_id'], u'verbose_name': u'Object ACL'}, name=u'ObjectACL'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'name', <django.db.models.fields.CharField>), (u'full_name', <django.db.models.fields.CharField>), (u'units', <django.db.models.fields.CharField>), (u'data_type', <django.db.models.fields.IntegerField>), (u'immutable', <django.db.models.fields.BooleanField>), (u'comparison_type', <django.db.models.fields.IntegerField>), (u'is_searchable', <django.db.models.fields.BooleanField>), (u'choices', <django.db.models.fields.CharField>), (u'order', <django.db.models.fields.PositiveIntegerField>)], options={u'ordering': (u'order', u'name')}, name=u'ParameterName'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'namespace', <django.db.models.fields.URLField>), (u'name', <django.db.models.fields.CharField>), (u'type', <django.db.models.fields.IntegerField>), (u'subtype', <django.db.models.fields.CharField>), (u'immutable', <django.db.models.fields.BooleanField>), (u'hidden', <django.db.models.fields.BooleanField>)], name=u'Schema'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'django_storage_class', <django.db.models.fields.TextField>), (u'max_size', <django.db.models.fields.BigIntegerField>), (u'status', <django.db.models.fields.CharField>), (u'name', <django.db.models.fields.CharField>), (u'description', <django.db.models.fields.TextField>), (u'master_box', <django.db.models.fields.related.ForeignKey>)], options={u'verbose_name_plural': u'storage boxes'}, name=u'StorageBox'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key', <django.db.models.fields.TextField>), (u'value', <django.db.models.fields.TextField>), (u'storage_box', <django.db.models.fields.related.ForeignKey>)], name=u'StorageBoxAttribute'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'key', <django.db.models.fields.TextField>), (u'value', <django.db.models.fields.TextField>), (u'storage_box', <django.db.models.fields.related.ForeignKey>), (u'value_type', <django.db.models.fields.CharField>)], name=u'StorageBoxOption'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'token', <django.db.models.fields.CharField>), (u'expiry_date', <django.db.models.fields.DateField>), (u'experiment', <django.db.models.fields.related.ForeignKey>), (u'user', <django.db.models.fields.related.ForeignKey>)], name=u'Token'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'username', <django.db.models.fields.CharField>), (u'authenticationMethod', <django.db.models.fields.CharField>)], name=u'UserAuthentication'>, <CreateModel fields=[(u'id', <django.db.models.fields.AutoField>), (u'isDjangoAccount', <django.db.models.fields.BooleanField>), (u'rapidConnectEduPersonTargetedID', <django.db.models.fields.CharField>), (u'user', <django.db.models.fields.related.OneToOneField>)], name=u'UserProfile'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'userProfile', model_name=u'userauthentication'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'parametername'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'instrumentparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'instrumentparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'instrumentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'instrumentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameter_name', model_name=u'freetextsearchfield'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'experimentparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'experimentparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'experimentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'experimentparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'license', model_name=u'experiment'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'datasetparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'datasetparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'datasetparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'datasetparameter'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'experiments', model_name=u'dataset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'instrument', model_name=u'dataset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'schema', model_name=u'datafileparameterset'>, <AddField field=<django.db.models.fields.related.ManyToManyField>, name=u'storage_box', model_name=u'datafileparameterset'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'name', model_name=u'datafileparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'parameterset', model_name=u'datafileparameter'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'storage_box', model_name=u'datafileobject'>, <AddField field=<django.db.models.fields.related.ForeignKey>, name=u'dataset', model_name=u'datafile'>, <AlterUniqueTogether unique_together=set([(u'schema', u'name')]), name=u'parametername'>, <AlterUniqueTogether unique_together=set([(u'experiment', u'author')]), name=u'experimentauthor'>, <AlterUniqueTogether unique_together=set([(u'datafile', u'storage_box')]), name=u'datafileobject'>, <AlterUniqueTogether unique_together=set([(u'dataset', u'directory', u'filename', u'version')]), name=u'datafile'>, <AlterField field=<django.db.models.fields.IntegerField>, name=u'data_type', model_name=u'parametername'>, <AddField field=<django.db.models.fields.BigIntegerField>, name=u'_size', model_name=u'datafile'>, <RunPython <function cast_string_to_integer>>, <RemoveField name=u'size', model_name=u'datafile'>, <RenameField new_name=u'size', model_name=u'datafile', old_name=u'_size'>, <AlterUniqueTogether unique_together=set([(u'name', u'facility')]), name=u'instrument'>, <AlterField field=<django.db.models.fields.CharField>, name=u'mimetype', model_name=u'datafile'>, <AlterField field=<django.db.models.fields.CharField>, name=u'directory', model_name=u'datafile'>, <AlterField field=<django.db.models.fields.CharField>, name=u'directory', model_name=u'dataset'>, <AlterField field=<django.db.models.fields.URLField>, name=u'url', model_name=u'experimentauthor'>, <AlterField field=<django.db.models.fields.URLField>, name=u'image_url', model_name=u'license'>, <AlterField field=<django.db.models.fields.CharField>, name=u'name', model_name=u'license'>, <AlterField field=<django.db.models.fields.URLField>, name=u'url', model_name=u'license'>, <AlterField field=<django.db.models.fields.CharField>, name=u'name', model_name=u'storagebox'>]
replaces = [('tardis_portal', u'0001_initial'), ('tardis_portal', u'0002_auto_20150528_1128'), ('tardis_portal', u'0003_auto_20150907_1315'), ('tardis_portal', u'0004_storageboxoption_value_type'), ('tardis_portal', u'0005_datafile_add_size_int_column'), ('tardis_portal', u'0006_datafile_remove_size_string_column'), ('tardis_portal', u'0007_remove_parameter_string_value_index'), ('tardis_portal', u'0008_string_value_partial_index_postgres'), ('tardis_portal', u'0009_auto_20160128_1119'), ('tardis_portal', u'0010_auto_20160503_1443'), ('tardis_portal', u'0011_auto_20160505_1643')]
tardis.tardis_portal.migrations.0001_squashed_0011_auto_20160505_1643.cast_string_to_integer(apps, schema_editor)
tardis.tardis_portal.migrations.0002_auto_20150528_1128 module
class tardis.tardis_portal.migrations.0002_auto_20150528_1128.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0001_initial')]
operations = [<AlterField field=<django.db.models.fields.related.OneToOneField>, name=u'user', model_name=u'userprofile'>]
tardis.tardis_portal.migrations.0003_auto_20150907_1315 module
class tardis.tardis_portal.migrations.0003_auto_20150907_1315.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0002_auto_20150528_1128')]
operations = [<AlterField field=<django.db.models.fields.IntegerField>, name=u'data_type', model_name=u'parametername'>]
tardis.tardis_portal.migrations.0004_storageboxoption_value_type module
class tardis.tardis_portal.migrations.0004_storageboxoption_value_type.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0003_auto_20150907_1315')]
operations = [<AddField field=<django.db.models.fields.CharField>, name=u'value_type', model_name=u'storageboxoption'>]
tardis.tardis_portal.migrations.0005_datafile_add_size_int_column module
class tardis.tardis_portal.migrations.0005_datafile_add_size_int_column.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0004_storageboxoption_value_type')]
operations = [<AddField field=<django.db.models.fields.BigIntegerField>, name=u'_size', model_name=u'datafile'>, <RunPython <function cast_string_to_integer>>]
tardis.tardis_portal.migrations.0005_datafile_add_size_int_column.cast_string_to_integer(apps, schema_editor)
tardis.tardis_portal.migrations.0006_datafile_remove_size_string_column module
class tardis.tardis_portal.migrations.0006_datafile_remove_size_string_column.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0005_datafile_add_size_int_column')]
operations = [<RemoveField name=u'size', model_name=u'datafile'>, <RenameField new_name=u'size', model_name=u'datafile', old_name=u'_size'>]
tardis.tardis_portal.migrations.0007_remove_parameter_string_value_index module
class tardis.tardis_portal.migrations.0007_remove_parameter_string_value_index.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0006_datafile_remove_size_string_column')]
operations = [<AlterField field=<django.db.models.fields.TextField>, name=u'string_value', model_name=u'datafileparameter'>, <AlterField field=<django.db.models.fields.TextField>, name=u'string_value', model_name=u'datasetparameter'>, <AlterField field=<django.db.models.fields.TextField>, name=u'string_value', model_name=u'experimentparameter'>, <AlterField field=<django.db.models.fields.TextField>, name=u'string_value', model_name=u'instrumentparameter'>]
tardis.tardis_portal.migrations.0008_string_value_partial_index_postgres module
class tardis.tardis_portal.migrations.0008_string_value_partial_index_postgres.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0007_remove_parameter_string_value_index')]
operations = []
tardis.tardis_portal.migrations.0009_auto_20160128_1119 module
class tardis.tardis_portal.migrations.0009_auto_20160128_1119.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0008_string_value_partial_index_postgres')]
operations = [<AlterUniqueTogether unique_together=set([(u'name', u'facility')]), name=u'instrument'>]
tardis.tardis_portal.migrations.0010_auto_20160503_1443 module
class tardis.tardis_portal.migrations.0010_auto_20160503_1443.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0009_auto_20160128_1119')]
operations = [<AlterField field=<django.db.models.fields.CharField>, name=u'mimetype', model_name=u'datafile'>]
tardis.tardis_portal.migrations.0011_auto_20160505_1643 module
class tardis.tardis_portal.migrations.0011_auto_20160505_1643.Migration(name, app_label)

Bases: django.db.migrations.migration.Migration

dependencies = [(u'tardis_portal', u'0010_auto_20160503_1443')]
operations = [<AlterField field=<django.db.models.fields.CharField>, name=u'directory', model_name=u'datafile'>, <AlterField field=<django.db.models.fields.CharField>, name=u'directory', model_name=u'dataset'>, <AlterField field=<django.db.models.fields.URLField>, name=u'url', model_name=u'experimentauthor'>, <AlterField field=<django.db.models.fields.URLField>, name=u'image_url', model_name=u'license'>, <AlterField field=<django.db.models.fields.CharField>, name=u'name', model_name=u'license'>, <AlterField field=<django.db.models.fields.URLField>, name=u'url', model_name=u'license'>, <AlterField field=<django.db.models.fields.CharField>, name=u'name', model_name=u'storagebox'>]
Module contents
tardis.tardis_portal.models package
Submodules
tardis.tardis_portal.models.access_control module
class tardis.tardis_portal.models.access_control.GroupAdmin(*args, **kwargs)

Bases: django.db.models.base.Model

GroupAdmin links the Django User and Group tables for group administrators

Attribute user:a forign key to the django.contrib.auth.models.User
Attribute group:
 a forign key to the django.contrib.auth.models.Group
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception GroupAdmin.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

GroupAdmin.group
GroupAdmin.objects = <django.db.models.manager.Manager object>
GroupAdmin.user
class tardis.tardis_portal.models.access_control.ObjectACL(*args, **kwargs)

Bases: django.db.models.base.Model

The ObjectACL (formerly ExperimentACL) table is the core of the Tardis Authorisation framework

Attribute pluginId:
 the the name of the auth plugin being used
Attribute entityId:
 a foreign key to auth plugins
Attribute object_type:
 a foreign key to ContentType
Attribute object_id:
 the primary key/id of the object_type
Attribute canRead:
 gives the user read access
Attribute canWrite:
 gives the user write access
Attribute canDelete:
 gives the user delete permission
Attribute isOwner:
 the experiment owner flag.
Attribute effectiveDate:
 the date when access takes into effect
Attribute expiryDate:
 the date when access ceases
Attribute aclOwnershipType:
 system-owned or user-owned.

System-owned ACLs will prevent users from removing or editing ACL entries to a particular experiment they own. User-owned ACLs will allow experiment owners to remove/add/edit ACL entries to the experiments they own.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ObjectACL.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ObjectACL.OWNER_OWNED = 1
ObjectACL.SYSTEM_OWNED = 2
ObjectACL.content_object

Provides a generic relation to any object through content-type/object-id fields.

ObjectACL.content_type
ObjectACL.get_aclOwnershipType_display(*moreargs, **morekwargs)
classmethod ObjectACL.get_effective_query()

If possible, resolve the pluginId/entityId combination to a user or group object.

If possible, resolve the pluginId/entityId combination to a user or group object.

ObjectACL.objects = <django.db.models.manager.Manager object>
class tardis.tardis_portal.models.access_control.UserAuthentication(id, userProfile, username, authenticationMethod)

Bases: django.db.models.base.Model

CHOICES = ()
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception UserAuthentication.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

UserAuthentication.getAuthMethodDescription()
UserAuthentication.objects = <django.db.models.manager.Manager object>
UserAuthentication.userProfile
class tardis.tardis_portal.models.access_control.UserProfile(*args, **kwargs)

Bases: django.db.models.base.Model

UserProfile class is an extension to the Django standard user model.

Attribute isDjangoAccount:
 is the user a local DB user
Attribute user:a foreign key to the django.contrib.auth.models.User
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception UserProfile.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

UserProfile.ext_groups
UserProfile.getUserAuthentications()
UserProfile.isValidPublicContact()

Checks if there’s enough information on the user for it to be used as a public contact.

Note: Last name can’t be required, because people don’t necessarilly have a last (or family) name.

UserProfile.objects = <django.db.models.manager.Manager object>
UserProfile.user
UserProfile.userauthentication_set
tardis.tardis_portal.models.access_control.create_user_api_key(sender, **kwargs)

Auto-create ApiKey objects using Tastypie’s create_api_key

tardis.tardis_portal.models.access_control.create_user_profile(sender, instance, created, **kwargs)
tardis.tardis_portal.models.datafile module
class tardis.tardis_portal.models.datafile.DataFile(*args, **kwargs)

Bases: django.db.models.base.Model

Class to store meta-data about a file. The physical copies of a file are described by distinct DataFileObject instances.

Attribute dataset:
 the foreign key to the tardis.tardis_portal.models.Dataset the file belongs to.
Attribute filename:
 the name of the file, excluding the path.
Attribute size:the size of the file.
Attribute created_time:
 time the file was added to tardis
Attribute modification_time:
 last modification time of the file
Attribute mimetype:
 for example ‘application/pdf’
Attribute md5sum:
 digest of length 32, containing only hexadecimal digits
Attribute sha512sum:
 digest of length 128, containing only hexadecimal digits
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DataFile.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

DataFile.cache_file()
DataFile.datafileparameterset_set
DataFile.dataset
DataFile.file_object
DataFile.file_objects
DataFile.getParameterSets(schemaType=None)

Return datafile parametersets associated with this datafile.

DataFile.get_absolute_filepath()
DataFile.get_as_temporary_file(*args, **kwds)

Returns a traditional file-system-based file object that is a copy of the original data. The file is deleted when the context is destroyed. :param basestring directory: the directory in which to create the temp

file
Returns:the temporary file object
Return type:NamedTemporaryFile
DataFile.get_default_storage_box()

try to guess appropriate box from files, dataset or experiment

DataFile.get_download_url()
DataFile.get_file(verified_only=True)

Returns the file as a readable file-like object from the best avaiable storage box.

If verified_only=False, the return of files without a verified checksum is allowed, otherwise None is returned for unverified files.

Parameters:verified_only (bool) – if False return files without verified checksums
Returns:Python file object
Return type:Python File object
DataFile.get_image_data()
DataFile.get_mimetype()
DataFile.get_preferred_dfo(verified_only=True)
DataFile.get_receiving_storage_box()
DataFile.get_size()
DataFile.get_view_url()
DataFile.has_image()
DataFile.is_image()

returns True if it’s an image and not an x-icon and not an img the image/img mimetype is made up though and may need revisiting if there is an official img mimetype that does not refer to diffraction images

DataFile.is_local()
DataFile.is_online

return False if a file is on tape. At this stage it checks it returns true for no file objects, because those files are offline through other checks

DataFile.is_public()
DataFile.objects = <django.db.models.manager.Manager object>
DataFile.save(*args, **kwargs)
DataFile.status

returns information about the status of the file. States are defined in StorageBox

classmethod DataFile.sum_sizes(datafiles)

Takes a query set of datafiles and returns their total size.

DataFile.update_mimetype(mimetype=None, force=False, save=True)
DataFile.verified
DataFile.verify(reverify=False)
class tardis.tardis_portal.models.datafile.DataFileObject(*args, **kwargs)

Bases: django.db.models.base.Model

holds one copy of the data for a datafile

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DataFileObject.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

DataFileObject.cache_file()
DataFileObject.copy_file(dest_box=None, verify=True)

copies verified file to new storage box checks for existing copy triggers async verification if not disabled :param StorageBox dest_box: StorageBox instance :param bool verify: :returns: DataFileObject of copy :rtype: DataFileObject

DataFileObject.create_set_uri(force=False, save=False)

sets the uri as well as building it :param bool force: :param book save: :return: :rtype: basestring

DataFileObject.datafile
DataFileObject.delete_data()
DataFileObject.file_object

A set of accessor functions that convert the file information to a standard Python file object for reading and copy the contents of an existing file_object into the storage backend.

Returns:a file object
Return type:Python File object
DataFileObject.get_full_path()
DataFileObject.get_next_by_created_time(*moreargs, **morekwargs)
DataFileObject.get_previous_by_created_time(*moreargs, **morekwargs)
DataFileObject.modified_time
DataFileObject.move_file(dest_box=None)

moves a file copies first, then synchronously verifies deletes file if copy is true copy and has been verified

Parameters:dest_box (StorageBox) – StorageBox instance
Returns:moved file dfo
Return type:DataFileObject
DataFileObject.objects = <django.db.models.manager.Manager object>
DataFileObject.save(*args, **kwargs)
DataFileObject.storage_box
DataFileObject.storage_type
Returns:storage_box type
Return type:StorageBox type constant
DataFileObject.verify(add_checksums=True, add_size=True)
tardis.tardis_portal.models.datafile.compute_checksums(file_object, compute_md5=True, compute_sha512=True, close_file=True)

Computes checksums for a python file object

Parameters:
  • file_object (object) – Python File object
  • compute_md5 (bool) – whether to compute md5 default=True
  • compute_sha512 (bool) – whether to compute sha512, default=True
  • close_file (bool) – whether to close the file_object, default=True
Returns:

the checksums as {‘md5sum’: result, ‘sha512sum’: result}

Return type:

dict

tardis.tardis_portal.models.datafile.delete_dfo(sender, instance, **kwargs)
tardis.tardis_portal.models.dataset module
class tardis.tardis_portal.models.dataset.Dataset(*args, **kwargs)

Bases: django.db.models.base.Model

Class to link datasets to experiments

Attribute experiment:
 a forign key to the tardis.tardis_portal.models.Experiment
Attribute facility:
 the foreign key to the facility that generated this data
Attribute instrument:
 the foreign key to the instrument that generated this data
Attribute description:
 description of this dataset
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Dataset.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Dataset.datafile_set
Dataset.datasetparameterset_set
Dataset.equipment_set
Dataset.experiments
Dataset.getParameterSets(schemaType=None)

Return the dataset parametersets associated with this experiment.

Dataset.get_absolute_url(*args, **kwargs)

Return the absolute url to the current Dataset

Dataset.get_all_storage_boxes_used()
Dataset.get_download_urls()
Dataset.get_edit_url(*args, **kwargs)

Return the absolute url to the edit view of the current Dataset

Dataset.get_first_experiment()
Dataset.get_images()
Dataset.get_path()
Dataset.get_size()
Dataset.get_thumbnail_url()
Dataset.image
Dataset.instrument
Dataset.is_online
Dataset.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
tardis.tardis_portal.models.experiment module
class tardis.tardis_portal.models.experiment.Experiment(*args, **kwargs)

Bases: django.db.models.base.Model

The Experiment model inherits from django.db.models.Model

Attribute url:Undocumented
Attribute approved:
 Undocumented
Attribute title:
 the title of the experiment.
Attribute institution_name:
 the name of the institution who created the dataset.
Attribute start_time:
 Undocumented
Attribute end_time:
 Undocumented
Attribute created_time:
 Undocumented
Attribute handle:
 Undocumented
Attribute public:
 Undocumented
Attribute objects:
 default model manager
Attribute safe:ACL aware model manager
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Experiment.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Experiment.PUBLICATION_DETAILS_SCHEMA = 'http://www.tardis.edu.au/schemas/publication/details/'
Experiment.PUBLICATION_DRAFT_SCHEMA = 'http://www.tardis.edu.au/schemas/publication/draft/'
Experiment.PUBLICATION_SCHEMA_ROOT = 'http://www.tardis.edu.au/schemas/publication/'
Experiment.PUBLIC_ACCESS_CHOICES = ((1, 'No public access (hidden)'), (25, 'Ready to be released pending embargo expiry'), (50, 'Public Metadata only (no data file access)'), (100, 'Public'))
Experiment.PUBLIC_ACCESS_EMBARGO = 25
Experiment.PUBLIC_ACCESS_FULL = 100
Experiment.PUBLIC_ACCESS_METADATA = 50
Experiment.PUBLIC_ACCESS_NONE = 1
Experiment.created_by
Experiment.datasets
Experiment.experimentauthor_set
Experiment.experimentparameterset_set
Experiment.getParameterSets(schemaType=None)

Return the experiment parametersets associated with this experiment.

Experiment.get_absolute_url(*args, **kwargs)

Return the absolute url to the current Experiment

Experiment.get_create_token_url(*args, **kwargs)

Return the absolute url to the create token view of the current Experiment

Experiment.get_ct()
Experiment.get_datafiles()
Experiment.get_download_urls()
Experiment.get_edit_url(*args, **kwargs)

Return the absolute url to the edit view of the current Experiment

Experiment.get_groups()
Experiment.get_images()
Experiment.get_next_by_created_time(*moreargs, **morekwargs)
Experiment.get_next_by_update_time(*moreargs, **morekwargs)
Experiment.get_or_create_directory()
Experiment.get_owners()
Experiment.get_previous_by_created_time(*moreargs, **morekwargs)
Experiment.get_previous_by_update_time(*moreargs, **morekwargs)
Experiment.get_public_access_display(*moreargs, **morekwargs)
Experiment.get_size()
Experiment.is_publication()
Experiment.is_publication_draft()
Experiment.license
Experiment.objectacls

This class provides the functionality that makes the related-object managers available as attributes on a model class, for fields that have multiple “remote” values and have a GenericRelation defined in their model (rather than having another model pointed at them). In the example “article.publications”, the publications attribute is a ReverseGenericRelatedObjectsDescriptor instance.

Experiment.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
classmethod Experiment.public_access_implies_distribution(public_access_level)

Determines if a level of public access implies that distribution should be allowed, or alternately if it should not be allowed. Used to prevent free-distribution licences for essentially private data, and overly-restrictive licences for public data.

Experiment.public_download_allowed()

instance method version of ‘public_access_implies_distribution’

Experiment.safe = <tardis.tardis_portal.managers.ExperimentManager object>
Experiment.save(*args, **kwargs)
Experiment.token_set
class tardis.tardis_portal.models.experiment.ExperimentAuthor(id, experiment, author, institution, email, order, url)

Bases: django.db.models.base.Model

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ExperimentAuthor.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ExperimentAuthor.experiment
ExperimentAuthor.objects = <django.db.models.manager.Manager object>
ExperimentAuthor.save(*args, **kwargs)
tardis.tardis_portal.models.facility module
class tardis.tardis_portal.models.facility.Facility(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a facility that produces data

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Facility.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Facility.instrument_set
Facility.manager_group
Facility.objects = <django.db.models.manager.Manager object>
tardis.tardis_portal.models.facility.facilities_managed_by(user)

Returns a list of facilities managed by a user

tardis.tardis_portal.models.facility.is_facility_manager(user)

Returns true if the user manages one or more facilities

tardis.tardis_portal.models.hooks module
tardis.tardis_portal.models.hooks.ensure_doi_exists(sender, **kwargs)
tardis.tardis_portal.models.hooks.post_save_experiment(sender, **kwargs)
tardis.tardis_portal.models.hooks.post_save_experiment_parameter(sender, **kwargs)
tardis.tardis_portal.models.hooks.post_save_experimentauthor(sender, **kwargs)
tardis.tardis_portal.models.hooks.publish_public_expt_rifcs(experiment)
tardis.tardis_portal.models.instrument module
class tardis.tardis_portal.models.instrument.Instrument(*args, **kwargs)

Bases: django.db.models.base.Model

Represents an instrument belonging to a facility that produces data

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Instrument.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Instrument.dataset_set
Instrument.facility
Instrument.getParameterSets(schemaType=None)

Return the instrument parametersets associated with this instrument.

Instrument.instrumentparameterset_set
Instrument.objects = <django.db.models.manager.Manager object>
tardis.tardis_portal.models.jti module
class tardis.tardis_portal.models.jti.JTI(id, jti, created_time)

Bases: django.db.models.base.Model

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception JTI.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

JTI.get_next_by_created_time(*moreargs, **morekwargs)
JTI.get_previous_by_created_time(*moreargs, **morekwargs)
JTI.objects = <django.db.models.manager.Manager object>
tardis.tardis_portal.models.license module
class tardis.tardis_portal.models.license.License(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a licence for experiment content.

Instances should provide enough detail for both researchers to select the licence, and for the users of their data to divine correct usage of experiment content.

(Non-US developers: We’re using US spelling in the code.)

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception License.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

License.experiment_set
classmethod License.get_none_option_license()
classmethod License.get_suitable_licenses(public_access_method=None)
License.objects = <django.db.models.manager.Manager object>
tardis.tardis_portal.models.parameters module
class tardis.tardis_portal.models.parameters.DatafileParameter(id, name, string_value, numerical_value, datetime_value, link_id, link_ct, parameterset)

Bases: tardis.tardis_portal.models.parameters.Parameter

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DatafileParameter.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Provides a generic relation to any object through content-type/object-id fields.

DatafileParameter.name
DatafileParameter.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
DatafileParameter.parameter_type = 'Datafile'
DatafileParameter.parameterset
class tardis.tardis_portal.models.parameters.DatafileParameterSet(id, schema, datafile)

Bases: tardis.tardis_portal.models.parameters.ParameterSet

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DatafileParameterSet.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

DatafileParameterSet.datafile
DatafileParameterSet.datafileparameter_set
DatafileParameterSet.objects = <django.db.models.manager.Manager object>
DatafileParameterSet.parameter_class

alias of DatafileParameter

DatafileParameterSet.schema
DatafileParameterSet.storage_box
class tardis.tardis_portal.models.parameters.DatasetParameter(id, name, string_value, numerical_value, datetime_value, link_id, link_ct, parameterset)

Bases: tardis.tardis_portal.models.parameters.Parameter

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DatasetParameter.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Provides a generic relation to any object through content-type/object-id fields.

DatasetParameter.name
DatasetParameter.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
DatasetParameter.parameter_type = 'Dataset'
DatasetParameter.parameterset
class tardis.tardis_portal.models.parameters.DatasetParameterSet(id, schema, dataset)

Bases: tardis.tardis_portal.models.parameters.ParameterSet

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception DatasetParameterSet.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

DatasetParameterSet.dataset
DatasetParameterSet.datasetparameter_set
DatasetParameterSet.objects = <django.db.models.manager.Manager object>
DatasetParameterSet.parameter_class

alias of DatasetParameter

DatasetParameterSet.schema
DatasetParameterSet.storage_box
class tardis.tardis_portal.models.parameters.ExperimentParameter(id, name, string_value, numerical_value, datetime_value, link_id, link_ct, parameterset)

Bases: tardis.tardis_portal.models.parameters.Parameter

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ExperimentParameter.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Provides a generic relation to any object through content-type/object-id fields.

ExperimentParameter.name
ExperimentParameter.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
ExperimentParameter.parameter_type = 'Experiment'
ExperimentParameter.parameterset
ExperimentParameter.save(*args, **kwargs)
class tardis.tardis_portal.models.parameters.ExperimentParameterSet(id, schema, experiment)

Bases: tardis.tardis_portal.models.parameters.ParameterSet

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ExperimentParameterSet.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ExperimentParameterSet.experiment
ExperimentParameterSet.experimentparameter_set
ExperimentParameterSet.objects = <django.db.models.manager.Manager object>
ExperimentParameterSet.parameter_class

alias of ExperimentParameter

ExperimentParameterSet.schema
ExperimentParameterSet.storage_box
class tardis.tardis_portal.models.parameters.FreeTextSearchField(id, parameter_name)

Bases: django.db.models.base.Model

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception FreeTextSearchField.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

FreeTextSearchField.objects = <django.db.models.manager.Manager object>
FreeTextSearchField.parameter_name
class tardis.tardis_portal.models.parameters.InstrumentParameter(id, name, string_value, numerical_value, datetime_value, link_id, link_ct, parameterset)

Bases: tardis.tardis_portal.models.parameters.Parameter

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception InstrumentParameter.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Provides a generic relation to any object through content-type/object-id fields.

InstrumentParameter.name
InstrumentParameter.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
InstrumentParameter.parameter_type = 'Instrument'
InstrumentParameter.parameterset
class tardis.tardis_portal.models.parameters.InstrumentParameterSet(id, schema, instrument)

Bases: tardis.tardis_portal.models.parameters.ParameterSet

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception InstrumentParameterSet.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

InstrumentParameterSet.instrument
InstrumentParameterSet.instrumentparameter_set
InstrumentParameterSet.objects = <django.db.models.manager.Manager object>
InstrumentParameterSet.parameter_class

alias of InstrumentParameter

InstrumentParameterSet.schema
InstrumentParameterSet.storage_box
class tardis.tardis_portal.models.parameters.Parameter(*args, **kwargs)

Bases: django.db.models.base.Model

class Meta
abstract = False
app_label = 'tardis_portal'
ordering = ['name']
Parameter.get()

Provides a generic relation to any object through content-type/object-id fields.

Parameter.name
Parameter.objects
Parameter.parameter_type = 'Abstract'
Parameter.set_value(value)

Sets the parameter value, converting into the appropriate data type. Deals with date/time strings that are timezone naive or aware, based on the USE_TZ setting.

Parameters:value (basestring) – a string (or string-like) repr of the value
Raises:SuspiciousOperation
class tardis.tardis_portal.models.parameters.ParameterName(id, schema, name, full_name, units, data_type, immutable, comparison_type, is_searchable, choices, order)

Bases: django.db.models.base.Model

CONTAINS_COMPARISON = 8
DATETIME = 6
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

ParameterName.EXACT_VALUE_COMPARISON = 1
ParameterName.FILENAME = 5
ParameterName.GREATER_THAN_COMPARISON = 4
ParameterName.GREATER_THAN_EQUAL_COMPARISON = 5
ParameterName.JSON = 8
ParameterName.LESS_THAN_COMPARISON = 6
ParameterName.LESS_THAN_EQUAL_COMPARISON = 7
ParameterName.LONGSTRING = 7
exception ParameterName.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ParameterName.NOT_EQUAL_COMPARISON = 2
ParameterName.NUMERIC = 1
ParameterName.RANGE_COMPARISON = 3
ParameterName.STRING = 2
ParameterName.URL = 3
ParameterName.datafileparameter_set
ParameterName.datasetparameter_set
ParameterName.experimentparameter_set
ParameterName.freetextsearchfield_set
ParameterName.getUniqueShortName()
ParameterName.get_comparison_type_display(*moreargs, **morekwargs)
ParameterName.get_data_type_display(*moreargs, **morekwargs)
ParameterName.instrumentparameter_set
ParameterName.isDateTime()
ParameterName.isFilename()
ParameterName.isLongString()
ParameterName.isNumeric()
ParameterName.isString()
ParameterName.isURL()
ParameterName.is_json()
ParameterName.natural_key()
ParameterName.objects = <tardis.tardis_portal.managers.ParameterNameManager object>
ParameterName.schema
class tardis.tardis_portal.models.parameters.ParameterSet(*args, **kwargs)

Bases: django.db.models.base.Model, tardis.tardis_portal.models.parameters.ParameterSetManagerMixin

class Meta
abstract = False
app_label = 'tardis_portal'
ordering = ['id']
ParameterSet.parameter_class = None
ParameterSet.save(*args, **kwargs)
ParameterSet.schema
ParameterSet.storage_box
class tardis.tardis_portal.models.parameters.ParameterSetManagerMixin(parameterset=None, parentObject=None, schema=None)

Bases: tardis.tardis_portal.ParameterSetManager.ParameterSetManager

for clarity’s sake and for future extension this class makes ParameterSetManager local to this file. At the moment its only function is increasing the line count

class tardis.tardis_portal.models.parameters.Schema(id, namespace, name, type, subtype, immutable, hidden)

Bases: django.db.models.base.Model

DATAFILE = 3
DATASET = 2
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

Schema.EXPERIMENT = 1
Schema.INSTRUMENT = 5
exception Schema.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Schema.NONE = 4
exception Schema.UnsupportedType(msg)

Bases: exceptions.Exception

Schema.datafileparameterset_set
Schema.datasetparameterset_set
Schema.experimentparameterset_set
classmethod Schema.getNamespaces(type_, subtype=None)

Return the list of namespaces for equipment, sample, and experiment schemas.

classmethod Schema.getSubTypes()
classmethod Schema.get_internal_schema(schema_type)
classmethod Schema.get_schema_type_name(schema_type, short=False)
Schema.get_type_display(*moreargs, **morekwargs)
Schema.instrumentparameterset_set
Schema.natural_key()
Schema.objects = <tardis.tardis_portal.managers.SchemaManager object>
Schema.parametername_set
tardis.tardis_portal.models.storage module
class tardis.tardis_portal.models.storage.StorageBox(*args, **kwargs)

Bases: django.db.models.base.Model

table that holds storage boxes of any type. to extend to new types, add fields if necessary

Attribute max_size:
 max size in bytes
BUNDLE = 6
CACHE = 3
DISK = 1
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception StorageBox.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

StorageBox.TAPE = 2
StorageBox.TEMPORARY = 4
StorageBox.TYPES = {'disk': 1, 'cache': 3, 'tape': 2, 'receiving': 4, 'bundle': 6}
StorageBox.TYPE_UNKNOWN = 5
StorageBox.attributes
StorageBox.cache_box

Get cache box if set up

StorageBox.child_boxes
StorageBox.copy_files(dest_box=None)
StorageBox.copy_to_master()
classmethod StorageBox.create_local_box(location=None)
StorageBox.datafileparametersets
StorageBox.datasetparametersets
StorageBox.experimentparametersets
StorageBox.file_objects
classmethod StorageBox.get_default_storage(location=None, user=None)

gets default storage box or get local storage box with given base location or create one if it doesn’t exist.

policies: Have a StorageBoxAttribute: key=’default’, value=True find a storage box where location is DEFAULT_STORAGE_BASE_DIR create a default storage box at DEFAULT_STORAGE_BASE_DIR lowest id storage box is default no storage box defined, use hard coded default for now TODO: consider removing this

Would be nice: get largest free space one, test for authorisation

StorageBox.get_initialised_storage_instance()
StorageBox.get_options_as_dict()
StorageBox.instrumentparametersets
StorageBox.master_box
StorageBox.move_files(dest_box=None)
StorageBox.move_to_master()
StorageBox.objects = <django.db.models.manager.Manager object>
StorageBox.offline_types = [2]
StorageBox.online_types = [3, 1, 4, 6]
StorageBox.options
StorageBox.storage_type
StorageBox.type_order = [3, 6, 1, 2, 4, 5]
class tardis.tardis_portal.models.storage.StorageBoxAttribute(*args, **kwargs)

Bases: django.db.models.base.Model

can hold attributes/metadata about different storage locations.

built-ins: key values description type receiving holds files temporarily for ingestion only

permanent permanent location (assumed by default) cache holds files for fast access
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception StorageBoxAttribute.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

StorageBoxAttribute.objects = <django.db.models.manager.Manager object>
StorageBoxAttribute.storage_box
class tardis.tardis_portal.models.storage.StorageBoxOption(*args, **kwargs)

Bases: django.db.models.base.Model

holds the options passed to the storage class defined in StorageBox. key->value store with support for typed values through pickling when value_type is set to ‘pickle’

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception StorageBoxOption.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

StorageBoxOption.PICKLE = 'pickle'
StorageBoxOption.STRING = 'string'
StorageBoxOption.TYPE_CHOICES = (('string', 'String value'), ('pickle', 'Pickled value'))
StorageBoxOption.get_value_type_display(*moreargs, **morekwargs)
StorageBoxOption.objects = <django.db.models.manager.Manager object>
StorageBoxOption.storage_box
StorageBoxOption.unpickled_value
tardis.tardis_portal.models.token module
class tardis.tardis_portal.models.token.Token(id, token, experiment, expiry_date, user)

Bases: django.db.models.base.Model

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Token.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Token.experiment
Token.get_absolute_url(*args, **kwargs)
Token.get_next_by_expiry_date(*moreargs, **morekwargs)
Token.get_previous_by_expiry_date(*moreargs, **morekwargs)
Token.get_session_expiry()

A token login should expire at the earlier of a) tomorrow at 4am b) the (end of) the token’s expiry date

It is the responsibility of token_auth to set the session expiry

Token.is_expired()
Token.objects = <tardis.tardis_portal.managers.OracleSafeManager object>
Token.save_with_random_token()
Token.user
Module contents

models/__init__.py

tardis.tardis_portal.publish package
Subpackages
tardis.tardis_portal.publish.provider package
Submodules
tardis.tardis_portal.publish.provider.rifcsprovider module
class tardis.tardis_portal.publish.provider.rifcsprovider.RifCsProvider

Bases: object

can_publish(experiment)
get_rifcs_context(experiment)
get_template(experiment)
is_schema_valid(experiment)
tardis.tardis_portal.publish.provider.schemarifcsprovider module
Module contents
Submodules
tardis.tardis_portal.publish.publishservice module
class tardis.tardis_portal.publish.publishservice.PublishService(providers, experiment)
get_context()
get_template()
manage_rifcs(oaipath)
Module contents
tardis.tardis_portal.storage package
Submodules
tardis.tardis_portal.storage.file_system module
class tardis.tardis_portal.storage.file_system.MyTardisLocalFileSystemStorage(location=None, base_url=None)

Bases: django.core.files.storage.FileSystemStorage

Simply changes the FileSystemStorage default store location to the MyTardis file store location. Makes it easier to migrate 2.5 installations.

tardis.tardis_portal.storage.squashfs module

SquashFS Storage Box

setup requirements: user-owned mount root dir mytardis user must have access to files in squashfs file based on immutable permissions set in squashfile. In easy cases, one group owns everything and the mytardis user can be added to that group.

set up autofs to auto-mount squashfiles

class tardis.tardis_portal.storage.squashfs.SquashFSStorage(sq_filename=None, datafile_id=None, sq_dir=None)

Bases: django.core.files.storage.FileSystemStorage

Only works for autofs mounted squashfs files Please provide a setting that maps source dir and mount dir, e.g.: SQUASHFS_DIRS = {‘tape’: {‘source’: ‘/my/tape/mount’,

‘autofs’: ‘/taped-squashfiles’},
‘volume’: {‘source’: ‘/my/volume/storage’,
‘autofs’: ‘/volumed-squashfiles’}}
squashfs_dirs = None
walk(top='.', topdown=True, onerror=None, ignore_dotfiles=True)
tardis.tardis_portal.storage.squashfs.get_parse_status(squash_datafile, ns)
Module contents
class tardis.tardis_portal.storage.DummyStorage

Bases: django.core.files.storage.Storage

Does nothing except serve as a place holder for Storage classes not implemented yet

exception tardis.tardis_portal.storage.MisconfiguredStorageError

Bases: exceptions.Exception

tardis.tardis_portal.templatetags package
Submodules
tardis.tardis_portal.templatetags.basiccomparisonfilters module
tardis.tardis_portal.templatetags.basiccomparisonfilters.gt(value, arg)

Returns a boolean of whether the value is greater than the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.gte(value, arg)

Returns a boolean of whether the value is greater than or equal to the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.length_gt(value, arg)

Returns a boolean of whether the value’s length is greater than the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.length_gte(value, arg)

Returns a boolean of whether the value’s length is greater than or equal to the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.length_lt(value, arg)

Returns a boolean of whether the value’s length is less than the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.length_lte(value, arg)

Returns a boolean of whether the value’s length is less than or equal to the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.lt(value, arg)

Returns a boolean of whether the value is less than the argument.

tardis.tardis_portal.templatetags.basiccomparisonfilters.lte(value, arg)

Returns a boolean of whether the value is less than or equal to the argument.

tardis.tardis_portal.templatetags.bleach_tag module
tardis.tardis_portal.templatetags.bleach_tag.bleach_value(value)
tardis.tardis_portal.templatetags.capture module
class tardis.tardis_portal.templatetags.capture.CaptureNode(nodelist, varname)

Bases: django.template.base.Node

render(context)
tardis.tardis_portal.templatetags.capture.capture(parser, token)

{% capture as [foo] %}

tardis.tardis_portal.templatetags.dataset_tags module
tardis.tardis_portal.templatetags.dynurl module
class tardis.tardis_portal.templatetags.dynurl.DynUrlNode(*args)

Bases: django.template.base.Node

render(context)
tardis.tardis_portal.templatetags.dynurl.dynurl(parser, token)
tardis.tardis_portal.templatetags.experiment_tags module
tardis.tardis_portal.templatetags.experimentstats module
tardis.tardis_portal.templatetags.experimentstats.experiment_file_count(value)
tardis.tardis_portal.templatetags.facility_tags module
tardis.tardis_portal.templatetags.facility_tags.check_if_facility_manager(request)

Custom template filter to identify whether a user is a facility manager.

tardis.tardis_portal.templatetags.feed module
tardis.tardis_portal.templatetags.feed.todatetime(value)
tardis.tardis_portal.templatetags.formfieldfilters module

This module holds filters that can be used in postprocessing a form field.

@author: Gerson Galang

tardis.tardis_portal.templatetags.formfieldfilters.parametername_form(value)

Removes all values of arg from the given string

tardis.tardis_portal.templatetags.formfieldfilters.sanitize_html(html, bad_tags=['body'])

Removes identified malicious HTML content from the given string.

tardis.tardis_portal.templatetags.formfieldfilters.size(value, actualSize)

Add the size attribute to the text field.

tardis.tardis_portal.templatetags.lookupfilters module
tardis.tardis_portal.templatetags.lookupfilters.get_item(dictionary, key)

Returns a value from a dictionary.

tardis.tardis_portal.templatetags.uploadify_tags module
tardis.tardis_portal.templatetags.uploadify_tags.multi_file_upload(context, upload_complete_url, dataset_id)

Displays a Flash-based interface for uploading multiple files. When all files have been uploaded, the given URL is POSTed to. The returned page replaces (AJAX) the upload interface.

  • filesUploaded - The total number of files uploaded
  • errors - The total number of errors while uploading
  • allBytesLoaded - The total number of bytes uploaded
  • speed - The average speed of all uploaded files
tardis.tardis_portal.templatetags.xmldate module
tardis.tardis_portal.templatetags.xmldate.toxmldatetime(value)
Module contents
tardis.tardis_portal.tests package
Subpackages
tardis.tardis_portal.tests.auth package
Submodules
tardis.tardis_portal.tests.auth.test_httpbasicendpoint_auth module
Module contents
tardis.tardis_portal.tests.filters package
Submodules
tardis.tardis_portal.tests.filters.test_jeolsem module
class tardis.tardis_portal.tests.filters.test_jeolsem.JEOLSEMFilterTestCase(methodName='runTest')

Bases: django.test.testcases.TransactionTestCase

setUp()
testJEOLComplex()
testJEOLSimple()
tardis.tardis_portal.tests.filters.test_middleware module
class tardis.tardis_portal.tests.filters.test_middleware.Filter1
classmethod getTuples()
tuples = []
class tardis.tardis_portal.tests.filters.test_middleware.Filter2
classmethod getTuples()
tuples = []
class tardis.tardis_portal.tests.filters.test_middleware.FilterInitTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
testFiltering()
Module contents
Submodules
tardis.tardis_portal.tests.ldap_ldif module
tardis.tardis_portal.tests.mock_vbl_auth module

Created on 24/01/2011

@author: gerson

class tardis.tardis_portal.tests.mock_vbl_auth.MockBackend

Authenticate against the VBL SOAP Webservice. It is assumed that the request object contains the username and password to be provided to the VBLgetExpIDs function.

a new local user is created if it doesn’t already exist

if the authentication succeeds, the session will contain a VBL session key used for downloads as well as the user’s EPN list

authenticate(request)
get_user(user_id)
tardis.tardis_portal.tests.mock_vbl_download module
tardis.tardis_portal.tests.mock_vbl_download.download_datafile(request, *args, **kwargs)

Dummy view for vbl download tests

tardis.tardis_portal.tests.slapd module

Utilities for starting up a test slapd server and talking to it with ldapsearch/ldapadd.

class tardis.tardis_portal.tests.slapd.Slapd

Controller class for a slapd instance, OpenLDAP’s server.

This class creates a temporary data store for slapd, runs it on a private port, and initialises it with a top-level dc and the root user.

When a reference to an instance of this class is lost, the slapd server is shut down.

PATH_LDAPADD = None
PATH_LDAPSEARCH = None
PATH_SCHEMA_DIR = '/home/docs/checkouts/readthedocs.org/user_builds/mytardis/checkouts/develop/tardis/tardis_portal/tests/ldap_schemas/'
PATH_SLAPD = None
PATH_SLAPTEST = None
PATH_TMPDIR = '/tmp/tmpT8GitO'
TEST_UTILS_DIR = '/home/docs/checkouts/readthedocs.org/user_builds/mytardis/checkouts/develop/tardis/tardis_portal/tests'
classmethod check_paths()

Checks that the configured executable paths look valid. If they don’t, then logs warning messages (not errors).

configure(cfg)

Appends slapd.conf configuration lines to cfg. Also re-initializes any backing storage. Feel free to subclass and override this method.

get_address()
get_dn_suffix()
get_root_dn()
get_root_password()
get_tmpdir()
get_url()
ldapadd(ldif, extra_args=[])

Runs ldapadd on this slapd instance, passing it the ldif content

ldapsearch(base=None, filter='(objectClass=*)', attrs=[], scope='sub', extra_args=[])
restart()

Restarts the slapd server; ERASING previous content. Starts the server even it if isn’t already running.

set_debug()
set_dn_suffix(dn)
set_port(port)
set_root_cn(cn)
set_root_password(pw)
set_slapd_debug_level(level)
set_tmpdir(path)
start()

Starts the slapd server process running, and waits for it to come up.

started()

This method is called when the LDAP server has started up and is empty. By default, this method adds the two initial objects, the domain object and the root user object.

stop()

Stops the slapd server, and waits for it to terminate

wait()

Waits for the slapd process to terminate by itself.

tardis.tardis_portal.tests.slapd.delete_directory_content(path)
tardis.tardis_portal.tests.slapd.find_available_tcp_port(host='127.0.0.1')
tardis.tardis_portal.tests.slapd.mkdirs(path)

Creates the directory path unless it already exists

tardis.tardis_portal.tests.slapd.quote(s)

Quotes the ” and characters in a string and surrounds with ”...”

tardis.tardis_portal.tests.slapd.which(executable)
tardis.tardis_portal.tests.test_ands_doi module

test_ands_doi.py

class tardis.tardis_portal.tests.test_ands_doi.ANDSDOITestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_get_doi_none()
test_init()
tardis.tardis_portal.tests.test_api module

Testing the tastypie-based mytardis api

class tardis.tardis_portal.tests.test_api.ACLAuthorizationTest(methodName='runTest')

Bases: django.test.testcases.TestCase

class tardis.tardis_portal.tests.test_api.DataFileResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

setUp()
test_post_single_file()
test_shared_fs_many_files()

tests sending many files with known permanent location (useful for Australian Synchrotron ingestions)

test_shared_fs_single_file()
class tardis.tardis_portal.tests.test_api.DatafileParameterResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.DatafileParameterSetResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.DatasetParameterResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.DatasetParameterSetResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.DatasetResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

setUp()
test_get_dataset_no_instrument()
test_get_dataset_with_instrument()
test_post_dataset()
class tardis.tardis_portal.tests.test_api.ExperimentParameterResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.ExperimentParameterSetResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.ExperimentResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

setUp()
test_get_experiment()
test_post_experiment()
class tardis.tardis_portal.tests.test_api.FacilityResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

test_get_facility_by_id()
test_get_facility_by_manager_group_id()

This type of query can be used to iterate through a user’s groups, and use each group’s id to determine which facilities a user manages, i.e. a way to obtain the functionality implemented by

tardis.tardis_portal.models.facility.facilities_managed_by

via the API

test_get_facility_by_name()
class tardis.tardis_portal.tests.test_api.GroupResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

test_get_group_by_id()
test_get_group_by_name()
class tardis.tardis_portal.tests.test_api.InstrumentResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

test_get_instrument_by_id()
test_get_instrument_by_name()
test_post_instrument()
test_rename_instrument()
class tardis.tardis_portal.tests.test_api.LocationResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.MyTardisAuthenticationTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

test_apikey_authentication()
test_bad_credentials()
class tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase(methodName='runTest')

Bases: tastypie.test.ResourceTestCase

abstract class without tests to combine common settings in one place

get_apikey_credentials()
get_credentials()
setUp()
class tardis.tardis_portal.tests.test_api.ReplicaResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.SchemaResourceTest(methodName='runTest')

Bases: tardis.tardis_portal.tests.test_api.MyTardisResourceTestCase

class tardis.tardis_portal.tests.test_api.SerializerTest(methodName='runTest')

Bases: django.test.testcases.TestCase

test_debug_serializer()
test_pretty_serializer()
tardis.tardis_portal.tests.test_authentication module

Created on 19/01/2011

class tardis.tardis_portal.tests.test_authentication.AuthenticationTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
testCreateNewAccount()
testManageAuthMethods()
testSimpleAuthenticate()
test_djangoauth()
tardis.tardis_portal.tests.test_authorisation module
class tardis.tardis_portal.tests.test_authorisation.ObjectACLTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testCantAddTokenuserToGroups()
testCantEditLockedExperiment()
testChangeUserPermissions()
testOwnedExperiments()
testReadAccess()
testWriteAccess()
urls = 'tardis.urls'
tardis.tardis_portal.tests.test_authservice module
class tardis.tardis_portal.tests.test_authservice.AuthServiceTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testAuthenticate()
testGetGroupsForEntity()
testGroupProvider()
testGroupSearch()
testInitialisation()
urls = 'tardis.tardis_portal.tests.urls'
class tardis.tardis_portal.tests.test_authservice.MockAuthProvider
authenticate(request)
class tardis.tardis_portal.tests.test_authservice.MockGroupProvider

Bases: tardis.tardis_portal.auth.interfaces.GroupProvider

getGroupById(id)
getGroups(user)
getGroupsForEntity(id)
searchGroups(**filter)
class tardis.tardis_portal.tests.test_authservice.MockRequest

Bases: django.http.request.HttpRequest

setPost(field, value)
class tardis.tardis_portal.tests.test_authservice.MockSettings

Bases: object

tardis.tardis_portal.tests.test_collectstatic module
class tardis.tardis_portal.tests.test_collectstatic.CollectstaticTest(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_collectstatic()
tardis.tardis_portal.tests.test_download module
class tardis.tardis_portal.tests.test_download.DownloadTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testDatasetFile()
testDownload()
testView()
tardis.tardis_portal.tests.test_download.get_size_and_sha512sum(testfile)
tardis.tardis_portal.tests.test_download_apikey module
class tardis.tardis_portal.tests.test_download_apikey.ApiKeyDownloadTestCase(methodName='runTest')

Bases: tastypie.test.ResourceTestCase

setUp()
tearDown()
test_download_apikey()
tardis.tardis_portal.tests.test_fetcher module
class tardis.tardis_portal.tests.test_fetcher.PrivilegedOpenerTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

classmethod setUpClass()
classmethod tearDownClass()
testCredentials()

Test that the walker manages credentials.

class tardis.tardis_portal.tests.test_fetcher.TestWebServer

Utility class for running a test web server with a given handler.

class QuietSimpleHTTPRequestHandler(request, client_address, server)

Bases: SimpleHTTPServer.SimpleHTTPRequestHandler

Simple subclass that only prints output to STDOUT, not STDERR

do_GET()
log_message(msg, *args)
class TestWebServer.ThreadedTCPServer(server_address, RequestHandlerClass, bind_and_activate=True)

Bases: SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer

classmethod TestWebServer.getPort()
TestWebServer.getUrl()
TestWebServer.start()
TestWebServer.stop()
tardis.tardis_portal.tests.test_forms module

test_models.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.test_forms.RightsFormTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
test_ensures_suitable_license()
test_needs_confirmation()
tardis.tardis_portal.tests.test_iiif module
class tardis.tardis_portal.tests.test_iiif.ExtraTestCases(methodName='runTest')

Bases: django.test.testcases.TestCase

As per: http://library.stanford.edu/iiif/image-api/compliance.html

setUp()
testImageCacheControl()
testImageHasEtags()
testInfoHasEtags()
class tardis.tardis_portal.tests.test_iiif.Level0TestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

As per: http://library.stanford.edu/iiif/image-api/compliance.html

setUp()
testCanGetInfoAsJSON()
testCanGetInfoAsXML()
testCanGetOriginalImage()
class tardis.tardis_portal.tests.test_iiif.Level1TestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

As per: http://library.stanford.edu/iiif/image-api/compliance.html

setUp()
testCanGetJpegFormat()
testHandleRegions()
testHandleRotation()
testHandleSizing()
class tardis.tardis_portal.tests.test_iiif.Level2TestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

As per: http://library.stanford.edu/iiif/image-api/compliance.html

setUp()
testCanGetRequiredFormats()
testHandleSizing()
tardis.tardis_portal.tests.test_ldap module
class tardis.tardis_portal.tests.test_ldap.LDAPErrorTest(methodName='runTest')

Bases: django.test.testcases.TestCase

class tardis.tardis_portal.tests.test_ldap.LDAPTest(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_authenticate()
test_getgroupbyid()
test_getgroups()
test_getgroupsforentity()
test_getuserbyid()
test_searchgroups()
tardis.tardis_portal.tests.test_models module

test_models.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.test_models.ModelTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
test_authors()
test_create_user_automatically_generate_api_key()
test_datafile()
test_dataset()
test_experiment()
test_parameter()
tardis.tardis_portal.tests.test_parametersetmanager module

test_views.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.test_parametersetmanager.ParameterSetManagerTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_existing_parameterset()

Test that Parameter.link_gfk (GenericForeignKey) is correctly assigned after using Parameter.set_value(some_url) for a LINK Parameter.

test_new_parameterset()
test_tz_aware_date_handling()

Ensure that dates are handling in a timezone-aware way.

test_tz_naive_date_handling()

Ensure that dates are handling in a timezone-aware way.

Test that LINK Parameters that can’t be resolved to a model (including non-URL values) still work.

tardis.tardis_portal.tests.test_publishservice module
class tardis.tardis_portal.tests.test_publishservice.MockRifCsProvider

Bases: tardis.tardis_portal.publish.provider.rifcsprovider.RifCsProvider

get_beamline(experiment)
get_license_uri(experiment)
get_rifcs_context(experiment)
get_template(experiment)
is_schema_valid(experiment)
class tardis.tardis_portal.tests.test_publishservice.PublishServiceTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
testContext()
testInitialisation()
testInitialisationNoProvider()
testManageRifCsCheckContent()
testManageRifCsCreateAndRemove()
tardis.tardis_portal.tests.test_rmexperiment module
class tardis.tardis_portal.tests.test_rmexperiment.RmExperimentTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

testList()
testRemove()
tardis.tardis_portal.tests.test_staging module

test_views.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.test_staging.TestStagingFiles(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_stage_replica()
tardis.tardis_portal.tests.test_storage module

test_storage.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.test_storage.ModelTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_storageboxoption()
tardis.tardis_portal.tests.test_tar_download module
class tardis.tardis_portal.tests.test_tar_download.TarDownloadTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_tar_experiment_download()
tardis.tardis_portal.tests.test_tasks module
class tardis.tardis_portal.tests.test_tasks.BackgroundTaskTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

As per: http://library.stanford.edu/iiif/image-api/compliance.html

setUp()
testLocalFile()
test_wrong_size_verification()
tardis.tardis_portal.tests.test_tokens module

test_tokens.py

class tardis.tardis_portal.tests.test_tokens.FrozenTime(*args, **kwargs)
classmethod freeze_time(time)
classmethod now()
class tardis.tardis_portal.tests.test_tokens.TokenTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
test_default_expiry()
test_get_session_expiry()
test_get_session_expiry_expired_token()
test_get_session_expiry_near_expiry()
test_is_expired()
test_save_with_random_token()
test_save_with_random_token_failures()
test_save_with_random_token_gives_up()
urls = 'tardis.tardis_portal.tests.urls'
tardis.tardis_portal.tests.test_views module
tardis.tardis_portal.tests.tests module

tests.py http://docs.djangoproject.com/en/dev/topics/testing/

class tardis.tardis_portal.tests.tests.SearchTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

setUp()
tearDown()
testSearchDatafileAuthentication(*args, **kwargs)
testSearchDatafileForm(*args, **kwargs)
testSearchExperimentAuthentication(*args, **kwargs)
testSearchExperimentForm(*args, **kwargs)
testSearchExperimentResults(*args, **kwargs)
class tardis.tardis_portal.tests.tests.UserInterfaceTestCase(methodName='runTest')

Bases: django.test.testcases.TestCase

test_login()
test_root()
test_search_urls(*args, **kwargs)
test_urls()
test_urls_with_some_content()
tardis.tardis_portal.tests.tests.suite()
tardis.tardis_portal.tests.urls module
Module contents
tardis.tardis_portal.views package
Submodules
tardis.tardis_portal.views.ajax_actions module
tardis.tardis_portal.views.ajax_json module
tardis.tardis_portal.views.ajax_pages module
tardis.tardis_portal.views.authentication module
tardis.tardis_portal.views.authorisation module
tardis.tardis_portal.views.facilities module
tardis.tardis_portal.views.images module
tardis.tardis_portal.views.machine module
tardis.tardis_portal.views.pages module
tardis.tardis_portal.views.parameters module
tardis.tardis_portal.views.upload module
tardis.tardis_portal.views.utils module
Module contents
Submodules
tardis.tardis_portal.MultiPartForm module
class tardis.tardis_portal.MultiPartForm.MultiPartForm

Bases: object

Accumulate the data to be used when posting a form.

add_field(name, value)

Add a simple field to the form data.

add_file(fieldname, filename, fileHandle, mimetype=None)

Add a file to be uploaded.

get_content_type()
tardis.tardis_portal.ParameterSetManager module
class tardis.tardis_portal.ParameterSetManager.ParameterSetManager(parameterset=None, parentObject=None, schema=None)

Bases: object

blank_param = None
delete_all_params()
delete_params(parname)
get_param(parname, value=False)
get_params(parname, value=False)
get_schema()
new_param(parname, value, fullparname=None)
parameters = None
parameterset = None
set_param(parname, value, fullparname=None, example_value=None)
set_param_list(parname, value_list, fullparname=None)
set_params_from_dict(dict)
tardis.tardis_portal.admin module
class tardis.tardis_portal.admin.DataFileObjectInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
form

alias of DataFileObjectInlineForm

media
model

alias of DataFileObject

class tardis.tardis_portal.admin.DataFileObjectInlineForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of DataFileObject

widgets = {'uri': <django.forms.widgets.TextInput object>}
DataFileObjectInlineForm.base_fields = OrderedDict([('datafile', <django.forms.models.ModelChoiceField object>), ('storage_box', <django.forms.models.ModelChoiceField object>), ('uri', <django.forms.fields.CharField object>), ('verified', <django.forms.fields.BooleanField object>), ('last_verified_time', <django.forms.fields.DateTimeField object>)])
DataFileObjectInlineForm.declared_fields = OrderedDict()
DataFileObjectInlineForm.media
class tardis.tardis_portal.admin.DatafileAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

form

alias of DatafileAdminForm

inlines = [<class 'tardis.tardis_portal.admin.DataFileObjectInline'>]
media
search_fields = ['filename', 'id']
class tardis.tardis_portal.admin.DatafileAdminForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of DataFile

widgets = {'directory': <django.forms.widgets.TextInput object>}
DatafileAdminForm.base_fields = OrderedDict([('dataset', <django.forms.models.ModelChoiceField object>), ('filename', <django.forms.fields.CharField object>), ('directory', <django.forms.fields.CharField object>), ('size', <django.forms.fields.IntegerField object>), ('created_time', <django.forms.fields.DateTimeField object>), ('modification_time', <django.forms.fields.DateTimeField object>), ('mimetype', <django.forms.fields.CharField object>), ('md5sum', <django.forms.fields.CharField object>), ('sha512sum', <django.forms.fields.CharField object>), ('deleted', <django.forms.fields.BooleanField object>), ('deleted_time', <django.forms.fields.DateTimeField object>), ('version', <django.forms.fields.IntegerField object>)])
DatafileAdminForm.declared_fields = OrderedDict()
DatafileAdminForm.media
class tardis.tardis_portal.admin.DatasetAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

media
search_fields = ['description', 'id']
class tardis.tardis_portal.admin.ExperimentAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

inlines = [<class 'tardis.tardis_portal.admin.ObjectACLInline'>]
media
search_fields = ['title', 'id']
class tardis.tardis_portal.admin.ExperimentParameterInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
formfield_overrides = {<class 'django.db.models.fields.TextField'>: {'widget': <class 'django.forms.widgets.TextInput'>}}
media
model

alias of ExperimentParameter

class tardis.tardis_portal.admin.ExperimentParameterSetAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

inlines = [<class 'tardis.tardis_portal.admin.ExperimentParameterInline'>]
media
class tardis.tardis_portal.admin.FreeTextSearchFieldAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

media
class tardis.tardis_portal.admin.InstrumentParameterInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
media
model

alias of InstrumentParameter

class tardis.tardis_portal.admin.InstrumentParameterSetAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

inlines = [<class 'tardis.tardis_portal.admin.InstrumentParameterInline'>]
media
class tardis.tardis_portal.admin.ObjectACLAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

list_display = ['__unicode__', 'pluginId', 'entityId', 'canRead', 'canWrite', 'canDelete', 'isOwner']
media
search_fields = ['content_type', 'object_id']
class tardis.tardis_portal.admin.ObjectACLInline(parent_model, admin_site)

Bases: django.contrib.contenttypes.admin.GenericTabularInline

extra = 0
media
model

alias of ObjectACL

class tardis.tardis_portal.admin.ParameterNameAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

media
search_fields = ['name', 'schema__id']
class tardis.tardis_portal.admin.ParameterNameInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
media
model

alias of ParameterName

class tardis.tardis_portal.admin.SchemaAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

inlines = [<class 'tardis.tardis_portal.admin.ParameterNameInline'>]
media
search_fields = ['name', 'namespace']
class tardis.tardis_portal.admin.StorageBoxAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

form

alias of StorageBoxForm

inlines = [<class 'tardis.tardis_portal.admin.StorageBoxOptionInline'>, <class 'tardis.tardis_portal.admin.StorageBoxAttributeInline'>]
media
class tardis.tardis_portal.admin.StorageBoxAttributeInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
form

alias of StorageBoxAttributeInlineForm

media
model

alias of StorageBoxAttribute

class tardis.tardis_portal.admin.StorageBoxAttributeInlineForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of StorageBoxAttribute

widgets = {'value': <django.forms.widgets.TextInput object>, 'key': <django.forms.widgets.TextInput object>}
StorageBoxAttributeInlineForm.base_fields = OrderedDict([('storage_box', <django.forms.models.ModelChoiceField object>), ('key', <django.forms.fields.CharField object>), ('value', <django.forms.fields.CharField object>)])
StorageBoxAttributeInlineForm.declared_fields = OrderedDict()
StorageBoxAttributeInlineForm.media
class tardis.tardis_portal.admin.StorageBoxForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of StorageBox

widgets = {'name': <django.forms.widgets.TextInput object>, 'django_storage_class': <django.forms.widgets.TextInput object>, 'description': <django.forms.widgets.TextInput object>}
StorageBoxForm.base_fields = OrderedDict([('django_storage_class', <django.forms.fields.CharField object>), ('max_size', <django.forms.fields.IntegerField object>), ('status', <django.forms.fields.CharField object>), ('name', <django.forms.fields.CharField object>), ('description', <django.forms.fields.CharField object>), ('master_box', <django.forms.models.ModelChoiceField object>)])
StorageBoxForm.declared_fields = OrderedDict()
StorageBoxForm.media
class tardis.tardis_portal.admin.StorageBoxOptionInline(parent_model, admin_site)

Bases: django.contrib.admin.options.TabularInline

extra = 0
form

alias of StorageBoxOptionInlineForm

media
model

alias of StorageBoxOption

class tardis.tardis_portal.admin.StorageBoxOptionInlineForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False, instance=None)

Bases: django.forms.models.ModelForm

class Meta
fields = '__all__'
model

alias of StorageBoxOption

widgets = {'value': <django.forms.widgets.TextInput object>, 'key': <django.forms.widgets.TextInput object>}
StorageBoxOptionInlineForm.base_fields = OrderedDict([('storage_box', <django.forms.models.ModelChoiceField object>), ('key', <django.forms.fields.CharField object>), ('value', <django.forms.fields.CharField object>), ('value_type', <django.forms.fields.TypedChoiceField object>)])
StorageBoxOptionInlineForm.declared_fields = OrderedDict()
StorageBoxOptionInlineForm.media
tardis.tardis_portal.ands_doi module

ands_doi.py

class tardis.tardis_portal.ands_doi.DOIService(experiment)

Bases: object

Mints DOIs using ANDS’ Cite My Data service POSTs DataCite XML to a web services endpoint

get_doi()
Returns:DOI or None
Return type:string
get_or_mint_doi(url)
Parameters:url (string) – the URL the DOI will resolve to
Returns:the DOI string
Return type:string
class tardis.tardis_portal.ands_doi.DOIXMLProvider(experiment)

Bases: object

provides datacite XML metadata for a given experiment

datacite_xml()
Returns:datacite XML for self.experiment
Return type:string
tardis.tardis_portal.api module

RESTful API for MyTardis models and data. Implemented with Tastypie.

class tardis.tardis_portal.api.ACLAuthorization

Bases: tastypie.authorization.Authorization

Authorisation class for Tastypie.

create_detail(object_list, bundle)
create_list(object_list, bundle)
delete_detail(object_list, bundle)
delete_list(object_list, bundle)
read_detail(object_list, bundle)
read_list(object_list, bundle)
update_detail(object_list, bundle)
update_list(object_list, bundle)
class tardis.tardis_portal.api.DataFileResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

class Meta

Bases: tardis.tardis_portal.api.Meta

filtering = {'directory': ('exact', 'startswith'), 'filename': ('exact',), 'dataset': 2}
object_class

alias of DataFile

ordering = ['filename', 'modification_time']
queryset
resource_name = 'dataset_file'
DataFileResource.base_fields = {'sha512sum': <tastypie.fields.CharField object>, 'replicas': <tastypie.fields.ToManyField object>, 'deleted': <tastypie.fields.BooleanField object>, 'dataset': <tastypie.fields.ForeignKey object>, 'modification_time': <tastypie.fields.DateTimeField object>, 'created_time': <tastypie.fields.DateTimeField object>, u'id': <tastypie.fields.IntegerField object>, 'size': <tastypie.fields.CharField object>, 'mimetype': <tastypie.fields.CharField object>, 'datafile': <tastypie.fields.FileField object>, 'deleted_time': <tastypie.fields.DateTimeField object>, 'md5sum': <tastypie.fields.CharField object>, 'filename': <tastypie.fields.CharField object>, 'version': <tastypie.fields.IntegerField object>, 'directory': <tastypie.fields.CharField object>, 'parameter_sets': <tastypie.fields.ToManyField object>, u'resource_uri': <tastypie.fields.CharField object>}
DataFileResource.declared_fields = {'datafile': <tastypie.fields.FileField object>, 'dataset': <tastypie.fields.ForeignKey object>, 'parameter_sets': <tastypie.fields.ToManyField object>, 'replicas': <tastypie.fields.ToManyField object>}
DataFileResource.deserialize(request, data, format=None)

from https://github.com/toastdriven/django-tastypie/issues/42 modified to deserialize json sent via POST. Would fail if data is sent in a different format. uses a hack to get back pure json from request.POST

DataFileResource.download_file(request, **kwargs)

curl needs the -J switch to get the filename right auth needs to be added manually here

DataFileResource.hydrate(bundle)
DataFileResource.obj_create(bundle, **kwargs)

Creates a new DataFile object from the provided bundle.data dict.

If a duplicate key error occurs, responds with HTTP Error 409: CONFLICT

DataFileResource.post_list(request, **kwargs)
DataFileResource.prepend_urls()
DataFileResource.put_detail(request, **kwargs)

from https://github.com/toastdriven/django-tastypie/issues/42

DataFileResource.temp_url = None
DataFileResource.verify_file(request, **kwargs)

triggers verification of file, e.g. after non-POST upload complete

class tardis.tardis_portal.api.DatafileParameterResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of DatafileParameter

queryset
DatafileParameterResource.base_fields = {'name': <tastypie.fields.ForeignKey object>, 'string_value': <tastypie.fields.CharField object>, 'datetime_value': <tastypie.fields.DateTimeField object>, 'value': <tastypie.fields.CharField object>, 'link_id': <tastypie.fields.IntegerField object>, 'numerical_value': <tastypie.fields.FloatField object>, 'parameterset': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, u'resource_uri': <tastypie.fields.CharField object>}
DatafileParameterResource.declared_fields = {'parameterset': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.DatafileParameterSetResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterSetResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of DatafileParameterSet

queryset
DatafileParameterSetResource.base_fields = {'datafile': <tastypie.fields.ForeignKey object>, u'resource_uri': <tastypie.fields.CharField object>, u'id': <tastypie.fields.IntegerField object>, 'parameters': <tastypie.fields.ToManyField object>, 'schema': <tastypie.fields.ForeignKey object>}
DatafileParameterSetResource.declared_fields = {'datafile': <tastypie.fields.ForeignKey object>, 'parameters': <tastypie.fields.ToManyField object>}
class tardis.tardis_portal.api.DatasetParameterResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of DatasetParameter

queryset
DatasetParameterResource.base_fields = {'name': <tastypie.fields.ForeignKey object>, 'string_value': <tastypie.fields.CharField object>, 'datetime_value': <tastypie.fields.DateTimeField object>, 'value': <tastypie.fields.CharField object>, 'link_id': <tastypie.fields.IntegerField object>, 'numerical_value': <tastypie.fields.FloatField object>, 'parameterset': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, u'resource_uri': <tastypie.fields.CharField object>}
DatasetParameterResource.declared_fields = {'parameterset': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.DatasetParameterSetResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterSetResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of DatasetParameterSet

queryset
DatasetParameterSetResource.base_fields = {'dataset': <tastypie.fields.ForeignKey object>, u'resource_uri': <tastypie.fields.CharField object>, u'id': <tastypie.fields.IntegerField object>, 'parameters': <tastypie.fields.ToManyField object>, 'schema': <tastypie.fields.ForeignKey object>}
DatasetParameterSetResource.declared_fields = {'parameters': <tastypie.fields.ToManyField object>, 'dataset': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.DatasetResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

class Meta

Bases: tardis.tardis_portal.api.Meta

always_return_data = True
filtering = {'directory': ('exact',), 'experiments': 2, 'id': ('exact',), 'description': ('exact',)}
object_class

alias of Dataset

ordering = ['description']
queryset
DatasetResource.base_fields = {'description': <tastypie.fields.CharField object>, 'immutable': <tastypie.fields.BooleanField object>, u'id': <tastypie.fields.IntegerField object>, 'instrument': <tastypie.fields.ForeignKey object>, 'experiments': <tastypie.fields.ToManyField object>, 'directory': <tastypie.fields.CharField object>, 'parameter_sets': <tastypie.fields.ToManyField object>, u'resource_uri': <tastypie.fields.CharField object>}
DatasetResource.declared_fields = {'instrument': <tastypie.fields.ForeignKey object>, 'experiments': <tastypie.fields.ToManyField object>, 'parameter_sets': <tastypie.fields.ToManyField object>}
DatasetResource.get_datafiles(request, **kwargs)
DatasetResource.prepend_urls()
class tardis.tardis_portal.api.ExperimentParameterResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of ExperimentParameter

queryset
ExperimentParameterResource.base_fields = {'name': <tastypie.fields.ForeignKey object>, 'string_value': <tastypie.fields.CharField object>, 'datetime_value': <tastypie.fields.DateTimeField object>, 'value': <tastypie.fields.CharField object>, 'link_id': <tastypie.fields.IntegerField object>, 'numerical_value': <tastypie.fields.FloatField object>, 'parameterset': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, u'resource_uri': <tastypie.fields.CharField object>}
ExperimentParameterResource.declared_fields = {'parameterset': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.ExperimentParameterSetResource(api_name=None)

Bases: tardis.tardis_portal.api.ParameterSetResource

API for ExperimentParameterSets

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of ExperimentParameterSet

queryset
ExperimentParameterSetResource.base_fields = {u'id': <tastypie.fields.IntegerField object>, u'resource_uri': <tastypie.fields.CharField object>, 'experiment': <tastypie.fields.ForeignKey object>, 'parameters': <tastypie.fields.ToManyField object>, 'schema': <tastypie.fields.ForeignKey object>}
ExperimentParameterSetResource.declared_fields = {'experiment': <tastypie.fields.ForeignKey object>, 'parameters': <tastypie.fields.ToManyField object>}
class tardis.tardis_portal.api.ExperimentResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

API for Experiments also creates a default ACL and allows ExperimentParameterSets to be read and written.

TODO: catch duplicate schema submissions for parameter sets

class Meta

Bases: tardis.tardis_portal.api.Meta

always_return_data = True
filtering = {'id': ('exact',), 'title': ('exact',)}
object_class

alias of Experiment

ordering = ['title', 'created_time', 'update_time']
queryset
ExperimentResource.base_fields = {'update_time': <tastypie.fields.DateTimeField object>, 'handle': <tastypie.fields.CharField object>, 'description': <tastypie.fields.CharField object>, 'start_time': <tastypie.fields.DateTimeField object>, 'created_time': <tastypie.fields.DateTimeField object>, 'public_access': <tastypie.fields.IntegerField object>, 'approved': <tastypie.fields.BooleanField object>, 'locked': <tastypie.fields.BooleanField object>, 'title': <tastypie.fields.CharField object>, 'url': <tastypie.fields.CharField object>, 'institution_name': <tastypie.fields.CharField object>, 'created_by': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, 'end_time': <tastypie.fields.DateTimeField object>, 'parameter_sets': <tastypie.fields.ToManyField object>, u'resource_uri': <tastypie.fields.CharField object>}
ExperimentResource.declared_fields = {'parameter_sets': <tastypie.fields.ToManyField object>, 'created_by': <tastypie.fields.ForeignKey object>}
ExperimentResource.dehydrate(bundle)
ExperimentResource.hydrate_m2m(bundle)

create ACL before any related objects are created in order to use ACL permissions for those objects.

ExperimentResource.obj_create(bundle, **kwargs)

experiments need at least one ACL to be available through the ExperimentManager (Experiment.safe) Currently not tested for failed db transactions as sqlite does not enforce limits.

ExperimentResource.obj_get_list(bundle, **kwargs)

responds to EPN query for Australian Synchrotron

class tardis.tardis_portal.api.FacilityResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

class Meta

Bases: tardis.tardis_portal.api.Meta

always_return_data = True
filtering = {'manager_group': 2, 'id': ('exact',), 'name': ('exact',)}
object_class

alias of Facility

queryset
FacilityResource.base_fields = {'manager_group': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, 'name': <tastypie.fields.CharField object>, u'resource_uri': <tastypie.fields.CharField object>}
FacilityResource.declared_fields = {'manager_group': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.GroupResource(api_name=None)

Bases: tastypie.resources.ModelResource

class Meta
authentication = <tardis.tardis_portal.api.MyTardisAuthentication object>
authorization

Authorisation class for Tastypie.

filtering = {'id': ('exact',), 'name': ('exact',)}
object_class

alias of Group

queryset
GroupResource.base_fields = {u'id': <tastypie.fields.IntegerField object>, 'name': <tastypie.fields.CharField object>, u'resource_uri': <tastypie.fields.CharField object>}
GroupResource.declared_fields = {}
class tardis.tardis_portal.api.InstrumentResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

class Meta

Bases: tardis.tardis_portal.api.Meta

always_return_data = True
filtering = {'id': ('exact',), 'name': ('exact',), 'facility': 2}
object_class

alias of Instrument

queryset
InstrumentResource.base_fields = {'facility': <tastypie.fields.ForeignKey object>, u'id': <tastypie.fields.IntegerField object>, 'name': <tastypie.fields.CharField object>, u'resource_uri': <tastypie.fields.CharField object>}
InstrumentResource.declared_fields = {'facility': <tastypie.fields.ForeignKey object>}
class tardis.tardis_portal.api.LocationResource(api_name=None)

Bases: tardis.tardis_portal.api.MyTardisModelResource

class Meta

Bases: tardis.tardis_portal.api.Meta

object_class

alias of StorageBox

queryset
LocationResource.base_fields = {'status': <tastypie.fields.CharField object>, 'description': <tastypie.fields.CharField object>, 'name': <tastypie.fields.CharField object>, 'max_size': <tastypie