nosedbreport : save the smell

Introduction

nosedbreport exposes a single plugin that can front various backend databases to store the result of a nose test execution. Having the results of your tests, whether they are part of a continuous integration system or not, allows you to ask interesting questions about your project such as

  • What were the test suites that ran in the last five minutes
  • What is the average time to run test case ‘x’
  • What is the standard time to failure for test suite ‘y’
  • and so on...

These questions also allow you to build reporting, and monitoring tools based on automated functional tests that you may be running against your development, staging or production systems, such as heartbeat or availability pages.

Installation travis-ci

  • with easy_install

    sudo easy_install nosedbreport
    
  • with pip

    sudo pip install nosedbreport
    
  • from source (git repository):

    hg clone http://github.com/alisaifee/nosedbreport
    cd nosedbreport
    python setup.py build
    sudo python setup.py install
    

Usage

  • The most basic use case is to report the results of a test run into a mysql database, which can be achieved by adding the following options to your nose execution:

    nosetests --dbreport-dbtype=mysql --dbreport-host=your.mysql.com\
     --dbreport-username=ali --dbreport-password=some-pass --dbreport-db=nosereport
    
  • To create the appropriate schema in your mysql database:

    nosetests --dbreport-dbtype=mysql --dbreport-host=your.mysql.com\
     --dbreport-username=root  --dbreport-password=your-root-pass\
     --dbreport-db=nosereport --dbreport-create-schema
    
  • For detailed usage refer to read the docs

Development

Contributing

The source is maintained in a github repository. Feel free to fork it to modify/extend the plugin. Currently, the plugin is only backed by a MySQL & SQLite connector, but it can be easily extended to support other databases.

To add a new database connector, you will need to:

  • add a new class that extends NoseDBReporterBase

  • implement the configure(), startTest(), report() and construct_schema() methods. (For an example see the MySQL implementation in NoseMySQLReporter)

  • add your new class to the connectors in plugin in the following way:

    import newconnector
    connectors = {
                  "newconnector" : newconnector.NoseNewConnectorReporter,
                  "mysql" : mysql.NoseMySQLReporter
                }
    
  • this will make the newconnector available with the command line option –dbreport_dbtype=newconnector

Class Structure

Inheritance diagram of nosedbreport.plugin, nosedbreport.base, nosedbreport.mysql, nosedbreport.sqlite

Source Documentation

class nosedbreport.plugin.NoseDBReporter[source]

The main plugin that is loaded by nose.plugin.PluginManager

configure(options, conf)[source]

Configure plugin. Plugin is disabled by default.

connectors = {'sqlite': <class 'nosedbreport.sqlite.NoseSQLiteReporter'>, 'mysql': <class 'nosedbreport.mysql.NoseMySQLReporter'>}

list of db connectors available for use when specifying db_type.

options(parser, env)[source]

Register commandline options

class nosedbreport.base.NoseDBReporterBase[source]

Base class for Nose plugins that stash test results into a database.

addError(test, err, capt=None)[source]

sets the status of the test to either ‘skipped’ or ‘error’, collects the trace and time taken to execute.

addFailure(test, err, capt=None, tb_info=None)[source]

sets the status of the test to ‘fail’, collects the trace and time taken to execute.

addSuccess(test, capt=None)[source]

sets the status of the test to ‘pass’, and sets the time taken to execute.

get_full_doc(test)[source]

via various nasty inspection methods, return the full docstring of the test being executed now.

startTest(test)[source]

collect information about a test before it begins, and initialize a timer to record time taken.

test_case_results = None

dictionary to keep track of individual test case executions, including status, time taken and tracebacks.

test_suites = None

dictionary to keep track of the overall suite results

class nosedbreport.mysql.NoseMySQLReporter[source]

MySQL Connector. Reports the results of each test run into the tables testcase, testsuite, testcaseexecution and testsuiteexecution

configure(options, conf)[source]

sets up the MySQL database connection based on the options provided on the command line.

construct_schema()[source]

called when the –dbreport_create_schema command option is passed to the plugin to create the mysql table schema.

report(stream)[source]

After successful completion of a nose run, perform the final reporting of the test results to the MySQL database.

startTest(test)[source]

record initiation of a test case. Update the last start time of the test suite & test case.

class nosedbreport.sqlite.NoseSQLiteReporter[source]

SQLLite Connector. Reports the results of each test run into the tables testcase, testsuite,``testcaseexecution`` and testsuiteexecution

configure(options, conf)[source]

sets up the sqlite database connection

construct_schema()[source]

called when the –dbreport_create_schema command option is passed to the plugin to create the sqlite table schema.

report(stream)[source]

After successful completion of a nose run, perform the final reporting of the test results to the sqlite database.

startTest(test)[source]

record initiation of a test case (test). Update the last start time of the test suite & test case.

Data Structures

Overview

By extending NoseDBReporterBase you essentially have access to two relevant dictionaries, test_suites and test_case_results.

MySQL Example

The NoseMySQLReporter backend translates the data structures of the plugin into mysql tables using 4 tables:

  • testsuite
  • testsuiteexecution
  • testcase
  • testcaseexecution

The MySQL tables are roughly related as follows:

digraph MySQLDb {
graph [
        rankdir="LR",
        fontname="Courier"
];
"testsuite" [
        label="<f0>testsuite|<f1>name",
        shape="record"
        ];
"testcase" [
        label="<f0>testcase|<f1>id|<f2>suite",
        shape="record"
        ];
"testsuiteexecution" [
        label="<f0>testsuiteexecution|<f1>id|<f2>suite",
        shape="record"
        ];
"testcaseexecution" [
        label="<f0>testcaseexecution|<f1>id|<f2>testcase",
        shape="record"
        ];
"testcase":f2->"testsuite":f1;
"testcaseexecution":f2->"testcase":f1;
"testsuiteexecution":f2->"testsuite":f1;
}