Welcome to OpenREM’s documentation!

OpenREM logo

OpenREM is an opensource framework created for the purpose of radiation exposure monitoring. The software is capable of importing and displaying data from a wide variety of x-ray dose related sources, and then enables easy export of the data in a form that is suitable for further analysis by suitably qualified medical physics personnel.

Please see openrem.org for more details.

Contents:

Installation

Pre-installation preparations

Install Python 2.7.x and pip

  • Linux – likely to be installed already
  • Windows – instructions and downloads are available at python.org
Add Python and the scripts folder to the path

Windows only – this is usually automatic in linux

During the Windows Python 2.7 installation, ‘install’ Add Python.exe to Path:

Add Python to Path image

Python installation customisation dialogue

If Python is already installed, you can add Python to Path yourself:

Add the following to the end of the path environment variable (to see how to edit the environment variables, see http://www.computerhope.com/issues/ch000549.htm):

;C:\Python27\;C:\Python27\Scripts\
Setuptools and pip

Install setuptools and pip – for details go to http://www.pip-installer.org/en/latest/installing.html. The quick version is as follows:

Linux

Download the latest version using the same method as for Windows, or get the version in your package manager, for example:

sudo apt-get install python-pip

Windows

Pip is normally installed with Python. If it hasn’t been, download the installer script get-pip.py and save it locally – right click and Save link as… or equivalent.

Open a command window (Start menu, cmd.exe) and navigate to the place you saved the get‑pip.py file:

python get-pip.py
Quick check of python and pip

To check everything is installed correctly so far, type the following in a command window/shell. You should have the version number of pip returned to you:

pip -V

Install RabbitMQ

For either install, just follow the defaults – no special configurations required. Please note that RabbitMQ requires that Erlang is installed first, as described in the above links.

Note

If you encounter problems running RabbitMQ as a service under Windows then try the following:

  • Create a folder called c:\rabbitmq

  • From an administrator command prompt run Advanced System Properties by typing sysdm.cpl

  • Create a new system environment variable called RABBITMQ_BASE and set its value to c:\rabbitmq

  • In the command prompt navigate to the folder containing the RabbitMQ commands and run:

    rabbitmq_service.bat remove
    rabbitmq_service.bat install
    rabbitmq_service.bat start
    

Note

Before continuing, consider virtualenv

Install NumPy

Numpy is required for charts.

For linux:

sudo apt-get install python-numpy
# If using a virtualenv, you might need to also do:
pip install numpy

For Windows:

Download NumPy from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy

  • Find the right version - look for numpy-X+mkl-cp27-cp27m-win32.whl for 32-bit Windows or
  • numpy-X+mkl-cp27-cp27m-win_amd64.whl for 64-bit Windows.
  • At the time of writing, X was 1.14.0 - choose the latest version
  • Install using pip:
pip install C:\path\to\numpy‑1.14.0+mkl‑cp27-cp27m‑win32.whl  # update the version number
# or
pip install C:\path\to\numpy‑1.14.0+mkl‑cp27‑cp27m‑win_amd64.whl  # update the version number
# changing the path and filename to the numpy appropriately

Install PostgreSQL database

For production use, you will need to install and configure a database. We strongly recommend PostgreSQL, but you can use any of the databases listed on the Django website such as MySQL, Oracle or MS SQL Server, with the limitations listed there. There is one additional limitation - the calculation of median values for charts in OpenREM is dependent on using PostgreSQL.

If this is your first time installing OpenREM and you just want to test it out, you can skip this step and make use of the in-built SQLite database. However, you should expect to start again when you move to a production grade database.

Install a DICOM Store service

To have modalities send DICOM objects to your OpenREM server, or to use query-retrieve from a PACS, you need to install a DICOM Store service. For testing, you can make use of the DICOM Store OpenREM can provide. However, because this is not stable over longer periods of time we recommend using a third-party DICOM Store service. You can use any one you like, as long as it can be scripted to call OpenREM scripts when DICOM objects are received. We recommend Orthanc or Conquest for this and provide details of how to configure them in the Third-party DICOM Stores section.

Orthanc
Alternative - Conquest
  • Download Conquest DICOM server from https://ingenium.home.xs4all.nl/dicom.html

  • Install using the instructions included in the download - there is a PDF with Windows install instructions and general usage instructions, and another PDF with Linux install instructions. The guides in Third-party DICOM Stores should be consulted when making configuration decisions.

  • Alternatively, Ubuntu 16.04 users can use the following instructions:

    Conquest DICOM store node on Ubuntu 16.04
    Installation

    Ubuntu 16.04 has the Conquest DICOM server in its repositories, so this makes installation very easy. However, it isn’t the latest version and isn’t going to be updated further. However, if you are using Ubuntu 16.04 then it does make for a relatively easy install.

    There are options to install with different databases – for OpenREM we’re not really going to use the database so the easiest option is to use SQLite:

    sudo apt-get install conquest-sqlite
    
    Basic configuration
    Modify dgatesop.lst

    Edit the dgatesop.lst file in the /etc/conquest-dicom-server folder, for example

    sudo nano /etc/conquest-dicom-server/dgatesop.lst
    

    And add the following line

    XRayRadiationDoseSR 1.2.840.10008.5.1.4.1.1.88.67   sop
    

    It isn’t critical where it goes, but I tend to add it where it belongs between KeyObjectSelectionDocument and PETStorage. I also add in the spaces to make it line up, but again this is just for aesthetic reasons!

    If you are pasting from the clipboard into nano from within Linux, use Shift-Ctrl-v. If you are using PuTTY in Windows to interact with Ubuntu, a right click on the mouse or Shift-Insert should paste the text into the terminal.

    To save and exit from nano, use Ctrl-o (out), press return to confirm the filename and then Ctrl-x (exit).

    Configuration

    Here will be a link to the dicom.ini config… the text below will be subsumed into it

    Configure the Store SCP

    Edit the dicom.ini file in the /etc/conquest-dicom-server folder, for example

    sudo nano /etc/conquest-dicom-server/dicom.ini
    

    Modify the following lines as required. The server name field – with the Conquest default of CONQUESTSRV1 – is the AE Title, so should be 16 characters or less and consist of letters and numbers with no spaces. It is case insensitive. The TCPPort is normally either 104, the standard DICOM port, or any number greater than 1023.

    # Network configuration: server name and TCP/IP port#
    MyACRNema                = CONQUESTSRV1
    TCPPort                  = 11112
    

    Again, save and exit.

    If you’ve changed the AE Title and/or port, restart conquest:

    sudo /etc/init.d/dgate restart
    
    Testing basic configuration

    Test the Store SCP by returning to OpenREM and navigating to Config -> DICOM network configuration.

    Click to Add new store and enter the AE title and port you have set, along with a reference name.

    Click to Submit, and you will return to the summary page which should inform you if the server is running.

    Configure Conquest to work with OpenREM

    The next stage is to configure Conquest to store the incoming object and ask OpenREM to process them.

    Conquest configuration

    At the end of the /etc/conquest-dicom-server/dicom.ini file, add the following lines. You will need to tailor them to save the file to an appropriate place. The _conquest user will need to be able to write to that location. You will also need to make sure the path to the scripts you just created are correct.

    The example below assumes images will be saved in /var/lib/conquest-dicom-server/incoming/, which you can create as follows:

    sudo mkdir /var/lib/conquest-dicom-server/incoming
    sudo chown _conquest:_conquest /var/lib/conquest-dicom-server/incoming
    

    Each instruction in the dicom.ini file below has a destroy instruction to delete Conquest’s copy of the file and to remove it from it’s database. This isn’t the version we’ve saved in incoming to process.

    sudo nano /etc/conquest-dicom-server/dicom.ini
    
    # RDSR
    ImportConverter0  = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.67"; {save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-rdsr.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy}
    # Import arguments for GE CT - uses Enhanced SR instead of Radiation Dose SR
    ImportConverter1  = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.22"; {save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-rdsr.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy}
    
    # MG images
    ImportModality2   = MG
    ImportConverter2  = save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-mg.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy
    
    # DX images
    ImportModality3   = DX
    ImportConverter3  = save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-dx.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy
    # CR images
    ImportModality4   = CR
    ImportConverter4  = save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-dx.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy
    
    # Philips CT
    ImportConverter5  = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.7"; {save to /var/lib/conquest-dicom-server/incoming/%o.dcm; system /etc/conquest-dicom-server/openrem-ctphilips.sh /var/lib/conquest-dicom-server/incoming/%o.dcm; destroy}
    
    # Other objects
    ImportConverter6  = destroy
    

    Finally, restart conquest to make use of the new settings:

    sudo /etc/init.d/dgate restart
    

Unlike with the database, it is possible to change DICOM Store service at a later point.

Resources for creating RDSR for older Toshiba CT scanners

New in version 0.8.0

If you need to import data from older Toshiba CT scanners into OpenREM then the following tools need to be available on the same server as OpenREM:

For more information see For CT dose summary files from older Toshiba CT scanners. The locations of these executables needs to be configured in the local_settings.py - see Toshiba CT RDSR creation.

Install OpenREM

You are now ready to install OpenREM, so go to the Installing OpenREM docs.

Further instructions

Virtualenv and virtualenvwrapper

If the server is to be used for more than one python application, or you wish to be able to test different versions of OpenREM or do any development, it is highly recommended that you use virtualenv or maybe virtualenvwrapper

Virtualenv sets up an isolated python environment and is relatively easy to use.

If you do use virtualenv, all the paths referred to in the documentation will be changed to:

  • Linux: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: virtualenvfolder\Lib\site-packages\openrem

In Windows, even when the virtualenv is activated you will need to call python and provide the full path to script in the Scripts folder. If you call the script (such as openrem_rdsr.py) without prefixing it with python, the system wide Python will be used instead. This doesn’t apply to Linux, where once activated, the scripts can be called without a python prefix from anywhere.

Quickstart Ubuntu install using virtualenv

If you want to get everything installed quickly, you could do the following on a Ubuntu server:

dose@ubuntu1604:~$ sudo apt update
dose@ubuntu1604:~$ sudo apt install python python-pip rabbitmq-server postgresql libpq-dev orthanc dcmtk default-jre
dose@ubuntu1604:~$ pip install virtualenv
dose@ubuntu1604:~$ virtualenv veopenrem
dose@ubuntu1604:~$ . veopenrem/bin/activate
(veopenrem) dose@ubuntu1604:~$ pip install numpy psycopg2

You will then need to setup the PostgreSQL database (Linux) and download the latest version of the pixelmed.jar application e.g.:

(veopenrem) dose@ubuntu1604:~$ wget http://www.dclunie.com/pixelmed/software/20180419_current/pixelmed.jar

We can now install OpenREM and a customised version of pynetdicom:

(veopenrem) dose@ubuntu1604:~$ pip install openrem
(veopenrem) dose@ubuntu1604:~$ pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

You can now go straight to the Configuration.

Installing OpenREM

Install OpenREM

pip install openrem

Will need ``sudo`` or equivalent if installing on linux without using a virtualenv

Install pynetdicom (edited version)

Pynetdicom is used for the DICOM Store SCP and Query Retrieve SCU functions. See Direct from modalities for details.

pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

Note

You must install the pynetdicom package from the link above - the version in pypi or the newer versions in GitHub won’t work with the current version of OpenREM. Future versions of OpenREM will be adapted to work with pynetdicom3 and pydicom>1.0.

If you are using the latest version of pip you will get error messages including the phrase Failed building wheel for pynetdicom - it should be ok to ignore this message as long as you end up with the message Successfully installed pynetdicom-0.8.2b2

Configuration

Locate install location
  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\

There are two files that need renaming:

  • openremproject/local_settings.py.example to openremproject/local_settings.py
  • openremproject/wsgi.py.example to openremproject/wsgi.py
Edit local_settings.py

Note

Windows notepad will not recognise the Unix style line endings. Please use an editor such as Notepad++ or Notepad2 if you can, else use WordPad – on the View tab you may wish to set the Word wrap to ‘No wrap’

Important

In local_settings.py, always use forward slashes and not backslashes, even for paths on Windows systems.

The directories in this local_settings.py file must already exist - OpenREM will not create them for you.

Database

Note

SQLite is great for getting things running quickly and testing if the setup works, but is not recommended for production use.

We recommend using PostgreSQL as it is the best supported database for Django, and the only one for which the median value will be calculated and displayed in OpenREM charts. Alternatively, other databases such as MySQL/MariaDB, Oracle, and some others with lower levels of support can be used.

There are some further guides to setting up PostgreSQL – see Databases

If you are using SQLite:

'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/ENTER/PATH/WHERE/DB/FILE/CAN/GO',
  • Linux example: 'NAME': '/home/user/openrem/openrem.db',
  • Windows example: 'NAME': 'C:/Users/myusername/Documents/OpenREM/openrem.db',

If you are using PostgreSQL:

'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'openremdb',
'USER': 'openremuser',
'PASSWORD': 'openrem_pw',
Location for imports and exports

Csv and xlsx study information exports and patient size csv imports are written to disk at a location defined by MEDIA_ROOT.

The path set for MEDIA_ROOT is up to you, but the user that runs the webserver must have read/write access to the location specified because it is the webserver than reads and writes the files. In a debian linux, this is likely to be www-data for a production install. Remember to use forward slashes for the config file, even for Windows.

Linux example:

MEDIA_ROOT = "/var/openrem/media/"

Windows example:

MEDIA_ROOT = "C:/Users/myusername/Documents/OpenREM/media/"
Secret key

Generate a new secret key and replace the one in the local_settings.py file. You can use http://www.miniwebtool.com/django-secret-key-generator/ for this.

Allowed hosts

The ALLOWED_HOSTS needs to be defined, as the DEBUG mode is now set to False. This needs to contain the OpenREM server hostname or IP address that will be used in the URL in the web browser. For example:

ALLOWED_HOSTS = [
    '192.168.56.102',
    '.doseserver.',
    'localhost',
]

A dot before a hostname allows for subdomains (eg www.doseserver), a dot after a hostname allows for FQDNs (eg doseserver.ad.trust.nhs.uk). Alternatively, a single '*' allows any host, but removes the security the feature gives you.

Customised date format in xlsx exports

# TODO: check csv situation The default date form at for OpenREM xlsx exports is dd/mm/yyyy. If you wish to customise this, uncomment the XLSX_DATE line, for example the standard US date format would be:

XLSX_DATE = 'mm/dd/yyyy'
Log file

There are two places logfiles need to be configured - here and when starting Celery. The logs defined here capture most of the information; the Celery logs just capture workers starting and tasks starting and finishing.

Configure the filename to determine where the logs are written. In linux, you might want to send them to a sub-folder of /var/log/. In this example, they are written to the MEDIA_ROOT; change as appropriate:

import os
logfilename = os.path.join(MEDIA_ROOT, "openrem.log")
qrfilename = os.path.join(MEDIA_ROOT, "openrem_qr.log")
storefilename = os.path.join(MEDIA_ROOT, "openrem_store.log")
extractorfilename = os.path.join(MEDIA_ROOT, "openrem_extractor.log")

LOGGING['handlers']['file']['filename'] = logfilename          # General logs
LOGGING['handlers']['qr_file']['filename'] = qrfilename        # Query Retrieve SCU logs
LOGGING['handlers']['store_file']['filename'] = storefilename  # Store SCP logs
LOGGING['handlers']['extractor_file']['filename'] = extractorfilename  # Extractor logs

If you want all the logs in one file, simply set them all to the same filename.

In the settings file, there are simple and verbose log message styles. We recommend you leave these as verbose:

LOGGING['handlers']['file']['formatter'] = 'verbose'        # General logs
LOGGING['handlers']['qr_file']['formatter'] = 'verbose'     # Query Retrieve SCU logs
LOGGING['handlers']['store_file']['formatter'] = 'verbose'  # Store SCP logs
LOGGING['handlers']['extractor_file']['formatter'] = 'verbose'  # Extractor logs

Next, you can set the logging level. Options are DEBUG, INFO, WARNING, ERROR, and CRITICAL, with progressively less logging. INFO is probably a good choice for most circumstances. DEBUG is useful if something is going wrong, but it is quite chatty for routine use!

LOGGING['loggers']['remapp']['level'] = 'INFO'                    # General logs
LOGGING['loggers']['remapp.netdicom.qrscu']['level'] = 'INFO'     # Query Retrieve SCU logs
LOGGING['loggers']['remapp.netdicom.storescp']['level'] = 'INFO'  # Store SCP logs
LOGGING['loggers']['remapp.extractors.ct_toshiba']['level'] = 'INFO'  # Toshiba RDSR creation extractor logs

Finally, if you are using Linux you can set the system to start a new log file automatically when the current one gets to a certain size. The settings described below don’t work with Windows - we’ll try to include Windows settings in the next release. See issue 483 to find out the progress on this!

To activate the ‘rotating’ log function, uncomment the remaining lines by removing the # from the beginning of the lines. For example for the query retrieve logs:

LOGGING['handlers']['qr_file']['class'] = 'logging.handlers.RotatingFileHandler'
LOGGING['handlers']['qr_file']['maxBytes'] = 10 * 1024 * 1024  # 10*1024*1024 = 10 MB
LOGGING['handlers']['qr_file']['backupCount'] = 5  # number of log files to keep before deleting the oldest one
Time zone

Configure the local timezone in order to get correct times in the logfiles. The default timezone is set at ‘Europe/London’. Valid options can be found here: http://en.wikipedia.org/wiki/List_of_tz_zones_by_name

TIME_ZONE = 'Europe/London'
Language

Configure the local language. Default language is set at us-english. Valid options can be found here: http://www.i18nguy.com/unicode/language-identifiers.html

LANGUAGE_CODE = 'en-us'
Toshiba CT RDSR creation

If you need to import data from older Toshiba CT scanners into OpenREM then the following tools need to be available on the same server as OpenREM:

The paths to these must be set in local_settings.py for your system:

# Locations of various tools for DICOM RDSR creation from CT images
DCMTK_PATH = 'C:/Apps/dcmtk-3.6.0-win32-i386/bin'
DCMCONV = os.path.join(DCMTK_PATH, 'dcmconv.exe')
DCMMKDIR = os.path.join(DCMTK_PATH, 'dcmmkdir.exe')
JAVA_EXE = 'C:/Apps/doseUtility/windows/jre/bin/java.exe'
JAVA_OPTIONS = '-Xms256m -Xmx512m -Xss1m -cp'
PIXELMED_JAR = 'C:/Apps/doseUtility/pixelmed.jar'
PIXELMED_JAR_OPTIONS = '-Djava.awt.headless=true com.pixelmed.doseocr.OCR -'

The example above is for Windows. On linux, if you have installed the Offis DICOM toolkit with sudo apt install dcmtk or similar, you can find the path for the configuration above using the command which dcmconv. This will be something like /usr/bin/dcmconv, so the DCMTK_PATH would be '/usr/bin and the DCMCONV would be os.path.join(DCMTK_PATH, 'dcmconv'). Similarly for DCMMKDIR and JAVA_EXE, which might be /usr/bin/java. The pixelmed.jar file should be downloaded from the link above, and you will need to provide the path to where you have saved it.

Create the database

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: cd /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: cd /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: cd virtualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: cd C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: cd virtualenvfolder\Lib\site-packages\openrem\

Create the database:

python manage.py makemigrations remapp
python manage.py migrate
python manage.py showmigrations

The last command will list each Django app migrations. Each should have a cross inside a pair of square brackets something like below:

admin
 [X] 0001_initial
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
remapp
 [X] 0001_initial
sessions
 [X] 0001_initial
sites
 [X] 0001_initial

Finally, create a Django super user:

python manage.py createsuperuser

Answer each question as it is asked – this user is needed to set up the other users and the permissions.

Add the median database function: PostgreSQL databases only

Rename the file

remapp/migrations/0002_0_7_fresh_install_add_median.py.inactive

to

remapp/migrations/0002_0_7_fresh_install_add_median.py

and then run

python manage.py migrate

This command runs the migration file, and will display the text Applying remapp.0002_0_7_fresh_install_add_median... OK, indicating that the median function has been added.

Start all the services!

You are now ready to start the services to allow you to use OpenREM - go to Start all the services to see how!

A standard installation assumes access to the internet from the computer where OpenREM is being installed. Sometimes this isn’t possible, so we’ve added instructions for an offline installation too. Currently it focuses on Windows only (for the server - the computer connected to the internet can be running any operating system).

Offline Installation on Windows

In order to carry out an offline installation you will need to download the OpenREM package and dependencies. The instructions below should work for downloading on any operating system, as long as you have Python 2.7 and a reasonably up to date version of pip installed.

If you have trouble when installing the Python packages due to incorrect architecture, you may need to either download on a Windows system similar to the server (matching 32-bit/64-bit), or to download the files from http://www.lfd.uci.edu/~gohlke/pythonlibs/ instead.

On a computer with internet access

Download independent binaries

Python from https://www.python.org/downloads/windows/

  • Follow the link to the ‘Latest Python 2 release’
  • Download either the Windows x86 MSI installer for 32-bit Windows or
  • Download Windows x86-64 MSI installer for 64-bit Windows

Erlang from https://www.erlang.org/downloads

  • Download the latest version of Erlang/OTP. Again, choose between
  • Windows 32-bit Binary File or
  • Windows 64-bit Binary File

RabbitMQ from http://www.rabbitmq.com/install-windows.html

  • Download rabbitmq-server-x.x.x.exe from either option

PostgreSQL from http://www.enterprisedb.com/products-services-training/pgdownload#windows

Note: Other databases such as MySQL are also suitable, though the median function for charts will not be available. For testing purposes only, you could skip this step and use SQLite3 which comes with OpenREM

  • Download by clicking on the icon for Win x86-32 or Win x86-64

PostgreSQL Python connector from http://www.lfd.uci.edu/~gohlke/pythonlibs/#psycopg

  • Find the right version - look for psycopg2-X-cp27-cp27m-win32.whl for 32-bit Windows or
  • psycopg2-X-cp27-cp27m-win_amd64.whl for 64-it Windows.
  • At the time of writing, X was 2.7.3.2 - choose the latest cp27 version

NumPy from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy

  • Find the right version - look for numpy-X+mkl-cp27-cp27m-win32.whl for 32-bit Windows or
  • numpy-X+mkl-cp27-cp27m-win_amd64.whl for 64-bit Windows.
  • At the time of writing, X was 1.14.0 - choose the latest cp27 version

Pynetdicom from https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

  • The downloaded file will be named something like edmcdonagh-pynetdicom-2da8a57b53b3.tar.gz
  • Note: this version is modified in comparison to the version in PyPI, and will malfunction if you use the official version

A webserver such as Apache, although this can be left till later - you can get started with the built-in web server

Download python packages from PyPI

In a console, navigate to a suitable place and create a directory to collect all the packages in, then use pip to download them all:

mkdir openremfiles
pip install -d openremfiles openrem

Note

If pip install complains that -d is not a valid option, then use the following command instead:

pip download -d openremfiles openrem

Copy everything to the Windows machine
  • Add the pynetdicom file, the psycopg2 file and the numpy file to the directory with the other python packages
  • Copy this directory plus all the binaries to the Windows server that you are using

On the Windows server without internet access

Installation of binaries

Install the binaries in the following order:

  1. Python
  2. Erlang
  3. RabbitMQ
Installation of the python packages

In a console, navigate to the directory that your openremfiles directory is in, and

pip install openremfiles\numpy‑1.14.0+mkl‑cp27-cp27m‑win32.whl  # update the version number
# or if you have the 64 bit version
pip install openremfiles\numpy‑1.14.0+mkl‑cp27-cp27m‑win_amd64.whl  # update the version number

pip install --no-index --find-links=openremfiles openrem  # where openremfiles is the directory you created

pip install openremfiles\edmcdonagh-pynetdicom-2da8a57b53b3.tar.gz
Install PostgreSQL

See the instructions to Install PostgreSQL on Windows.

Install webserver

If you are doing so at this stage.

Configure OpenREM ready for use

OpenREM is now installed, so go straight to the Configuration section of the standard installation docs

Upgrading an existing installation

Upgrade to OpenREM 0.8.0

Headline changes
  • This release has extensive automated testing for large parts of the codebase (for the first time)
  • Code quality is much improved, reduced duplication, better documentation, many bugs fixed
  • Imports: RDSR from a wider range of systems now import properly
  • Imports: Better distinction and control over defining RDSR studies as RF or DX
  • Imports: Code and instructions to generate and import RDSR from older Toshiba CT scanners
  • Imports: DICOM Query-Retrieve functionality has been overhauled
  • Imports: Duplicate checking improved to allow cumulative and continued study RDSRs to import properly
  • Imports: indicators that a study is not a patient can now be configured in the web interface
  • Imports, display and export: Better handling of non-ASCII characters
  • Interface: More detailed, consistent and faster rendering of the data in the web interface
  • Interface: Maps of fluoroscopy radiation exposure incident on a phantom (Siemens RDSRs only)
  • Interface: More and better charts, including scatter plots for mammography
  • Interface: Display names dialogue has been extended to allow administration of all studies from each source
  • Exports: Much faster, and more consistent
  • Documentation: Extensive user documentation improvements
Upgrading an OpenREM server with no internet access

Follow the instructions found at Upgrade an offline OpenREM installation, before returning here to update the database and configuration.

Upgrading from version 0.7.4 or previous 0.8.0 betas
Upgrade
  • Back up your database

    • For PostgreSQL on linux you can refer to Backup the database
    • For PostgreSQL on Windows you can refer to backupRestorePostgreSQL
    • For a non-production SQLite3 database, simply make a copy of the database file
  • Stop any Celery workers

  • Consider temporarily disabling your DICOM StoreSCP, or redirecting the data to be processed later

  • If you are using a virtualenv, activate it

  • Install the new version of OpenREM:

pip install openrem==0.8.0
Update the configuration

Locate and edit your local_settings file

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/openremproject/local_settings.py
  • Other linux: /usr/lib/python2.7/site-packages/openrem/openremproject/local_settings.py
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/openremproject/local_settings.py
  • Windows: C:\Python27\Lib\site-packages\openrem\openremproject\local_settings.py
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\openremproject\local_settings.py
Date format

Set the date format for xlsx exports (need to check csv situation). Copy the following code into your local_settings.py file if you want to change it from dd/mm/yyy:

# Date format for exporting data to Excel xlsx files.
# Default in OpenREM is dd/mm/yyyy. Override it by uncommenting and customising below; a full list of codes is available
# at https://msdn.microsoft.com/en-us/library/ee634398.aspx.
# XLSX_DATE = 'mm/dd/yyyy'
Time zone and language

Consider setting the timezone and language in local_settings.py. See local_settings.py.example.

Add additional log file configuration

Warning

If the configuration is not added for the new openrem_extractor.log you will find it being created whereever you start the webserver from, and starting the webserver may fail.

Add the new extractor log file configuration to the local_settings.py - you can copy the ‘Logging configuration’ section from local_settings.py.example if you haven’t made many changes to this section. See the Log file settings in the install instructions.

Warning

If you are upgrading from an earlier beta with the Toshiba RDSR creation logs defined, this has changed names and must be modified in local_settings.py before the migration below. It should be changed to:

LOGGING['loggers']['remapp.extractors.ct_toshiba']['level'] = 'INFO'  # Toshiba RDSR creation extractor logs

substituting INFO for whichever level of logging is desired.

Adding legacy Toshiba CT functionality

If you need to import data from older Toshiba CT scanners into OpenREM then the following tools need to be available on the same server as OpenREM:

The paths to these must be set in local_settings.py for your system:

# Locations of various tools for DICOM RDSR creation from CT images
DCMTK_PATH = 'C:/Apps/dcmtk-3.6.0-win32-i386/bin'
DCMCONV = os.path.join(DCMTK_PATH, 'dcmconv.exe')
DCMMKDIR = os.path.join(DCMTK_PATH, 'dcmmkdir.exe')
JAVA_EXE = 'C:/Apps/doseUtility/windows/jre/bin/java.exe'
JAVA_OPTIONS = '-Xms256m -Xmx512m -Xss1m -cp'
PIXELMED_JAR = 'C:/Apps/doseUtility/pixelmed.jar'
PIXELMED_JAR_OPTIONS = '-Djava.awt.headless=true com.pixelmed.doseocr.OCR -'

The example above is for Windows. On linux, if you have installed the Offis DICOM toolkit with sudo apt install dcmtk or similar, you can find the path for the configuration above using the command which dcmconv. This will be something like /usr/bin/dcmconv, so the DCMTK_PATH would be '/usr/bin and the DCMCONV would be os.path.join(DCMTK_PATH, 'dcmconv'). Similarly for DCMMKDIR and JAVA_EXE, which might be /usr/bin/java. The pixelmed.jar file should be downloaded from the link above, and you will need to provide the path to where you have saved it.

Migrate the database

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\
python manage.py makemigrations remapp
# if changes are detected (not expected between most beta versions)
python manage.py migrate remapp
Update static files

In the same shell/command window as you used above run the following command to clear the static files belonging to your previous OpenREM version and replace them with those belonging to the version you have just installed (assuming you are using a production web server…):

python manage.py collectstatic --clear
Restart all the services

Follow the guide at Start all the services.

Upgrade an offline OpenREM installation

Upgrading OpenREM requires new Python packages to be available as well as the latest version of OpenREM. These can be downloaded on any computer with Python 2.7 installed and an internet connection, though if you have trouble when installing the packages you might need to use a similar computer to the one you are installing on - same operating system and matching 32-bit or 64-bit.

On a computer with internet access

In a console, navigate to a suitable place and create a directory to collect all the packages in, then use pip to download them all:

mkdir openremfiles
pip install -d openremfiles openrem==0.8.0

Note

If pip install complains that -d is not a valid option, then use the following command instead:

pip download -d openremfiles openrem==0.8.0

Copy everything to the OpenREM server
  • Copy the directory to the OpenREM server
On the OpenREM server without internet access
  • Back up your database

    • For PostgreSQL you can refer to Backup the database
    • For a non-production SQLite3 database, simply make a copy of the database file
  • Stop any Celery workers

  • If you are using a virtualenv, activate it now, then

pip install --no-index --find-links=openremfiles openrem==0.8.0

Now go back to Update the configuration and update the configuration.

Databases

During the installation process, you will need to install a database. For testing only, you can use the built in SQLite3 database, but for production use you will need a production grade database. This is covered in the Pre-installation preparations documentation, but as you will probably want to find the database instructions again, the links are repeated here.

PostgreSQL database (Linux)

Creating the database
Install PostgreSQL and the python connector
sudo apt-get install postgresql libpq-dev

If you are using a virtualenv, make sure you are in it and it is active (source bin/activate)

pip install psycopg2
Change the security configuration

The default security settings are too restrictive to allow access to the database. Assumes version 9.5, change as appropriate.

sudo nano /etc/postgresql/9.5/main/pg_hba.conf

Scroll down to the bottom of the file and edit the following line from peer to md5:

local    all            all                         md5

Don’t worry about any lines that start with a # as they are ignored. If you can’t access the database when everything else is configured, you might need to revisit this file and see if there are other lines with a method of peer that need to be md5

Note

If you need to have different settings for different databases on your server, you can use the database name instead of the first all, and/or the the database user name instead of the second all.

Restart PostgreSQL so the new settings take effect:

sudo service postgresql restart
Optional: Specify the location for the database files

You might like to do this if you want to put the database on an encrypted location instead of /var/lib/postgresql.

For this example, I’m going to assume all the OpenREM programs and data are in the folder /var/openrem/ and PostgreSQL is at version 9.5 (change both as appropriate)

sudo service postgresql stop
mkdir /var/openrem/database
sudo cp -aRv /var/lib/postgresql/9.5/main /var/openrem/database/
sudo nano /etc/postgresql/9.5/main/postgresql.conf

Change the line

data_directory = '/var/lib/postgresql/9.5/main'

to

data_directory = '/var/openrem/database/main'

then restart PostgreSQL:

sudo service postgresql start
Create a user for the OpenREM database
sudo -u postgres createuser -P openremuser

Enter a new password for the openremuser, twice

Create the OpenREM database
sudo -u postgres createdb -T template1 -O openremuser -E 'UTF8' openremdb

If this is your initial install, you are now ready to install OpenREM, so go to the Installing OpenREM docs.

If you are replacing a SQLite test install with PostgreSQL, continue here.

Configure OpenREM to use the database

Move to the OpenREM install directory:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\

Edit the settings file, eg

nano openremproject/local_settings.py

Set the following (changing database name, user and password as appropriate)

'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'openremdb',
'USER': 'openremuser',
'PASSWORD': 'openrem_pw',
Backup the database
Ad-hoc backup from the command line
sudo -u postgres pg_dump openremdb > /path/to/backup.bak

If you are moving a backup file between systems, or keeping a few backups, you may like to compress the backup; for example a 345 MB OpenREM database compresses to 40 MB:

tar -czf backup.bak.tar.gz backup.bak
Automated backup with a bash script
#! /bin/bash
rm -rf /path/to/db/backups/*
PGPASSWORD="openrem_pw" /usr/bin/pg_dump -Uopenremuser openremdb > /path/to/db/backups/openrem.bak

This script could be called by a cron task, or by a backup system such as backuppc prior to running the system backup.

Restore the database

If the restore is taking place on a different system,

  • ensure that PostgreSQL is installed and the same user has been added as was used to create the initial database (see Creating the database).
  • Ensure that the new system has the same version of OpenREM installed as the system the database was backed up from.
  • Ensure the openrem/remapp/migrations/ folder has no files in except __init__.py

Create a fresh database and restore from the backup:

sudo -u postgres createdb -T template0 new_openremdb_name
sudo -u postgres psql new_openremdb_name < /path/to/db/backups/openrem.bak

Reconfigure local_settings.py with the new database details and introduce OpenREM to the restored database:

python manage.py migrate --fake-initial
python manage.py makemigrations remapp
python manage.py migrate remapp --fake

If you are creating a second system in order to test upgrading, you can do this now followed by the usual python manage.py makemigrations remapp then python manage.py migrate remapp as per the upgrade instructions.

Useful PostgreSQL commands
-- Start the PostgreSQL console
sudo -u postgres psql

-- List users
\du

-- List databases
\l

-- Exit the console
\q

PostgreSQL database (Windows)

Note

Original author JA Cole

Get PostgreSQL and the python connector
Install PostgreSQL

Run the the postgresql installer. It will ask for a location. Ensure the “data” directory is not under “Program Files” as this can cause permissions errors. Enter a superuser password when prompted. Make sure you keep this safe as you will need it.

Create a user and database

Open pgAdmin

  • Click on servers to expand
  • Double click on PostgreSQL 9.6 (or whichever version you have installed)
  • Enter your superuser password
  • Right click on “login roles” and choose “New login role”
  • Create the openremuser (or whatever you want your user to be called) and under “Definition” add a password
  • Under “Privileges” ensure that “Can login” and “Create databases” are set to “Yes”
  • Click OK
  • Right click on databases and choose “New database”
  • Name the database (openremdb is fine) and assign the the owner to the user you just created
Install psycopg2
pip install psycopg2-2.7.3.2-cp27-cp27m-win32.whl  # update the version number
# or if you have the 64-bit version
pip install psycopg2-2.7.3.2-cp27-cp27m-win_amd64.whl  # update the version number
# adjusting the version number appropriately

If this is your initial install, you are now ready to install OpenREM, so go to the Installing OpenREM docs.

If you are replacing a SQLite test install with PostgreSQL, continue here.

Configure OpenREM to use the database
Find and edit the settings file (notepad works fine). The path depends on your python install, but could be something like:
  • C:\lib\python2.7\site-packages\openrem\openremproject\local_settings.py
Set the following (changing name, user and password as appropriate):
  • 'ENGINE': 'django.db.backends.postgresql_psycopg2',
  • 'NAME': 'openremdb',
  • 'USER': 'openremuser',
  • 'PASSWORD': 'openrem_pw',

Backing up a PostgreSQL database (Windows)

Note

Content contributed by DJ Platten

These instructions are based on PostgreSQL 9.1 and OpenREM 0.5.0 running on Windows Server 2008. The database restore has been tested on Ubuntu 12.04 LTS.

As a one-off, create a PostgreSQL user called backup with a password of backup. This is easiest to do using the pgAdminIII tool: you’ll need to create a new login role. In the role privileges ensure that Can initiate streaming replication and backups is checked.

The pgAdminIII tool is available by default on Windows, but needs to be explicitly installed if using Ubuntu with the following command:

sudo apt-get install pgadmin3

For the remainder of this article I’m going to assume that your OpenREM database is called openrempostgresql.

To backup the contents of openrempostgresql to a file called backup.sql run the following at the command line in a command prompt (Windows), or terminal window (Ubuntu):

pg_dump -i -U backup -F c -b -v -f backup.sql openrempostgresql

Note that the pg_dump command needs to be in your path for this to work exactly as written. The -U backup indicates that the backup user is to carry out the task. The -F c option archives in a suitable format for input into the pg_restore command. Further information on pg_dump and backing up a PostgreSQL database can be found here: http://www.postgresql.org/docs/9.3/static/app-pgdump.html and here: http://www.postgresql.org/docs/9.3/static/backup-dump.html

Restoring a PostgreSQL database (Windows)

The pg_restore command can be used to restore the database using one of the backed-up SQL files that were produced using the pg_dump command.

Use the pgAdminIII tool to ensure that there is a PostgreSQL user called openremuser.

Use pgAdminIII to create a database called openrempostgresql; set the owner to openremuser and the encoding to UTF8.

Run the following command in a command prompt window (Windows) or terminal window (Ubuntu) to restore the contents of backupFile to the openrempostgresql database, where backupFile is the file created by the pg_dump command:

pg_restore -U postgres -d openrempostgresql backupFile

Ensure that openremuser has an entry in PostgreSQL’s pg_hpa.conf file for md5 authentication:

local all openremuser md5

The PostgreSQL server will need to be restarted if you have changed pg_hpa.conf.

See http://www.postgresql.org/docs/9.3/static/backup-dump.html#BACKUP-DUMP-RESTORE for further details.

Backing up MySQL on Windows

Note

Content contributed by DJ Platten

These instructions are based on Windows XP.

As a one-off, create a MySQL user called backup with a password of backup that has full rights to the database:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -u root -p -e "CREATE USER 'backup'@'localhost' IDENTIFIED BY 'backup'";

Grant the backup user full privileges on the database called openremdatabasemysql:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -u root -p -e "GRANT ALL PRIVILEGES ON openremdatabasemysql .* TO 'backup'@'localhost'";

Grant the backup user privileges to create databases:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -u root -p -e "GRANT CREATE ON *.* TO 'backup'@'localhost'";

Reload the privileges to ensure that MySQL registers the new ones:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -u root -p -e "FLUSH PRIVILEGES";

To backup the contents of the database from the command line to a file called backup.sql (note that the lack of spaces after the -u and -p is not a typo):

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysqldump.exe -ubackup -pbackup openremdatabasemysql > backup.sql

To restore the database, assuming that it doesn’t exist anymore, first it needs to be created:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -ubackup -pbackup -e "CREATE DATABASE openremdatabasemysql";

Then restore the contents of the database from a file called backup.sql:

C:\Program Files\MySQL\MySQL Server 5.6\bin\mysql.exe -ubackup -pbackup openremdatabasemysql < backup.sql

An example DOS batch file to back up the contents of the openremdatabasemysql database using a time stamp of the form yyyy-mm-dd_hhmm, zip up the resulting file, delete the uncompressed version and then copy it to a network location (the network copy will only work if the user that runs the batch file has permission on the network):

@echo off
For /f "tokens=1-4 delims=/ " %%a in ('date /t') do (set mydate=%%c-%%b-%%a)
For /f "tokens=1-2 delims=/:" %%a in ('time /t') do (set mytime=%%a%%b)

"C:\Program Files\MySQL\MySQL Server 5.6\bin\mysqldump.exe" -ubackup -pbackup openremdatabasemysql > F:\OpenREMdatabase\backup\%mydate%_%mytime%_openremdatabasemysql.sql

"C:\Program Files\7-Zip\7z.exe" a F:\OpenREMdatabase\backup\%mydate%_%mytime%_openremdatabasemysql.zip F:\OpenREMdatabase\backup\%mydate%_%mytime%_openremdatabasemysql.sql

del F:\OpenREMdatabase\backup\%mydate%_%mytime%_openremdatabasemysql.sql

copy F:\OpenREMdatabase\backup\%mydate%_%mytime%_openremdatabasemysql.zip "\\Srv-mps-001\xls_protect\PATDOSE\OpenREMbackup\"

Web servers

Unlike the database, the production webserver can be left till later and can be changed again at any time. However, for performance it is recommended that a production webserver is used instead of the inbuilt ‘runserver’.

On Windows or Linux, it is possible to use Apache, however for reasons relating to how Python, Apache and mod_wsgi are compiled using old Microsoft tools, this is now nearly impossible to do on the Windows platform. There is no reason for existing Windows installs with Apache to change webserver.

For Apache installs on Linux, the django website has instructions and links to get you set up.

Our recommendations for Windows and Linux are:

Running OpenREM on Windows with IIS

These instructions are for running OpenREM under IIS on Windows Server 2012, but should work on Windows 7/8/10 and later versions of Windows Server with minimal modification. The instructions are based on http://blog.mattwoodward.com/2016/07/running-django-application-on-windows.html

Why use IIS?

The built-in Django webserver is not advised for production environments. There are a few alternatives for serving a Django application. Apache is probably the best known web server, but has some requirements under Windows in combination with Python / Django that are hard to fulfill (see https://github.com/GrahamDumpleton/mod_wsgi/blob/develop/win32/README.rst). IIS is default available on Windows Server and doesn’t have these requirements.

Prerequisites
  • A working OpenREM installation, that serves web pages using the built-in web server.
    You can test this using the instructions in Start all the services
Steps to perform
Install wfastcgy Python package
  • Open a command prompt and navigate to the pythonscript directory if that directory is not in your path
  • Type pip install wfastcgi
  • Close the command prompt by entering exit
Install IIS
  • Open Control panel

  • Take care the view by is on small or large icons (not category view)

  • Select Program and Features

  • Select Turn Windows features on or off

  • Windows Server 2012

    • Click Next
    • Leave the Role-based or feature-based installation radio button selected and click Next
    • Leave the current server highlighted and click Next
    • Scroll to the bottom of the list and check Web Server (IIS)
    • In the dialog that appears, leave the Include management tools (if applicable) checkbox checked and click Add Features
    • On the Select server roles step, now that Web Server (IIS) is checked, click Next
    • Leave the defaults and click Next
    • On the Web Server Role (IIS) step, click Next
    • On the Select role services step, scroll down to Application Development, expand that section, and check the CGI box. click Next.
    • Click Install and after installation, click Close
  • Window 7 & Windows 10

    • Tick Internet information services to turn the default features on
    • In World Wide Web Services, Application Development Features, tick CGI
    • Click OK
    • Click Close (Windows 10 only, In Windows 7 the Windows Features window will close automatically)
  • Close the Server Manager and the Control Panel

To test the IIS installation browse to http://localhost. You should see the default IIS “Welcome” Page.

Configure IIS
  • Open Administrative Tools and double-click the Internet Information Services (IIS) Manager link. You may need to right click and Run as Adminstrator. - For Windows 7 the Administrative Tools are not available by default, you can also type inetmgr in the search program and files text box.
  • Click on the name of the server within the IIS manager
  • Click No if a pop-up about the Web Platform Components appears
  • Double-click on the FastCGI Settings icon
  • In the right pane under actions click Add Application...
  • In the Full Path box type the path to the Python executable, e.g.: c:\python27\python.exe
  • In the Arguments box type the path to wfastcgi.py file, e.g.: c:\python27\Lib\site-packages\wfastcgi.py.

Note

If your path contains spaces be sure to have double quotes(") around the argument

  • Under FastCGI properties, click on (Collection) next to Environment Variables and click on the grey ... box
  • In the EnvironmentVariables Collection Editor dialog, click Add
  • Under Name properties on the right, click the input box to the right of Name, and replace the text Name by DJANGO_SETTINGS_MODULE (capitals is important)
  • As Value enter openremproject.settings
  • Click Add again and add a variable with name PYTHON_PATH and value the path to the openrem path, e.g. C:\Python27\Lib\site-packages\openrem
  • Click Add for the third time and a variable with name WSGI_HANDLER and value django.core.wsgi.get_wsgi_application()
  • Click Ok to close the EnviromentVariables Collection Editor.
  • Under FastCGI properties find the Activity Timeout entry and increase the value to 300. This is to ensure that the server allows enough time for skin dose map data to be calculated.
  • Click Ok to close the Add FastCGI Application dialog
  • Start Windows Explorer and browse to the Python folder, e.g. C:\Python27, or if using a virtualenv browse to that
  • Right click on the Lib sub-folder and choose properties. In the security tab, make sure IIS_IUSRS has Read, Write, Execute and Change permissions for this directory and the subdirectories.
  • Repeat the above step for the Scripts sub-folder, and also for the MEDIA_ROOT folder (as configured in local_settings.py; default c:/Temp/OpenREM/media)
Create a new website
  • In the IIS manager under connections expand the tree under server name
  • Right-click on sites and click Add Website...
  • Enter as sitename OpenREM
  • As physical path enter the same path as the PYTHON_PATH in the FastCGI settings above, e.g. C:\Python27\Lib\site-packages\openrem
  • Set the port to the port you desire. If you wish to use the default port 80, you need to stop and/or remove the default website or change the port of the default website
  • Click OK
Configure the new website
  • In IIS manager double click on the OpenREM website under Sites
  • Double click on the Handler Mappings icon in the middle pane
  • In the right pane, under Actions, click Add Module Mappings or Add Module Mapping..
  • In the Request Path box enter an asterix (*)
  • In the Module box select FastCgiModule (not the CgiModule)
  • In the Executable box enter path\to\python.exe|path\to\wfastcgi.py, e.g.: c:\python27\python.exe|c:\python27\Lib\site-packages\wfastcgi.py. The | character between the two paths is usually to be found with Shift \.
  • In Name type OpenREM cgi handler (value of name is not important)

Note

If one of your paths contains a space use quotations marks around that path. Don’t use quotations marks around the full statement.

  • Click the Request Restrictions button and uncheck the Invoke handler only if request is mapped to: checkbox
  • Click Ok twice to close the Request Restrictions dialog and the Add Module Mapping dialog
  • When prompted Do you want to create a FastCGI application for this executable? click No

The website should work now: browse to http://localhost:port (port is the number you configured the website on. If the port is 80, you can omit the colon and port number).

Note

The website will look “ugly” as the static files (like the css-files) are not yet configured

Configure Django and IIS to serve static files
  • Create a directory called static in your openrem directory, e.g. C:\Python27\Lib\site-packages\openrem\static
  • In the Openrem local_settings.py file, located in the openremproject directory (e.g. C:\Python27\Lib\site-packages\openrem\oprenremproject\local_settings.py) find the STATIC_ROOT variable and set the value to match the directory you just created. The backslashes should be replaced by forward slashed. e.g. STATIC_ROOT = 'C:/Python27/Lib/site-packages/openrem/static'
  • Open a command prompt and navigate to the openrem directory, e.g. C:\Python27\Lib\site-packages\openrem
  • Type python manage.py collectstatic
  • Type Yes to confirm if the static root directory mentioned is correct
  • Close the command prompt by typing exit
  • In IIS right-click on the OpenREM website (under Sites)
  • Click Add Virtual Directory
  • Type static as alias and the path to the static directoy as Physical Path, e.g. C:\Python27\Lib\site-packages\openrem\static
  • Click Ok to close the dialog box
  • Click on the static directory in IIS within the OpenREM site (unfold the OpenREM site)
  • Double click on the Handler Mappings icon in the middle pane
  • On the right pane click View Ordered Lists... under Actions
  • Click on the StaticFile Handler in the middle pane and on Move Up in the right pane until the StaticFile Handler is on the top

Note

You may get a warning that you are detaching the virtual directory. Click Yes on this warning.

Check the website by browsing to http://localhost:port, everything should be fine now.

Running OpenREM on Linux with Gunicorn and NGINX

These instructions are for running OpenREM with Gunicorn and NGINX on Ubuntu Server, but should work on other Linux distributions with minimal modification. These instructions are based on a guide at obeythetestingoat.com

Steps to perform

Note

If you get stuck somewhere in these instructions, please check the Troubleshooting and tips section at the end of this page.

Install NGINX
sudo apt install nginx
sudo systemctl start nginx

You should now be able to see the ‘Welcome to nginx’ page if you go to your server address in a web browser.

Create initial nginx configuration

Create a new config file - you can name the file as you like, but it is usually has the server name.

sudo nano /etc/nginx/sites-available/openrem-server

Start with the following settings - replace the server name with the hostname of your server. For this example, I will use openrem-server

server {
    listen 80;
    server_name openrem-server;

    location / {
        proxy_pass http://localhost:8000;
    }
}

Save and exit (see nano below for tips).

Now delete the default nginx configuration from sites-enabled and make a link to our new one:

sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/openrem-server /etc/nginx/sites-enabled/openrem-server

Now we reload nginx and start our server as before to test this step. At this stage, nginx is simply passing requests to the default port (80) on to port 8000 to be dealt with (which is why we need to start the test server again).

sudo systemctl reload nginx
# activate your virtual environment if you are using one
# navigate to the openrem folder with manage.py in
python manage.py runserver

Now use your web browser to look at your server again - the ‘Welcome to nginx’ page should be replaced by an ugly version of the OpenREM website - this is because the ‘static’ files are not yet being served - we’ll fix this later.

Replace runserver with Gunicorn

Activate your virtualenv if you are using one (add sudo if your aren’t), and:

pip install gunicorn

Make sure you have stopped the test webserver (Ctrl-c in the shell runserver is running in), then from the same openrem folder:

gunicorn openremproject.wsgi:application

The Gunicorn server should start, and you should be able to see the same broken version of the web interface again.

Serve static files using nginx

Create a folder called static somewhere that your webserver user will be able to get to - for example alongside the media folder, and set the permissions. So if you created your media folder in /var/openrem/media, you might do this:

sudo mkdir /var/openrem/static
sudo chown $USER:www-data /var/openrem/static
sudo chmod 755 /var/openrem/static

Now edit your openremproject/local_settings.py config file to put the same path in the STATIC_ROOT:

nano local_settings.py

# Find the static files section
STATIC_ROOT = '/var/openrem/static/'  # replacing path as appropriate

Now use the Django manage.py application to pull all the static files into the new folder:

python manage.py collectstatic

Now we need to tell nginx to serve them:

sudo nano /etc/nginx/sites-available/openrem-server

And modify the file to add the static section - remember to put the path you have used instead of /var/openrem/static

server {
    listen 80;
    server_name openrem-server;

    location /static {
        alias /var/openrem/static;
    }

    location / {
        proxy_pass http://localhost:8000;
    }
}

Now reload nginx and gunicorn to see if it is all working…

sudo systemctl reload nginx
# activate your virtual environment if you are using one
# navigate to the openrem folder with manage.py in
gunicorn openremproject.wsgi:application

Take another look, and it should all be looking nice now!

Switch to using Unix Sockets

This step is optional, but does allow you more flexibility if you need to do anything else on this server using port 8000 as this installation of OpenREM will no longer be using that port. Instead we’ll use ‘sockets’, which are like files on the disk. We put these in /tmp/.

Change the nginx configuration again (sudo nano /etc/nginx/sites-available/openrem-server):

server {
    listen 80;
    server_name openrem-server;

    location /static {
        alias /var/openrem/static;
    }

    location / {
        proxy_pass http://unix:/tmp/openrem-server.socket;
    }
}

Now restart Gunicorn, this time telling it to use the socket, after reloading nginx:

sudo systemctl reload nginx
gunicorn --bind unix:/tmp/openrem-server.socket \
openremproject.wsgi:application

The \ just allows the command to spread to two lines - feel free to put it all on one line.

Check the web interface again, hopefully it should still be working!

Start Gunicorn automatically

We can use systemd on Ubuntu to ensure Gunicorn starts on boot and restarts if it crashes. As before, change each instance of openrem-server for the name of your server. You will need to change the WorkingDirectory to match the path to your openrem folder.

For the gunicorn command, you will need to provide the full path to gunicorn, whether that is in /usr/local/bin/gunicorn or the bin folder of your virtualenv.

# Customise the name of the file as you please - it must end in .service
 sudo nano /etc/systemd/system/gunicorn-openrem-server.service
[Unit]
Description=Gunicorn server for openrem-server

[Service]
Restart=on-failure
User=www-data
WorkingDirectory=/usr/local/lib/python2.7/dist-packages/openrem

ExecStart=/usr/local/bin/gunicorn \
    --bind unix:/tmp/openrem-server.socket \
    openremproject.wsgi:application

[Install]
WantedBy=multi-user.target

Make sure you have customised the User, the WorkingDirectory path, the path to gunicorn, and the name of the socket file.

Warning

If the user you have configured can’t write to the STATIC_ROOT folder, the MEDIA_ROOT folder and the location the logs are configured to be written (usually in MEDIA_ROOT), the systemd gunicorn service is likely to fail when started.

If you have installed everything in your user folder, you are likely to need to set User to your own username.

Now enable the new configuration:

# Load to config
sudo systemctl daemon-reload
# Enable start on boot - change the name as per how you created it
sudo systemctl enable gunicorn-openrem-server.service
# Now start the service
sudo systemctl start gunicorn-openrem-server.service

You might like to see if it worked…

sudo systemctl status gunicorn-openrem-server.service

Look for Active: active (running)

Making use of ALLOWED_HOSTS in local_settings.py

The default setting of ALLOWED_HOSTS is * which isn’t secure, but is convenient! We should really change this to match the hostname of the server.

If your hostname is openrem-server, and the fully qualified domain name is openrem-server.ad.hospital.org and IP address is 10.212.18.209, then you might configure ALLOWED_HOSTS in openremproject/local_settings.py to:

ALLOWED_HOSTS = [
    'openrem-server',
    'openrem-server.ad.hospital.org',
    '10.212.18.209',
]

Note

Which hostnames do I need to put in ALLOWED_HOSTS? You need to put in any hostnames you want people to be able to access your OpenREM web interface at. So if in your hospital you only type in the address bar the hostname (http://openrem-server in this example), then that is all you need to add. If you only use the IP address, then add that. If you can use any of them, add them all :-)

Next we need to edit the nginx configuration again to make sure Django can see the hostname by adding the proxy_set_header configuration (else it gets lost before Django can check it):

sudo nano /etc/nginx/sites-available/openrem-server
server {
    listen 80;
    server_name openrem-server;

    location /static {
        alias /var/openrem/static;
    }

    location / {
        proxy_pass http://unix:/tmp/openrem-server.socket;
        proxy_set_header Host $host;
    }
}

Now reload the nginx configuration and reload Gunicorn:

sudo systemctl reload nginx
sudo systemctl restart gunicorn-openrem-server.service

And check the web interface again. If it doesn’t work due to the ALLOWED_HOSTS setting, you will get a ‘Bad request 400’ error.

Increasing the timeout

You may wish to do this to allow for Skin dose maps that can take more than 30 seconds for complex studies. Both Gunicorn and nginx configurations need to be modified:

sudo nano /etc/systemd/system/gunicorn-openrem-server.service

Add the --timeout setting to the end of the ExecStart command, time is in seconds (300 = 5 minutes, 1200 = 20 minutes)

[Unit]
Description=Gunicorn server for openrem-server

[Service]
Restart=on-failure
User=www-data
WorkingDirectory=/usr/local/lib/python2.7/dist-packages/openrem

ExecStart=/usr/local/bin/gunicorn \
    --bind unix:/tmp/openrem-server.socket \
    openremproject.wsgi:application --timeout 300

[Install]
WantedBy=multi-user.target
sudo nano /etc/nginx/sites-available/openrem-server

Add the proxy_read_timeout setting in seconds (note the trailing s this time).

server {
    listen 80;
    server_name openrem-server;

    location /static {
        alias /var/openrem/static;
    }

    location / {
        proxy_pass http://unix:/tmp/openrem-server.socket;
        proxy_set_header Host $host;
        proxy_read_timeout 300s;
    }
}

Reload everything:

sudo systemctl daemon-reload
sudo systemctl restart gunicorn-openrem-server.service
sudo systemctl reload nginx

Note

If you have jumped straight to here to get the final config, then make sure you substitute all the following values to suit your install:

  • gunicorn-openrem-server.service - name not important (except the .service, but you need to use it in the reload commands etc
  • User=www-data as appropriate. This should either be your user or www-data. You will need to ensure folder permissions correspond
  • WorkingDirectory needs to match the path to your openrem folder (the one with manage.py in)
  • ExecStart=/usr/local/bin/gunicorn \ needs to match the path to your gunicorn executable - either in your virtualenv bin folder or system wide as per the example
  • --bind unix:/tmp/openrem-server.socket \ name in tmp doesn’t matter, needs to match in gunicorn and nginx configs
  • /etc/nginx/sites-available/openrem-server ie name of config file in nginx, doesn’t matter, usually matches hostname
  • server_name openrem-server - should match hostname
  • /var/openrem/static folder must exist, with the right permissions. Location not important, must match setting in local_settings
  • proxy_pass http://unix:/tmp/openrem-server.socket; must match setting in gunicorn config, prefixed with http://

You will also need to collectstatic, symlink the nginx configuration into enabled, enable the gunicorn systemd config to start on reboot, and you should configure the ALLOWED_HOST setting. And you will need to have installed nginx and gunicorn!

Troubleshooting and tips
less

Use less to review files without editing them

  • Navigate using arrow keys, page up and down,
  • Shift-G to go to the end
  • Shift-F to automatically update as new logs are added. Ctrl-C to stop.
  • / to search
nano

Use nano to edit the files.

  • Ctrl-o to save (‘out’)
  • Ctrl-x to exit
Nginx
  • Logs are located in /var/log/nginx/

  • You need root privileges to view the files:

    • To view latest error log: sudo less /var/log/nginx/error.log
  • Reload: sudo systemctl reload nginx

  • Check nginx config: sudo nginx -t

Systemd and Gunicorn
  • Review the logs with sudo journalctl -u gunicorn-openrem-server (change as appropriate for the the name you have used)
  • Check the systemd configuration with systemd-analyze verify /etc/systemd/system/gunicorn-openrem-server.service - again changing the name as appropriate.
  • If you make changes, you need to use sudo systemctl daemon-reload before the changes will take effect.
  • Restart: sudo systemctl restart gunicorn-openrem-server.service

Start all the services

Test web server

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/ (remember to activate the virtualenv)
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\ (remember to activate the virtualenv)

Web access on OpenREM server only

Run the built in web server:

python manage.py runserver --insecure

In a web browser on the same computer, go to http://localhost:8000/ - you should now see the message about creating users. For full functionality start the Celery task queue before moving on to Configure the settings.

Web access on other computers

The built-in webserver only provides a service on the computer OpenREM is installed on by default (it’s only there really for testing). To view the OpenREM interface on another computer, you need to modify the runserver command:

python manage.py runserver --insecure 0.0.0.0:8000

This will enable the web service to be available from other computers on the network. If your server has several network cards and you want to restrict it to one, then you can use a real address rather than 0.0.0.0. Likewise you can specify the port (here it is 8000).

In a web browser on a different computer on the same network, go to http://192.168.1.10:8000/ (changing the IP address to the one you are running the server on) and you should see the OpenREM interface and the message about creating users. For full functionality start the Celery task queue before moving on to Configure the settings.

Note

Why are we using the --insecure option? With DEBUG mode set to True the test web server would serve up the static files. In this release, DEBUG mode is set to False, which prevents the test web server serving those files. The --insecure option allows them to be served again.

Celery task queue

Celery will have been automatically installed with OpenREM, and along with RabbitMQ allows for asynchronous task processing for imports, exports and DICOM networking tasks.

Note

Celery needs to be able to write to the place where the Celery logs and pid file are to be stored, so make sure:

  • the folder exists (the suggestion below is to create a folder in the MEDIA_ROOT location)
  • the user that starts Celery can write to that folder

You can put the folder wherever you like, for example you might like to create a /var/log/openrem/ folder on a linux system.

If you are using the built-in Test web server then Celery and the webserver will be running as your user. If you are running a production webserver, such as Apache or nginx on linux, then the user that runs those daemons will need to be able to write to the MEDIA_ROOT and the Celery log files folder. In this case, you need to change the ownership of the folders and change to the right user before running Celery. On Ubuntu:

mkdir /path/to/media/celery  # change as appropriate
sudo chown www-data /path/to/media  # change as appropriate
sudo su -p www-data

Now start celery…

Move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/ (remember to activate the virtualenv)
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\ (remember to activate the virtualenv)

Linux - \ is the line continuation character:

celery multi start default -Ofair -A openremproject -c 4 -Q default \
--pidfile=/path/to/media/celery/%N.pid --logfile=/path/to/media/celery/%N.log

Windows - celery multi doesn’t work on Windows, and ^ is the continuation character:

celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default ^
--pidfile=C:\path\to\media\celery\default.pid --logfile=C:\path\to\media\celery\default.log

For production use, see Daemonising Celery below

Set the number of workers (concurrency, -c) as you see fit. The more you have, the more processes (imports, exports, query-retrieve operations etc) can take place simultaneously. However, each extra worker uses extra memory and if you have too many they will be competing for CPU resources too.

Note

Problems with Celery 4 on Windows

Full support for Celery on Windows was dropped with version 4 due to lack of Windows based developers. However, using the settings above (and as suggested in Daemonising Celery on Windows) have been shown to work well in production. It is possible that there may be issues with particular releases though - Celery 4.0.0 his known to work well. If you need to install a particular version of Celery, use:

pip install celery==4.0.0

To stop the celery queues in Linux:

celery multi stop default --pidfile=/path/to/media/celery/%N.pid

For Windows, just press Ctrl+c

You will need to do this twice if there are running tasks you wish to kill.

Celery periodic tasks: beat

Note

Celery beat is only required if you are using the Native DICOM store node with direct import. Please read the warnings there before deciding if you need to run Celery beat. At the current time, using a third party DICOM store service is recommended for most users. See the Third-party DICOM Stores documentation for more details

Celery beat is a scheduler. If it is running, then every 60 seconds a task is run to check if any of the DICOM Store SCP nodes are set to keep_alive, and if they are, it tries to verify they are running with a DICOM echo. If this is not successful, then the Store SCP is started.

To run celery beat, open a new shell and move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/ (remember to activate the virtualenv)
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\ (remember to activate the virtualenv)

Linux:

celery -A openremproject beat -s /path/to/media/celery/celerybeat-schedule \
-f /path/to/media/celery/celerybeat.log \
--pidfile=/path/to/media/celery/celerybeat.pid

Windows:

celery -A openremproject beat -s C:\path\to\media\celery\celerybeat-schedule ^
-f C:\path\to\media\celery\celerybeat.log ^
--pidfile=C:\path\to\media\celery\celerybeat.pid

For production use, see Daemonising Celery below

As with starting the Celery workers, the folder that the pid, log and for beat, schedule files are to be written must already exist and the user starting Celery beat must be able write to that folder.

To stop Celery beat, just press Ctrl+c

Configure the settings

  • Follow the link presented on the front page to get to the user and group administration.
Initial home page with no users in groups
Configuration menu
  • After the first users are configured, this link will no longer be presented and instead you can go to Config -> Users.

  • You will need the superuser username and password you created just after creating the database. The groups are

    • viewgroup can browse the data only
    • importsizegroup can use the csv import facility to add patient height and weight information
    • importqrgroup can use the DICOM query-retrieve facility to pull in studies, as long as they are pre-configured
    • exportgroup can view and export data to a spreadsheet
    • pidgroup can search using patient names and IDs depending on settings, and export with patient names and IDs if they are also a member of the exportgroup
    • admingroup can delete studies, configure DICOM Store/QR settings, configure DICOM keep or delete settings, configure patient ID settings, and abort and delete patient size import jobs. Members of the admingroup no longer inherit the other groups permissions.
Selecting groups in Django user admin
  • In addition to adding users to these groups, you may like to grant a second user superuser and staff status so that there are at least two people who can manage the users
  • Return to the OpenREM interface (click on View site at the top right)
Link from Django user admin back to OpenREM
  • Follow the link to see more information about how you want OpenREM to identify non-patient exposures, such as QA. See Not-patient indicator settings.
  • Go to Config -> DICOM object delete settings and configure appropriately (see Delete objects configuration)
  • Go to Config -> Patient ID settings and configure appropriately (see Patient identifiable data)
  • If you want to use OpenREM as a DICOM store, or to use OpenREM to query remote systems, go to Config -> Dicom network configuration. For more information go to Importing data to OpenREM.
  • With data in the system, you will want to go to Config -> View and edit display names and customise the display names. An established system will have several entries for each device, from each time the software version, station name or other elements changes. See Display names and user-defined modalities for more information

Start using it - add some data!

See Importing data to OpenREM

Further instructions

Daemonising Celery

In a production environment, Celery will need to start automatically and not depend on a particular user being logged in. Therefore, much like the webserver, it will need to be daemonised.

Daemonising Celery on Windows

To ensure that the Celery task queue is started at system start-up it is advisable to launch Celery using a batch file and configure Windows Task Scheduler to run this at system start-up.

Celery will sometimes fall over during the execution of a long task. In this situation it will not restart on its own. Windows Task Scheduler can be used to restart Celery on a regular basis. In addition it can be used to ensure celery is running a few minutes prior to a scheduled PACS query.

An example batch file is shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
 :: Change to the drive on which OpenREM is installed and navigate to the
 :: OpenREM folder
 D:
 CD D:\Server_Apps\python27\Lib\site-packages\openrem

 :: Attempt to shutdown celery gracefully
 celery -A openremproject control shutdown --timeout=10

 :: Pause this file for 10 s to ensure that the above has time to work (you may
 :: need to check that the 'timeout' command is available on your Windows
 :: system. Some systems may have 'sleep' instead, in which case replace the
 :: line below with:
 :: SLEEP 10
 TIMEOUT /T 10

 :: Kill any remaining celery tasks (ungraceful) and delete the pid file in case
 :: the above graceful shutdown did not work
 TASKKILL /IM /F celery.exe
 DEL /F E:\media_root\celery\default.pid

 :: Restart a new instance of celery
 celery worker -n default -P solo -Ofair -A openremproject -c 4 -Q default --pidfile=e:\media_root\celery\default.pid --logfile=e:\media_root\celery\default.log

Lines 3 and 4 navigate to the OpenREM drive and folder. Line 7 attempts to gracefully shutdown celery. Line 18 kills any celery.exe processes that are currently running in case the graceful shutdown didn’t work. Line 19 deletes the default.pid process ID file that exists in the celery log file location. Celery won’t restart if this pid file exists. Finally, line 22 runs a new instance of celery. If you wish to use this example you will have to ensure that the drive letters and paths are changed to match your own OpenREM system installation.

Setting up a scheduled task

Open Task Scheduler on the OpenREM server and then click on the Task Scheduler Library item in the left-hand pane. This should look something like figure 1 below, but without the OpenREM tasks present.

Task scheduler overview

Figure 1: An overview of Windows Task Scheduler

To create a new task for celery click on Create Task... in the Actions menu in the right-hand pane. Give the task a name and description. You may wish to use something similar to the following:

Attempts to gracefully shutdown any existing celery instances, then kills any remaining
celery instances after 10 seconds. Deletes the celery pid file and then starts a new
instance of celery. This is used to queue tasks for the OpenREM patient dose management
system. Celery can fail after running a long task and does not automtically restart.
This task is therefore also set to run 15 minutes before each scheduled OpenREM PACS
query to ensure that celery is up and running for these.

Next, click on the Change User or Group button and type system in to the box, then click Check Names, then click OK. This sets the server’s SYSTEM user to run the task. Also check the Run with highest prilileges box. Your task should now look similar to figure 2.

Task scheduler overview

Figure 2: General properties

Next, click on the Triggers tab so that you can set when the task will be run. As a minimum you should add an At startup trigger. To do this, click New.... In the dialogue box that appears select At startup from the Begin the task options and ensure that the Enabled checkbox is selected. Then click OK. You may wish to add other triggers that take place at specific times during the day, as shown in figure 3.

In the example shown in figure 3 celery is started at system start up, and restarted multiple times each day to ensure that it is running before any PACS queries. Your requirements may be more straightforward than this example.

Task scheduler overview

Figure 3: Trigger properties

Now click on the Actions tab so that you can add the action that is taken when the task is run. Click on New..., and in the dialogue box that appears select Start a program as the Action. Click on Browse and select the celery batch file that you created earlier. Click OK to close the New Action dialogue box. Figure 4 shows an example of the the Actions tab.

Task scheduler overview

Figure 4: Action properties

There are no particular conditions set for the task, as shown in figure 5.

Task scheduler overview

Figure 5: Condition properties

Finally, click on the Settings tab (figure 6). Check the Allow task to be run on demand box, and also the If the running task does not end when requested, force it to stop box. Choose Stop the existing instance from the If the task is already running, then the following rule applies: list. Then click the OK button to add the task to the scheduler library.

Task scheduler overview

Figure 6: Task settings

Configuration and adminstration

See also: local_settings.py Configuration

Delete objects configuration

OpenREM is able to automatically delete DICOM objects if they can’t be used by OpenREM or if they have been processed. This has the following advantages:

  • The server doesn’t need to have much storage space
  • It can help with information governance if the database is set to not store patient identifiable data (see Patient identifiable data)

Warning

If OpenREM is set to delete objects and you pass a local file to OpenREM using the command line, the source file will be deleted (as long as the filesystem permissions allow).

Configure what is deleted

Config menu

The Config menu

Use the Config menu and select DICOM object deletion:

This will open the configuration page:

Modify DICOM object delete settings

Modify DICOM object deletion policy

The initial settings are to not delete anything. However, you are likely to want to delete objects that don’t match any import filters, and also to delete images such as mammo, DX and Philips CT, as these will take up space much more quickly than the radiation dose structured reports.

Reviewing the settings

When you have set your preferences, you will be redirected to the DICOM network configuration page, where at the bottom you can review the current settings:

DICOM object deletion poligy review on DICOM config page

Deletion policies can be reviewed on the DICOM network configuration page

More information about the DICOM network configuration can be found on the Direct from modalities page.

Display names and user-defined modalities

Functionality changed in 0.8.0

The display name field

Previous versions of OpenREM used each X-ray system’s DICOM station name as the identifier for each X-ray system. The front page showed a summary of the number of studies for each unique station name stored in the system. This led to a problem if multiple X-ray systems used the same station name: the OpenREM home page would only show one station name entry for these systems, with the number of studies corresponding to the total from all the rooms. The name shown alongside the total was that of the system that had most recently sent data to the system.

This issue has been resolved by introducing a new field called display name. This is unique to each piece of X-ray equipment, based on the combination of the following eight fields:

  • manufacturer
  • institution name
  • station name
  • department name
  • model name
  • device serial number
  • software version
  • gantry id

The default text for display name is set to a combination of institution name and station name. The default display name text can be changed by a user in the admingroup — see Changing X-ray system display names and user defined modality

User defined modality field

OpenREM determines the modality type of a system based on the information in the DICOM radiation dose structured report. However sometimes this mechanism fails because vendors use templates meant for RF also for DX systems. Therefore it is possible from version 0.8.0 to set a modality type for each system manually. A manually set modality type overrides the automatically determined value.

Viewing X-ray system display names and user defined modality

User options menu

The Config menu (user)

If you log in as a normal user then the Config menu becomes available at the right-hand end of the navigation bar at the top of the screen.

The third option, View display names & modality, takes you to a page where you can view the list of X-ray systems with data in OpenREM together with their current display name and user defined modality. If the user defined modality is not set, the value contains None. The X-ray systems are grouped into modalities and displayed in five tables: CT; mammography; DX and CR; fluoroscopy; and other.

List of current display names

Example list of display names

Changing X-ray system display names and user defined modality

Admin menu

The Config menu (admin)

If you wish to make changes to a display name or to the user defined modality then you must log in as a user that is in the admingroup. You will then be able to use the Display names & modality item under the Config menu:

This will take you to a page where you can view the list of X-ray systems with data in OpenREM. If you wish to change a display name or the user defined modality then click on the corresponding row. The resulting page will allow you to edit these parameters. Click on the Update button to confirm your changes:

Update a display name

Example of the page for updating a display name and user defined modality

You can change multiple rows at once. For display names you may wish to do this if a system has a software upgrade, for example, as this will generate a new default display name for studies carried out after the software upgrade has taken place. The studies from these will be grouped together as a single entry on the OpenREM homepage and individual modality pages.

If you update the user defined modality, the modality type for already imported studies will also be set to the user defined modality type. Only changes from modality DX (planar X-ray) to RF (fluoroscopy) and vice versa are possible.

Dual modality systems

Some systems are dual purpose in that they can be used in both standard planar X-ray mode and in fluoroscopy mode. For these systems you can configure them as ‘Dual’ and OpenREM will attempt to reprocess all the studies related to the rows you have selected and assign them to DX or RF. The studies will then be displayed in the right sections in the web interface and will export correctly. New RDSRs relating to that X-ray system will be assigned a modality in the same way.

After an X-ray system has been set to Dual you may wish to reprocess the studies to assign modality again. To do this you can use the ‘reprocess’ link in the ‘User defined modality’ cell:

Reprocess Dual link

Re-sort studies into planar X-ray and fluoroscopy

Not-patient indicator settings

The standard configuration for OpenREM is to not store any patient identifiable information. Therefore it can be difficult to distinguish between real patients and test or quality assurance exposures.

Changed in 0.8.0

To aid identification of non-patient exposures, the patient name and the patient ID are checked against a set of patterns, and if a match is found then the pattern is recorded in the database before the patient name and ID are deleted or converted to a hash (see Patient identifiable data for details).

Setting the patterns to identify non-patient studies

Config menu

The Config menu

Use the Config menu and select Not-patient indicators:

The patient name and the ID are matched against the patterns you configure. The patterns make use of wildcards as per the following table, and are case insensitive:

Pattern Meaning
* matches everything
? matches any single character
[seq] matches any character in seq
[!seq] matches any character not in seq

To match all studies where the patient name begins with physics, the pattern should be set to physics*. This would match Physics^RoutIQ but not match Testing^Physics. The patient name in DICOM is normally formatted Family name^Given name^Middle name^Prefix^Suffix. Therefore to match any studies where the first name is Test, you would set the pattern to be *^test*.

If your test patient name always starts with PHY and then a number, you might use this pattern: phy[0-9]*. Here we have used a range for the sequence to match any number, but it will only match one character per sequence, so a * is required to match all the characters after the first number. This pattern will match Phy12345 and PHY6test but not Phyliss.

The pattern list for patient name and the list for patient ID are separate, so both need to be populated to meet your requirements.

Creating new patterns

Click on Add ID patterns or Add name patterns in the panel title bar and follow the instructions.

Modifying patterns

Click the Modify link in the row of the pattern you wish to modify.

Deleting patterns

Click the Delete link in the row of the pattern you wish to delete. You will be asked to confirm the deletion.

Replicating behaviour of release 0.7.4 and earlier

OpenREM releases before 0.8 had the not-patient identification patterns hard-coded. From release 0.8.0 the patterns are (admin) user configurable, but will start with no patterns in place. To add the patterns that would maintain the behaviour of previous releases, use the link at the bottom of the config page, or the link in the add/modify pages.

Patient identifiable data

Prior to version 0.7, no data that is generally considered to be patient identifiable was stored in the OpenREM database.

The following patient descriptors have always been recorded if they were available:

  • Patient age at the time of the study, but not date of birth (though this could be calculated from age)
  • Patient sex
  • Patient height
  • Patient weight

In addition, a key identifier for the exam that is normally not considered patient identifiable was stored:

  • Study accession number

It has become apparent that there are reasons where people need to store patient identifiable data to make the most of OpenREM, so this is now configurable from version 0.7 onwards.

Configure what is stored

Configuration menu

On the Config menu, select Patient ID:

The initial settings are as follows:

Modify patient identifiable data settings

The default for patient name, ID and date of birth is to not store them. There isn’t an option currently to not store the accession number, though OpenREM continues to work if it is missing.

To store patient identifiable data from now on, select the relevant box and press Submit. If you change the setting again later, then data already stored will remain in the database.

Store encrypted data only

If you wish to have the patient name and/or ID available for finding studies relating to a specific patient, but do not need to identify who that patient is, then it is possible to create an ‘encrypted’ version of the ID or name. In this case, a one-way SHA 256 hash is generated and the hash value is stored instead.

If exactly the same name or ID (including spelling, spacing, case etc) occurs more than once, then the same hash will be generated.

The same applies to accession numbers if the option to encrypt the accession number is selected.

Using patient identifiable data

Querying for patient studies

In the modality pages of the OpenREM web interface, if you are in the pidgroup you will have a filter for patient name and patient ID available:

Patient name and ID in search filter

If the values in the database are not encrypted, then partial search terms can be used as a case-insensitive ‘contains’ query will be applied.

If the values are encrypted, then only the entire string, with exactly the same case, spacing and punctuation will match. This is more likely to be successful with patient ID than with patient name.

Study export with patient identifiers

Users in the pidgroup will have extra export buttons available in the modality pages:

Export buttons for pidgroup users

If the IDs or names are encrypted, then these columns will contain the hash rather than the original values. However, it will be possible to see if more than one study belongs to one patient as the values should be the same for both. Due to the nature of the algorithm however, a single change in the name or ID - such as an upper case letter instead of a lower case one - will be recorded as a completely different hash value.

Any exports with either patient name or patient ID included will also have a date of birth column.

Deleting studies

Individual studies

If you log in as a user that is in the admingroup, then an extra column is appended in the filtered view tables to allow studies to be deleted:

Deleting studies

Clicking on delete takes you to a confirmation page before the delete takes place.

All studies from one source

Admin menu

The Config menu (admin)

If you log in as a user that is in the admingroup, on the Config menu select Display names & modality to get to a list of all the X-ray systems with data in OpenREM. More information about Display names and user-defined modalities.

Each row is a unique combination of all the column headers, so if a modality has a software update for example this will usually mean a new row is started.

In the last column is a link to Review the studies from that source. This can be useful for troubleshooting a particular source, or you can use it to delete all the studies from one source in one go.

Source study review

Source equipment review page with study delete options

The details for that source are displayed, along with a table showing which sort of data is contained in each study. Above the ‘Study deletion options’ panel the following two numbers are indicated:

  1. The number of studies associated with this equipment
  2. The number of studies associated with this equipment after being filtered by the indicated modality type

If the second number is smaller than the first, this will indicate that some of the studies from the equipment have been labelled with a different modality type. There will therefore be an entry in one of the other tables on the equipment display name page.

Delete studies and table entry

Use this button if you want to delete all the studies and remove the entry that has been made in the Unique Equipment Names table. Otherwise, the entry would remain but with zero studies associated with it. The deletion takes a second confirmation step.

If there are studies associated with this equipment that are listed with a modality type different to the one shown, those studies will not be deleted and the table entry will not be removed.

Delete studies

If you have associated this table entry with a Display name and you want any future studies to fall under the same name, you can leave the entry in the Unique Equipment Names table. You might want to do this for example if you have a Display name of ‘CR’ or ‘Imported’. Again, there is a confirmation step.

Again, only the studies associated with this equipment that have the same modality type as shown will be deleted.

Adding patient size information from csv using the web interface

Uploading patient size data

If you log in as a user that is in the admingroup, then a menu is available at the right hand end of the navigation bar:

Admin import patient size data menu

The first option takes you to a page where you can upload a csv file containing details of the patient height and weight, plus either the accession number or the Study Instance UID.

Uploading CSV files containing patient size information Upload patient size csv file button

The csv file needs to have at least the required columns. Additional columns will be ignored. If your source of patient size data does not have either the height or the weight column, simply add a new empty column with just the title in the first row.

When you have selected the csv file, press the button to upload it.

Importing the size data to the database

On the next page select the column header that corresponds to each of the head, weight and ID fields. Also select whether the ID field is an Accession number or a Study UID:

When the column headers are selected, click the ‘Process the data’ button.

Selecting header information

The progress of the import is then reported on the patient size imports page:

Patient size importing

During the import, it is possible to abort the process by clicking the button seen in the image above. The log file is available from the completed table whether it completed or not - there is no indication that the import was aborted.

As soon as the import is complete, the source csv file is deleted from the server.

Reviewing previous imports

After an import is complete, it is listed in the completed import tasks table. You can also get to this page from the Admin menu:

Imports link

For each import, there is a link to the logfile, which looks something like this. With this import accession numbers weren’t available so the patient size information was matched to the study instance UID:

Size import logs

Deleting import logs

The completed import tasks table also has a delete check box against each record and a delete button at the bottom. The csv file originally imported has already been deleted - this delete function is to remove the record of the import and the log file associated with it from the database/disk.

Adding patient size information from csv using the command line

Usage:

openrem_ptsizecsv.py [-h] [-u] [-v] csvfile id height weight
-h, --help
Print the help text.
-u, --si-uid
Use Study Instance UID instead of Accession Number.
-v, --verbose
New in 0.3.7 Print to the standard output the success or otherwise of inserting each value.
csvfile
csv file containing the height and/or weight information and study identifier. Other columns will be ignored. Use quotes if the filepath has spaces.
id
Column title for the accession number or study instance UID. Use quotes if the title has spaces.
height
Column title for the patient height (DICOM size) - if this information is missing simply add a blank column with a suitable title. Use quotes if the title has spaces.
weight
Column title for the patient weight - if this information is missing simply add a blank column with a suitable title. Use quotes if the title has spaces.

Importing data to OpenREM

From local DICOM files

If you have RDSRs, DX or MG images or Philips CT Dose Info images, you can import them directly into OpenREM:

Importing from DICOM files

If you are using linux, or for Windows if you have put C:\Python27\;C:\Python27\Lib\site-packages;C:\Python27\Scripts onto your system path, you should be able to import from the command line:

Radiation Dose Structured Reports
openrem_rdsr.py filename.dcm

You can use wildcards to process a number of files at once, ie:

openrem_rdsr.py *.dcm
Cumulative and continued study RDSRs
Background
Cumulative RDSRs

Some modalities are configured to send an RDSR after every exposure, with each new RDSR containing a complete record of the examination up to that point. For example, this is what the current version of the Siemens CT scanner software does.

Continued study RDSRs

On most systems the RDSR is sent when the study is completed. If the study is then restarted, the system must create a new RDSR. On a Siemens CT system, this new RDSR will have the same Study Instance UID and the same accession number, but the content will only refer to the continued study, not the original study.

Pre-0.8.0 OpenREM behaviour

Prior to release 0.8.0, OpenREM would check the Study Instance UID on import and check the value against the existing studies in the database. If a match was found, then the new RDSR was rejected on the basis that it must be a duplicate.

This would therefore ignore both cumulative and continued study RDSRs which means your database might be filled with single event studies, and you won’t have details of any continued studies.

Current OpenREM behaviour
New imports

On import of the first RDSR in a study, the SOP Instance UID of the RDSR is recorded with the study. This is an ID that is unique to that RDSR object - any further RDSRs might have the same Study Instance UID, but will always have a different SOP Instance UID.

When the second RDSR is imported, the duplicate StudyInstanceUID will trigger OpenREM to check the SOP Instance UID of the new RDSR against the one(s) stored with that study. If there is a match, the new RDSR is ignored as it has already been processed. If it does not match, then the Irradiation Event UID of each exposure in the new RDSR is compared to the Irradiation Event UIDs already in the database for that study, to establish if the new RDSR carries new information that should be imported.

In the case of a cumulative RDSR that is sent after each event, the original study is deleted from the database and is replaced by the newer one if it has additional events.

In the case of a continued study RDSR which has a completely different set of events, the new RDSR is imported alongside the existing one.

Existing studies imported before 0.8.0

RDSRs imported before upgrading to 0.8.0 will not have the SOP Instance UID recorded in the database and so the new RDSR will be compared at event level with the existing study before making an import decision, as with new studies.

Fixing existing studies
Importing from file

If you are have a store of the RDSRs that were previously rejected, import them all again and this time they should be processed properly.

For example on my system, using linux, each scanner started sending per-exposure RDSRs from the date they were upgraded. I found the RDSRs from that date to the date I upgraded OpenREM and imported them:

touch --date "2018-01-06" tmpdate20180106
touch --date "2018-02-07" tmpdate20180207
find RDSRs/ -newer tmpdate20180106 ! -newer tmpdate20180207 -name *.dcm -exec openrem_rdsr.py {} \;
Importing via query-retrieve

The query-retrieve duplicates processing has been updated to compare SOP Instance UIDs returned by the remote node (the PACS) with the SOP Instance UIDs stored with each study in OpenREM. Therefore, after an initial import of each RDSR in your search, any subsequent query should drop any RDSRs that have previously been processed and not move them a second time.

For mammography DICOM images
openrem_mg.py filename.dcm

The facility for extracting dose information from mammography DICOM images has been designed and tested with images created with the GE Senographe DS. It has now also been used with the images generated by the following systems:

  • GE Senographe Essential
  • Hologic Selenia
  • Siemens Inspiration
For radiographic DICOM images
openrem_dx.py filename.dcm
For CT dose summary files from Philips CT scanners
openrem_ctphilips.py filename.dcm

This extractor makes use of the information stored in the header data of the Philips Secondary Capture object with a series description of ‘Dose Info’. The value inserted into ‘Study description’ in the OpenREM database is actually taken from the Protocol field. The value in Study description is inserted into the study level comment field in the database, along with the protocol file name and any ‘comments on radiation dose’.

For CT dose summary files from older Toshiba CT scanners
openrem_cttoshiba.py path_to_files

This extractor is designed to create a DICOM radiation dose structured report from the information contained in secondary capture dose summary images, supplemented by data stored in image tags. It requires a folder of DICOM objects as input (suitable data can be retrieved from a DICOM node using the qrscu.py command with the -toshiba switch - see Query-retrieve using the command line interface). It creates an initial RDSR from the secondary capture dose summary, and then tries to enrich this with additional information contained in image tags. The routine attempts to extract the following information from the image tags and insert it into the initial RDSR:

Study-level information
  • Study description
  • Requested procedure description
  • Software versions
  • Device serial number
Series-level information
  • Protocol name
  • Exposure time (per rotation)
  • kVp
  • Spiral pitch factor
  • Nominal total collimation width
  • Nominal single collimation width
  • Exposure modulation type

The routine was developed for older Toshiba CT scanners that cannot create RDSR objects themselves. It is known to work with:

  • Toshiba CX, software version V4.40ER011
  • Toshiba CXL, software version V4.51ER014
  • Toshiba CXL, software version V4.86ER008 (this software version can produce RDSR objects directly, but may not populate some fields, such as requested procedure name and study description)

This extractor has also been used successfully on images from a GE LightSpeed Plus scanner, although in this case no supplementary data is extracted from image tags.

If you want some examples, you can find the DICOM files that we use for the automated testing in the openrem/remapp/tests/test_files folder in your OpenREM installation.

Direct from modalities

For production use, you will either need the modalities to send the RDSR or images directly to your OpenREM server using DICOM, or you will need to use query-retrieve to fetch the DICOM objects from the PACS or the modalities. In either of these situations, you will need to run a DICOM Store service on your OpenREM server.

To get started, you can make use of the in-built DICOM store that can be configured from within OpenREM:

DICOM Network Configuration

Configuring DICOM store nodes in OpenREM

You need to configure one or more DICOM Store nodes (Store Service Class Provider, or Store SCP) if you want either of the following:

  • OpenREM to provide DICOM store functionality
  • OpenREM to be able to query retrieve a third-party system (PACS or modality), using the OpenREM Store SCP or a third party one, such as Conquest

To configure a DICOM Store SCP, on the Config menu select DICOM networking, then click Add new Store and fill in the details (see figure 1):

DICOM Store SCP configuration

Figure 1: DICOM Store SCP configuration

  • Name of local store node: This is the friendly name, such as OpenREM store
  • Application Entity Title of the node: This is the DICOM name for the store, and must be letters or numbers only, no spaces, and a maximum of 16 characters
  • Port for store node: Port 104 is the reserved DICOM port, but it is common to use high ports such as 8104, partly because ports up to 1024 usually need more privileges than for the high ports. However, if there is a firewall between the remote nodes (modalities, PACS) and the OpenREM server, then you need to make sure that the firewall is configured to allow the port you choose here
Native DICOM store node with direct import

Warning

Native DICOM store functionality has not proved to be stable over long periods. Therefore we cannot recommend that you use this feature in a production environment. However, please do test it and help us to improve it if you are able to!

Warning

If you use supervisord or similar on Linux, then you might not be able to use the web interface or possibly the auto-start service as new threads spawned for the Store SCP tend to get killed. This wouldn’t prevent you starting the SCP in a shell. See Issue #337

An OpenREM DICOM Store SCP (service class provider) enables modalities or PACS to send DICOM structured reports and images directly to OpenREM where they are imported into the database.

The Store SCP service receives the data, checks whether it is one of the objects that OpenREM can extract data from, and starts an import task if applicable.

The object is then left in the dicom_in folder in the media folder, or it is deleted, depending on the policy set in Delete objects configuration.

For native DICOM store nodes, you need to open the Advanced - test/development use only section (see figure 2):

DICOM Store SCP advanced configuration

Figure 2: DICOM Store SCP advanced configuration

  • Control the server using OpenREM: this checkbox will enable OpenREM to create and control the node
  • Auto-start the server using celery beat: if checked, and if Celery periodic tasks: beat is running, then OpenREM will attempt to start the store node whenever it finds it not to be running.
Third-party DICOM store node for scripted import to OpenREM

If you are using Conquest or another third-party Store SCP to collect DICOM data, simply fill in the basic details as above without configuring the settings in the Advanced section. This will enable you to request remote hosts send data to your Store SCP in the retrieve part of the query-retrieve operation.

See Conquest DICOM store node on Ubuntu 16.04 and Running Conquest on Windows as a service for more information about using Conquest with OpenREM

Status of DICOM Store SCP nodes

DICOM Store SCP advanced configuration

DICOM Store SCP status "Alive"
DICOM Store SCP status "Association fail"

Figure 3: DICOM Store SCP status - Alive and Association failed

DICOM Store SCP nodes that have been configured are listed in the left column of the DICOM network configuration page. For each server, the basic details are displayed, including the Database ID which is required for command line/scripted use of the query-retrieve function.

In the title row of the Store SCP config panel, the status will be reported either as ‘Server is alive’ or ‘Error: Association fail - server not running?’ - see figure 3

Controlling native Store SCP nodes

If a native Store SCP node is not running, then a Start server button will be presented at the bottom right. If it is running, this buttin will change to Stop server, and the Delete button will become inactive.

If the node is configured to be auto-started, and if Celery periodic tasks: beat is running, then each minute if the server is not started Celery will try to start the node. If you intend to stop the node for some reason, modify the configuration so that auto-start is not selected, then stop the server.

Query retrieve of third-party system, such as a PACS or modality

To Query-Retrieve a remote host, you will need to configure both a local Store SCP and the remote host.

To configure a remote query retrieve SCP, on the Config menu select DICOM networking, then click Add new QR Node and fill in the details:

  • Name of QR node: This is the friendly name, such as PACS QR
  • AE Title of the remote node: This is the DICOM name of the remote node, 16 or fewer letters and numbers, no spaces
  • AE Title this server: This is the DICOM name that the query (DICOM C-Find) will come from. This may be important if the remote node filters access based on calling aet. Normal rules of 16 or fewer letters and numbers, no spaces
  • Remote port: Enter the port the remote node is using (eg 104)
  • Remote IP address: The IP address of the remote node, for example 192.168.1.100
  • Remote hostname: Alternatively, if your network has a DNS server that can resolve the hostnames, you can enter the hostname instead. If the hostname is entered, it will be used in preference to the IP address, so only enter it if you know it will be resolved.

Now go to the DICOM Query Retrieve Service documentation to learn how to use it.

Troubleshooting: openrem_store.log

If the default logging settings haven’t been changed then there will be a log files to refer to. The default location is within your MEDIAROOT folder:

This file contains information about each echo and association that is made against the store node, and any objects that are sent to it.

The following is an example of the log for a Philips dose info image being received:

[21/Feb/2016 21:13:43] INFO [remapp.netdicom.storescp:310] Starting AE... AET:MYSTOREAE01, port:8104
[21/Feb/2016 21:13:43] INFO [remapp.netdicom.storescp:314] Started AE... AET:MYSTOREAE01, port:8104
[21/Feb/2016 21:13:43] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:13:44] INFO [remapp.netdicom.storescp:54] Store SCP: Echo received
[21/Feb/2016 21:13:46] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:13:46] INFO [remapp.netdicom.storescp:54] Store SCP: Echo received
[21/Feb/2016 21:13:49] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:13:49] INFO [remapp.netdicom.storescp:54] Store SCP: Echo received
[21/Feb/2016 21:13:50] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:13:50] INFO [remapp.netdicom.storescp:54] Store SCP: Echo received
[21/Feb/2016 21:13:51] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:13:51] INFO [remapp.netdicom.storescp:54] Store SCP: Echo received
[21/Feb/2016 21:14:39] INFO [remapp.netdicom.storescp:46] Store SCP: association requested
[21/Feb/2016 21:14:39] INFO [remapp.netdicom.storescp:78] Received C-Store. Stn name NM-54316, Modality CT,
SOPClassUID Secondary Capture Image Storage, Study UID 1.2.840.113564.9.1.2843752344.47.2.5000947881 and Instance
UID 1.2.840.113704.7.1.1.4188.1234134540.349
[21/Feb/2016 21:14:39] INFO [remapp.netdicom.storescp:232] File
/var/openrem/media/dicom_in/1.2.840.113704.7.1.1.4188.1453134540.349.dcm written
[21/Feb/2016 21:14:39] INFO [remapp.netdicom.storescp:263] Processing as Philips Dose Info series
...etc

Third-party DICOM Stores

The DICOM store built in to OpenREM hasn’t proved to be stable over the longer term with the current implementation and library that it depends on. This will be rectified in a future version, but for now we recommend you use a third-party DICOM store. Previous releases have recommended the Conquest DICOM server which is very good for this task. However, due to difficulties with installation on some platforms, we are now recommending the Orthanc DICOM server instead:

Orthanc Store configuration
Create Lua file

Open the following link in a new tab and copy and paste the content into a text editor. Save this as a new file called openrem_orthanc_config.lua. This can be saved anywhere, provided that Orthanc is able to access it:

Edit the top two sections
-------------------------------------------------------------------------------------
-- OpenREM python environment and other settings

-- Set this to the path and name of the python executable used by OpenREM
local python_executable = 'D:\\Server_Apps\\python27\\python.exe'

-- Set this to the path of the python scripts folder used by OpenREM
local python_scripts_path = 'D:\\Server_Apps\\python27\\Scripts\\'

-- Set this to the path where you want Orthanc to temporarily store DICOM files
local temp_path = 'E:\\conquest\\dicom\\'

-- Set this to 'mkdir' on Windows, or 'mkdir -p' on Linux
local mkdir_cmd = 'mkdir'

-- Set this to '\\'' on Windows, or '/' on Linux
local dir_sep = '\\'

-- Set this to true if you want Orthanc to keep physics test studies, and have it
-- put them in the physics_to_keep_folder. Set it to false to disable this feature
local use_physics_filtering = true

-- Set this to the path where you want to keep physics-related DICOM images
local physics_to_keep_folder = 'E:\\conquest\\dicom\\physics\\'

-- Set this to the path and name of your zip utility (used with physics-related images)
local zip_executable = 'D:\\Server_Apps\\7zip\\7za.exe a'

-- Set this to the path and name of your remove folder command, including switches
-- for it to be quiet (used with physics-related images)
local rmdir_cmd = 'rmdir /s/q'
-------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------
-- User-defined lists that determine how Orthanc deals with certain studies

-- A list to check against patient name and ID to see if the images should be kept.
-- Orthanc will put anything that matches this in the physics_to_keep_folder.
local physics_to_keep = {'physics'}

-- Lists of things to ignore. Orthanc will ignore anything matching the content of
-- these lists: they will not be imported into OpenREM.
local manufacturers_to_ignore = {'Agfa', 'Agfa-Gevaert', 'Agfa-Gevaert AG', 'Faxitron X-Ray LLC', 'Gendex-KaVo'}
local model_names_to_ignore = {'CR 85', 'CR 75', 'CR 35', 'CR 25', 'ADC_5146', 'CR975'}
local station_names_to_ignore = {'CR85 Main', 'CR75 Main'}
local software_versions_to_ignore = {'VixWin Platinum v3.3'}
local device_serial_numbers_to_ignore = {'SCB1312016'}

-- Set this to true if you want to use the OpenREM Toshiba CT extractor. Set it to
-- false to disable this feature.
local use_toshiba_ct_extractor = true

-- A list of CT make and model pairs that are known to have worked with the Toshiba CT extractor
local toshiba_extractor_systems = {
        {'GE Medical Systems', 'Discovery 710'},
        {'GE Medical Systems', 'Discovery ste'},
        {'GE Medical Systems', 'Brightspeed'},
        {'GE Medical Systems', 'Lightspeed Plus'},
        {'GE Medical Systems', 'Lightspeed16'},
        {'GE Medical Systems', 'Lightspeed Pro 32'},
        {'GE Medical Systems', 'Lightspeed VCT'},
        {'Siemens', 'Biograph64'},
        {'Siemens', 'Somatom Definition'},
        {'Siemens', 'Somatom Definition Edge'},
        {'Siemens', 'Somatom Definition Flash'},
        {'Siemens', 'Somatom Force'},
        {'Toshiba', 'Aquilion'},
        {'Toshiba', 'Aquilion Prime'},
        {'Toshiba', 'Aquilion One'}
}
-------------------------------------------------------------------------------------
Guide to customising Orthanc configuration

python_executable Set this to the path and name of the python executable used by OpenREM:

# Linux, no virtualenv example:
local python_executable = '/usr/bin/python'
# Linux, using virtualenv example:
local python_executable = '/home/username/veopenrem/bin/python'
# Windows, not using virtualenv example:
local python_executable = 'C:\\Python27\\python.exe'
# Windows, using virtualenv example:
local python_executable = 'C:\\path\\to\\virtualenv\\Scripts\\python.exe'

python_scripts_path Set this to the path of the python scripts folder used by OpenREM:

# Linux, no virtualenv example:
local python_scripts_path = '/usr/local/bin/'
# Linux, using virtualenv example:
local python_scripts_path = '/home/username/veopenrem/bin/'
# Windows, not using virtualenv example:
local python_scripts_path = 'C:\\Python27\\Scripts\\'
# Windows, using virtualenv example:
local python_scripts_path = 'C:\\path\\to\\virtualenv\\Scripts\\'

temp_path Set this to the path where you want Orthanc to temporarily store DICOM files. Note: the folder must exist and Orthanc must be able to write to it. On Ubuntu Linux the user is orthanc:

# Linux example:
local temp_path = '/tmp/orthanc/'
# To create the directory:
mkdir /tmp/orthanc
sudo chown orthanc /tmp/orthanc/
# Windows example:
local temp_path = 'C:\\Temp\\orthanc\\'
  • Using Orthanc to collect Physics QA images:

    use_physics_filtering set this to false if you don’t want to use this facility. If this is false, the other physics image related values don’t matter. If it is true, the:

    physics_to_keep_folder Optional Set this to the path where you want to keep physics-related DICOM images:

    local physics_to_keep_folder = 'E:\\conquest\\dicom\\physics\\'
    

    physics_to_keep A list to check against patient name and ID to see if the images should be kept. Orthanc will put anything that matches this in the physics_to_keep_folder:

    local physics_to_keep = {'physics'}
    
  • Lists of things to ignore. Orthanc will ignore anything matching the content of these comma separated lists: they will not be imported into OpenREM:

    local manufacturers_to_ignore = {'Faxitron X-Ray LLC', 'Gendex-KaVo'}
    local model_names_to_ignore = {'CR 85', 'CR 75'}
    local station_names_to_ignore = {'CR85 Main', 'CR75 Main'}
    local software_versions_to_ignore = {'VixWin Platinum v3.3'}
    local device_serial_numbers_to_ignore = {'SCB1312016'}
    
  • Attempting to get dose data from CT studies with no RDSR using the OpenREMToshiba CT extractor

    use_toshiba_ct_extractor set this to false if you haven’t installed the additional Resources for creating RDSR for older Toshiba CT scanners or do not wish to use this function. Otherwise:

    toshiba_extractor_systems A list of CT make and model pairs that are known to have worked with the Toshiba CT extractor. These will only be considered if an RDSR is not found with the study, otherwise that will be used in preference. The format is {{'manufacturer', 'model'}, {'manufacturer two'}, {'model two'}} etc. They will be matched against the names presented in the DICOM headers:

    local toshiba_extractor_systems = {
            {'Toshiba', 'Aquilion'},
            {'GE Medical Systems', 'Discovery STE'},
    }
    
Configure Orthanc to make use of the openrem_orthanc_config.lua file

Edit orthanc.json which can be found in:

  • Ubuntu linux: /etc/orthanc/
  • Windows: C:\Program Files\Orthanc Server\Configuration\

Find and edit the section below:

Linux:

// List of paths to the custom Lua scripts that are to be loaded
// into this instance of Orthanc
"LuaScripts" : [
"/path/to/openrem_orthanc_config.lua"
],

Windows (note the double back-slash):

// List of paths to the custom Lua scripts that are to be loaded
// into this instance of Orthanc
"LuaScripts" : [
"C:\\path\\to\\openrem_orthanc_config.lua"
],
Check permissions

Linux

  • orthanc user needs to be able to write to the OpenREM logs
  • orthanc user needs to be able to write to the temp directory we specified

Windows

  • Orthanc will be running as a local admin user, so should be able to function without any special consideration
Restart Orthanc

Ubuntu linux:

sudo service orthanc force-reload

Windows:

  • Run Services.msc as an administrator
  • Right-hand click on the Orthanc entry and select Restart
Conquest Store configuration

The instructions for installing Conquest were included in the Install a DICOM Store service docs.

Now OpenREM is installed, you need to configure dicom.ini and setup the import scripts

Import scripts
Creating bash scripts on linux

Create a bash script for each of RDSR, mammo, DX and Philips CT dose images, as required. They should have content something like the following. The examples that follow assume the files have been saved in the folder /etc/conquest-dicom-server but you can save them where you like and change the dicom.ini commands accordingly.

These scripts have a line in them to activate the virtual environment; this is done in the line . /var/dose/venv/bin/activate – you should change the path to your virtualenv or remove it if you have installed without using a virtualenv.

Eash script also has a line to delete the object after it has been imported – OpenREM can also do this by configuration, but the file will be written by the _conquest user, and OpenREM will not be running as that user. Therefore it is easier to have conquest delete the file. If you don’t want them to be deleted, remove or comment out that line (add a # character to the start of the line).

  • Radiation Dose Structured Reports
  • Use which ever editor you are comfortable with – a good choice might be nano. For example:
sudo nano /etc/conquest-dicom-server/openrem-rdsr.sh
#!/bin/sh
#
# usage: ./openrem-rdsr.sh rdsrfilepath
#

# Get the name of the RDSR as variable 'rdsr'
rdsr="$1"

# Setup the python virtual environment - change to suit your path or remove if
# you are not using virtualenv
. /var/dose/venv/bin/activate

# Import RDSR into OpenREM
openrem_rdsr.py ${rdsr}

# Delete RDSR file - remove or comment (#) this line if you want the file to remain
rm ${rdsr}

Save and exit, then set the script to be executable:

sudo chmod +x /etc/conquest-dicom-server/openrem-rdsr.sh

And repeat for the other modality scripts below:

  • Mammography images
sudo nano /etc/conquest-dicom-server/openrem-mg.sh
#!/bin/sh
#
# usage: ./openrem-mg.sh mammofilepath
#

mamim="$1"

. /var/dose/venv/bin/activate

openrem_mg.py ${mamim}

rm ${mamim}
sudo chmod +x /etc/conquest-dicom-server/openrem-mg.sh
  • Radiography images (DX, and CR that might be DX)
sudo nano /etc/conquest-dicom-server/openrem-dx.sh
#!/bin/sh
#
# usage: ./openrem-dx.sh dxfilepath
#

dxim="$1"

. /var/dose/venv/bin/activate

openrem_dx.py ${dxim}

rm ${dxim}
sudo chmod +x /etc/conquest-dicom-server/openrem-dx.sh
  • Philips CT dose info images for Philips CT systems with no RDSR
sudo nano /etc/conquest-dicom-server/openrem-ctphilips.sh
#!/bin/sh
#
# usage: ./openrem-ctphilips.sh philipsctpath
#

philipsim="$1"

. /var/dose/venv/bin/activate

openrem_ctphilips.py ${philipsim}

rm ${philipsim}
sudo chmod +x /etc/conquest-dicom-server/openrem-ctphilips.sh
Creating batch scripts on windows

Create and save a bash script for each of RDSR, mammo, DX and Philips CT dose images, as required. They should have content something like the following.

These scripts assume there is no virtualenv in use. If you are using one, simply refer to the python executable and OpenREM script in your virtualenv rather than the system wide version.

  • Radiation Dose Structured Reports
C:\Python27\python.exe C:\Python27\scripts\openrem_rdsr.py %1
del %1
  • Mammography images
C:\Python27\python.exe C:\Python27\scripts\openrem_my.py %1
del %1
  • Radiography images (DX, and CR that might be DX)
C:\Python27\python.exe C:\Python27\scripts\openrem_dx.py %1
del %1
  • Philips CT dose info images for Philips CT systems with no RDSR
C:\Python27\python.exe C:\Python27\scripts\openrem_ctphilips.py %1
del %1
dicom.ini configuration
Example Windows Conquest dicom.ini file

Below is an example dicom.ini file, including comments describing the function of some sections. The file calls various lua scripts - see the Conquest import configuration document for an example - Advanced Lua Conquest configuration (Windows).

The example dicon.ini file:

# This file contains configuration information for the DICOM server
# Do not edit unless you know what you are doing

[sscscp]
MicroPACS                = sscscp

# Network configuration: server name (AE title) and TCP/IP port number. You may wish to add this OpenREM DICOM
# node to some of your imaging modalities, or to your PACS, so that you can send data to OpenREM from these systems.
# You'll need to know the AE title, port number and IP address of this server when doing this. Port 104 is commonly
# used for DICOM traffic. You may need to configure your server firewall to allow network traffic on this port.
MyACRNema                = OPENREM
TCPPort                  = 104

# Host, database, username and password for database. "localhost" means the server that Conquest is running on.
# The SQLServer is blank to prevent the incoming DICOM objects from being added to the Conquest database - this
# helps to avoid storing patient-identifiable data that you don't need to keep.
SQLHost                  = localhost
SQLServer                =
Username                 =
Password                 =
SqLite                   = 1
DoubleBackSlashToDB      = 0
UseEscapeStringConstants = 0

# Configure server
ImportExportDragAndDrop  = 1
ZipTime                  = 05:
UIDPrefix                = 1.2.826.0.1.3680043.2.135.736310.50024482
EnableComputedFields     = 1

# This option determins the folder structure used by Conquest when saving incoming DICOM objects on the server.
# Option 4 saves as ID\seriesuid_series#_image#_timecounter.dcm. At my Trust the ID is the patient NHS number.
FileNameSyntax           = 4

# Configuration of compression for incoming images and archival ("ul" saves images using little endian explicit
# encoding).
DroppedFileCompression   = ul
IncomingCompression      = ul
ArchiveCompression       = ul

# For debug information
PACSName                 = OPENREM
OperatorConsole          = 127.0.0.1
DebugLevel               = 0

# Configuration of disk(s) to store incoming DICOM objects.
MAGDeviceFullThreshHold  = 30
MAGDevices               = 1
MAGDevice0               = E:\conquest\dicom\


# Importing incoming data in to OpenREM
# The lua scripts that are called by some of these importers must be located in the same folder as this dicom.ini
# file.

# DICOM Radiation Dose Structured Report (RDSR) objects
ImportConverter0 = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.67"; {save to E:\conquest\dicom\sr\%o.dcm; openrem_import_rdsr.lua(E:\conquest\dicom\sr\%o.dcm::%V0018,1000); destroy;}

# Mammography objects (modality MG)
ImportConverter1 = ifequal "%m", "MG"; { save to E:\conquest\dicom\mammo\%o.dcm; openrem_import_mg.lua("E:\conquest\dicom\mammo\%o.dcm"); destroy; }

# Digital radiography (modality DX)
ImportConverter2 = ifequal "%m", "DX"; { save to E:\conquest\dicom\dx\%o.dcm; openrem_import_cr_or_dx.lua(E:\conquest\dicom\dx\%o.dcm::%V0008,0070::%V0008,1090::%V0008,1010::%V0018,1020::%V0008,0020::%V0010,0010::%V0010,0020); destroy; }

# Computed radiography (modality CR). Note: some digital radiography systems send their images as "CR" rather than "DX"
ImportConverter3 = ifequal "%m", "CR"; { save to E:\conquest\dicom\cr\%o.dcm; openrem_import_cr_or_dx.lua(E:\conquest\dicom\cr\%o.dcm::%V0008,0070::%V0008,1090::%V0008,1010::%V0018,1020::%V0008,0020::%V0010,0010::%V0010,0020); destroy; }

# Import converter for CT images. Conquest is configured to save images using the NHS number as the folder name, so
# I think it makes sense to process the images by patient, rather than by study ("process patient after"... rather
# than "process study after"...). If "process study" was used then the openrem_import_ct.lua script may be run
# multiple times on the same folder if the folder contains more than one study for that patient.
ImportConverter4 = ifequal "%V0008,1090","Brilliance 64"; { ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.7"; { save to E:\conquest\dicom\sr\%o.dcm; openrem_import_ctphilips.lua("E:\conquest\dicom\sr\%o.dcm"); }; destroy; }
ImportConverter5 = ifequal "%V0008,1090","Brilliance 16P"; { ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.7"; { save to E:\conquest\dicom\sr\%o.dcm; openrem_import_ctphilips.lua("E:\conquest\dicom\sr\%o.dcm"); }; destroy; }
ImportConverter6 = ifnotequal "%V0008,1090", "Brilliance 64"; { ifequal "%m", "CT"; { process patient after 0 by openrem_import_ct.lua %p::%V0008,0070::%V0008,1090::%V0018,1020::%V0008,1010::%V0010,0010::%V0010,0020::%V0008,0020::%V0018,1000; }; }

# Import converter for Presentation State objects - delete them
ImportConverter7 = ifequal "%m", "PR"; { destroy; }

# Import converter for Key Object Selection objects - delete them
ImportConverter8 = ifequal "%m", "KO"; { destroy; }

# Import converter for OT modality objects - delete them
ImportConverter9 = ifequal "%m", "OT"; { destroy; }

# Import converter for PT modality objects (PET) - delete them
ImportConverter10 = ifequal "%m", "PT"; { destroy; }

# Import converter for NM modality objects - delete them
ImportConverter11 = ifequal "%m", "NM"; { destroy; }

# Import converter for "Comprehensive SR Storage" type files - delete them
ImportConverter12 = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.33"; {destroy;}

# Import converter for "Basic Text SR Storage" type files - delete them
ImportConverter13 = ifequal "%V0008,0016","=BasicTextSRStorage"; {destroy;}

# Import converter for US modality objects - delete them
ImportConverter14 = ifequal "%m", "US"; { destroy; }

# Import converter for XA modality objects - delete them
ImportConverter15 = ifequal "%m", "XA"; { destroy; }

# Import converter for PX modality objects (panoramic x-ray) - delete them
ImportConverter16 = ifequal "%m", "PX"; { destroy; }

# Import converter for PX modality objects (panoramic x-ray) - delete them
ImportConverter17 = ifequal "%m", "PX"; { destroy; }

# Import converter for XRayAngiographicImageStorage images (graphical dose reports sent by the cath lab) - delete them
ImportConverter18 = ifequal "%V0008,0016","=XRayAngiographicImageStorage"; { destroy; }

# Import converter for XRayAngiographicImageStorage images (graphical dose reports sent by the cath lab) - delete them
ImportConverter19 = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.12.1"; { save to E:\conquest\dicom\cath_lab_protocols\%o.dcm; openrem_zip_angiostorage.lua(E:\conquest\dicom\cath_lab_protocols::E:\conquest\dicom\cath_lab_protocols\%o.dcm::%V0008,0020::%V0008,0030::%V0008,0050); destroy; }

# Enhanced SR Storage objects
ImportConverter20 = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.22"; {save to E:\conquest\dicom\sr\%o.dcm; openrem_import_rdsr.lua("E:\conquest\dicom\sr\%o.dcm"); destroy;}

# Import converter for PX modality objects (panoramic x-ray) - delete them as the systems we have contain no useful
# dose data information.
ImportConverter21 = ifequal "%m", "MR"; { destroy; }
Advanced Lua Conquest configuration (Windows)

Drop/merge the non lua section of this document

Configuring Conquest DICOM server to automatically forward data to OpenREM

Simple rules using the Conquest dicom.ini file

The Conquest DICOM server can be configured to automatically run tasks when it receives specific types of DICOM object. For example, a script can be run when a DX image is received that will extract dose information into OpenREM; Conquest will then delete the original image.

These actions are set up in the dicom.ini file, located in the root of the Conquest installation folder (an example dicom.ini file is available here: Example Windows Conquest dicom.ini file).

An example import converter:

ImportModality1   = MG
ImportConverter1  = save to C:\conquest\dosedata\mammo\%o.dcm; system C:\conquest\openrem-mam-launch.bat C:\conquest\dosedata\mammo\%o.dcm; destroy

ImportModality1 = MG tells Conquest that modality 1 is MG. The commands listed in the ImportConverter1 line are then run on all incoming MG images.

The ImportConverter instructions are separated by semicolons; the above example has three commands:

  • save to C:\conquest\dosedata\mammo\%o.dcm saves the incoming MG image to the specified folder with a file name set to the SOP instance UID contained in the image
  • system C:\conquest\openrem-mam-launch.bat C:\conquest\dosedata\mammo\%o.dcm runs a DOS batch file, using the newly saved file as the argument. On my system this batch file runs the OpenREM openrem_mg.py import script
  • destroy tells Conquest to delete the image that it has just received.

My system had three further import sections for DX, CR, and structured dose report DICOM objects:

# Import of DX images
ImportModality2   = DX
ImportConverter2  = save to C:\conquest\dosedata\dx\%o.dcm; system C:\conquest\openrem-dx-launch.bat C:\conquest\dosedata\dx\%o.dcm; destroy

# Import of CR images
ImportModality3   = CR
ImportConverter3  = save to C:\conquest\dosedata\dx\%o.dcm; system C:\conquest\openrem-dx-launch.bat C:\conquest\dosedata\dx\%o.dcm; destroy

# Import of structured dose reports (this checks the DICOM tag 0008,0016 to see if it matches the value for a dose report)
ImportConverter4  = ifequal "%V0008,0016","1.2.840.10008.5.1.4.1.1.88.67"; {save to C:\conquest\dosedata\sr\%o.dcm; system C:\conquest\openrem-sr-launch.bat "C:\conquest\dosedata\sr\%o.dcm"; destroy}

However, I have since moved to calling lua scripts from Conquest, as described in the next section.

Advanced Conquest DICOM object handling using lua scripts

Conquest can be configured to use lua scripts to handle incoming DICOM objects. This enables more flexibility than the examples provided in the section above. For example, you may wish to keep all incoming images that contain the word physics in the patient name or id fields. You may also wish to direct images from different makes and models of CT scanner to different OpenREM import scripts. I use this technique to forward studies from some Toshiba Aquilion CX and CXL scanners to an importer that creates a DICOM RDSR object from the Toshiba dose summary images and information stored in the image tags. These particular scanners are not capable of producing their own RDSR objects directly. I use the same script to delete images from CT scanners that I cannot extract data from.

A lua script designed to handle any objects with modality CT can be called from dicom.ini in the following way:

ImportConverter6 = ifequal "%m", "CT"; { process patient after 0 by openrem_import_ct.lua %p::%V0008,0070::%V0008,1090::%V0018,1020::%V0008,1010::%V0010,0010::%V0010,0020::%V0008,0020::%V0018,1000; }

The above line will run a script called openrem_import_ct.lua once the complete patient data has been received by Conquest and a delay of 10 minutes has elapsed (the default). The following information is passed on to the script:

  • %p, the path to a folder of DICOM objects
  • %V0008,0070, manufacturer
  • %V0008,1090, model
  • %V0018,1020, software version
  • %V0008,1010, station name
  • %V0010,0010, patient name
  • %V0010,0020, patient id
  • %V0008,0020, study date
  • %V0018,1000, device serial number

An example openrem_import_ct.lua script is shown below. It receives the parameters passed to it as single string. The individual parameters are recovered by splitting up the string using the :: substring as a delimiter. The script keeps any images that contain physics in the patient name or id fields; it looks for any Philips dose info objects and imports these with the appropriate routine; it deletes images from scanners that cannot be used; it runs the Toshiba RDSR creation importer on images from older Toshiba CT scanners:

require "openrem_string_split"

-- It is assumed that 'openrem_rdsr.py' and 'openrem_ctphilips.py' are in the Python scripts folder, and that this
-- and the Python executable location are included in the system path.

-- It is assumed that this code has been called from an ImportConverter within a Conquest dicom.ini file, and passed
-- the following:
--    path to a folder of DICOM objects
--    the manufacturer (0008,0070)
--    model (0008,1090)
--    software version (0018,1020)
--    station name (0008,1010)
--    patient name (0010,0010)
--    patient id (0010,0020)
--    study date (0008,0020)
--    device serial number (0018,1000)

local physics_to_keep = {'physics'}
local physics_folder = 'E:\\conquest\\dicom\\physics_images\\'

local split_input_text = split(command_line, '::')
local study_folder_path = split_input_text[1]
local manufacturer = split_input_text[2]
local model_name = split_input_text[3]
local software_version = split_input_text[4]
local station_name = split_input_text[5]

local patient_name, patient_id, study_date, device_serial_number

if split_input_text[6] == nil then
  patient_name = ''
else
  patient_name = string.lower(split_input_text[6])
end

if split_input_text[7] == nil then
  patient_id = ''
else
  patient_id = string.lower(split_input_text[7])
end

if split_input_text[8] == nil then
  study_date = 'blank'
else
  study_date = split_input_text[8]
end

if split_input_text[9] == nil then
  device_serial_number = 'blank'
else
  device_serial_number = split_input_text[9]
end

print(study_folder_path)

-- If any of the entries in physics_to_keep are present in the patient name or ID then the image is assumed to be a
-- physics test, and is kept.
for i = 1, #physics_to_keep do
  if string.match(patient_name, physics_to_keep[i]) or string.match(patient_id, physics_to_keep[i]) then
    print('Keeping the image: patient name is ' .. patient_name)
    print('and patient ID is ' .. patient_id)
    print('Trying to create folder ' .. physics_folder .. '\\' .. study_date)
    system('c:\\Windows\\system32\\cmd.exe /C mkdir ' .. physics_folder .. '\\' .. study_date)
    print('Trying to copy to the following folder: ' .. study_folder_path .. ' ' .. physics_folder .. '\\' .. study_date .. '\\')
    system('c:\\Windows\\system32\\cmd.exe /C copy ' .. study_folder_path .. '\\*.* ' .. physics_folder .. '\\' .. study_date .. '\\')
    return
  end
end

if (manufacturer == 'Philips' and model_name == 'Brilliance 64') then
  print('It is a Philips Brilliance 64')
  -- Look for a dose summary image and import it
  local files = assert(io.popen('dir /b ' .. study_folder_path))
  local output = files:read('*all')
  local file_list = split(output, '\n')

  for k, v in pairs(file_list) do
    current_file = study_folder_path .. '\\' .. v -- The fully qualified file name and path (Windows-specific)
    readdicom(current_file)
    if Data.SOPClassUID == '1.2.840.10008.5.1.4.1.1.7' then
      system('D:\\Server_Apps\\python27\\python.exe d:\\Server_Apps\\python27\\Scripts\\openrem_ctphilips.py ' .. current_file)
      print('The system command to import a Philips CT dose image has been executed on: ' .. current_file)
    end
  end

  -- Delete the study from disk
  print('Complete. Deleting study images.')
  system('C:\\Windows\\system32\\cmd.exe /C rmdir /S /Q ' .. study_folder_path)
  return
end

-- Check for images from a Toshiba CT simulator - images are of no use - need RDSR
if (manufacturer == 'TOSHIBA' and station_name == 'AQ16LB_SCAN') then
  print('It is a Toshiba Aquilion LB study. Cannot make use of these images - deleting them.')
  system('C:\\Windows\\system32\\cmd.exe /C rmdir /S /Q ' .. study_folder_path)
  print('The system command has been executed to delete the images from the server')
  return
end

-- Toshiba Aquilion CX and CXL scanners - try and create an RDSR from the data
if (manufacturer == 'TOSHIBA' and model_name == 'Aquilion') then
  print('It is a Toshiba Aquilion. Running openrem_rdsr_toshiba_ct_from_dose_images.py script: ' .. study_folder_path)
  system('d:\\Server_Apps\\python27\\python.exe d:\\Server_Apps\\python27\\Scripts\\openrem_rdsr_toshiba_ct_from_dose_images.py ' .. study_folder_path)
  print('The system command has been executed to create the rdsr and import it: ' .. study_folder_path)
  -- The openrem_rdsr_toshiba_ct_from_dose_images.py routine deletes the study from disk once the
  -- RDSR has been produced and imported in to OpenREM.
  return
end

-- Old Toshiba Asteion
if (manufacturer == 'TOSHIBA' and model_name == 'Asteion') then
  print('It is a Toshiba Asteion. Cannot make use of these images - deleting them: ' .. study_folder_path)
  system('C:\\Windows\\system32\\cmd.exe /C rmdir /S /Q ' .. study_folder_path)
  print('The system command has been executed to delete the images from the server')
  return
end

-- Old Picker PQS
if (manufacturer == 'Picker International, Inc.' and model_name == 'PQS') then
  print('It is a Picker PQS. Cannot make use of these images - deleting them: ' .. study_folder_path)
  system('C:\\Windows\\system32\\cmd.exe /C rmdir /S /Q ' .. study_folder_path)
  print('The system command has been executed to delete the images from the server')
  return
end

-- Image from a Vitrea workstation
if (manufacturer == 'Vital Images, Inc' and model_name == 'Vitrea 2') then
  print('It is a Vitrea 2. Cannot make use of these images - deleting them: ' .. study_folder_path)
  system('C:\\Windows\\system32\\cmd.exe /C rmdir /S /Q ' .. study_folder_path)
  print('The system command has been executed to delete the images from the server')
  return
end

The above script depends on openrem_string_split:

function split(str, pat)
   local t = {}  -- NOTE: use {n = 0} in Lua-5.0
   local fpat = "(.-)" .. pat
   local last_end = 1
   local s, e, cap = str:find(fpat, 1)
   while s do
      if s ~= 1 or cap ~= "" then
     table.insert(t,cap)
      end
      last_end = e+1
      s, e, cap = str:find(fpat, last_end)
   end
   if last_end <= #str then
      cap = str:sub(last_end)
      table.insert(t, cap)
   end
   return t
end
Preventing Conquest from adding incoming DICOM objects to the Conquest database

You may wish to prevent Conquest from adding patient data from incoming DICOM objects to the Conquest database, such as patient names and IDs. To do this set the SQLServer to a blank in the Conquest dicom.ini file:

# Host, database, username and password for database
SQLHost = localhost
# The SQLServer is blank below to prevent the incoming objects from being added to the Conquest database.
SQLServer =
Setting the compression for Conquest incoming DICOM images and archives

Setting the following options to ul within dicom.ini will make Conquest store DICOM objects using little endian explicit encoding:

# Configuration of compression for incoming images and archival
DroppedFileCompression   = ul
IncomingCompression      = ul
ArchiveCompression       = ul

For my system the ul above matches the compression that is set for Conquest’s known DICOM providers in the file acrnema.map, such as the Trust PACS and imaging modalities that have been set up to send data directly to Conquest.

Running Conquest on Windows as a service

Note

Content contributed by DJ Platten, with edit by ET McDonagh

This guide assumes Conquest has already been installed and runs ok. These instructions are based on Windows XP.

Run as a service
  1. Make sure conquest isn’t running.
  2. Open a file browser and navigate to your conquest folder.
  3. Right-hand click on the “ConquestDICOMServer.exe” file and choose “Run as…”
  4. Enter the username and password of a Windows user with administrator rights.
  5. Once conquest is running, click on the “Install server as NT service” on the “Configuration” tab.
  6. Close the conquest Window.
  7. Log in to Windows as a user with administrator rights.
  8. Go to “Control panel” -> “Administrative Tools” -> “Services”.
  9. There will be a service with the same name as conquest’s AE title. Right-hand click the mouse on this service and select “Properties”.
  10. On the “Log On” tab check the box that says “Allow service to interact with the desktop”.
  11. Click “Apply” then “OK”.
  12. Right-hand click on the service again and click “Restart”.

The “Allow service to interact with the desktop” seems to be necessary for the batch to run that puts the dose report into OpenREM.

Firewall settings

Windows is able to change its firewall settings after you think everything is working ok! Assuming you have control of the firewall, add three port exceptions to the Windows firewall on the server computer: ports 80 and 443 for Apache, and whichever port that was chosen for conquest (104 is the port allocated to DICOM, but you may have used a higher port above 1024 for permissions reasons).

The firewall instructions at portforward.com were found to be a useful guide for this.

You ony need one of these - if you already have one installed it is probably easiest to stick to it.

Query-retrieve from a PACS or similar

Before you can query-retrieve objects from a remote PACS, you need to do the following:

  • Create a DICOM Store service to receive the DICOM objects - see Direct from modalities above.

  • Configure OpenREM with the settings for the remote query-retrieve server:

    Configuration required for query-retrieve

    You need a DICOM store service set up - see Importing data to OpenREM for details.

    If you are using a third party DICOM Store server, then you will need to add the details as per DICOM Network Configuration but do not use the ‘advaanced’ section.

    To configure a remote query retrieve SCP, on the Config menu select DICOM networking, then click Add new QR Node and fill in the details:

    • Name of QR node: This is the friendly name, such as PACS QR
    • AE Title of the remote node: This is the DICOM name of the remote node, 16 or fewer letters and numbers, no spaces
    • AE Title this server: This is the DICOM name that the query (DICOM C-Find) will come from. This may be important if the remote node filters access based on calling aet. Normal rules of 16 or fewer letters and numbers, no spaces
    • Remote port: Enter the port the remote node is using (eg 104)
    • Remote IP address: The IP address of the remote node, for example 192.168.1.100
    • Remote hostname: Alternatively, if your network has a DNS server that can resolve the hostnames, you can enter the hostname instead. If the hostname is entered, it will be used in preference to the IP address, so only enter it if you know it will be resolved.
  • Configure the settings of your DICOM store service on the PACS

  • Learn how to use it:

    DICOM Query Retrieve Service

    To query retrieve dose related objects from a remote server, you need to review the Conquest Store configuration documents first to make sure you have created a DICOM Store node which will import objects to OpenREM.

    You will also need to set up the remote server to allow you to query-retrieve using it - the remote server will need to be configured with details of the store node that you have configured.

    Query-retrieve using the web interface
    Import Query-Retrieve menu

    Figure 1: Import Query-Retrieve menu

    • On the Imports menu, select Query remote server - see figure 1. If the menu isn’t there, you need to check your user permissions - see Configure the settings for details.
    • Each configured query-retrieve node and each local store node is automatically tested to make sure they respond to a DICOM echo - the results are presented at the top of the page. See fingure 2 for an example.
    local and remote QR statuses

    Figure 2: Local and remote QR statuses

    • Select the desired remote host, ie the PACS or modality you wish to query.
    • Select the local store node you want to retrieve to.
    • Select which modalities you want to query for - at least one must be ticked.
    • Select a date range - the wider this is, the more stress the query will place on the remote server, and the higher the likelyhood of the query being returned with zero results (a common configuration on the remote host to prevent large database queries affecting other services). Defaults to ‘from yesterday’.
    • If you wish to exclude studies based on their study description, enter the text here. Add several terms by separating them with a comma. One example would be to exclude any studies with imported in the study description, if your institution modifies this field on import. The matching is case-insensitive.
    • Alternatively, you might want to only keep studies with particular terms in the study description. If so, enter them in the next box, comma separated.
    • You can also exclude studies by station name, or only keep them if they match the station name. This is only effective if the remote system (the PACS) supports sending back station name information.
    Advanced query options
    • Attempt to get Toshiba dose images default not ticked: If you have done the extra installation and configuration required for creating RDSRs from older Toshiba scanners, then you can tick this box for CT searches to get the images needed for this process. See the logic description below for details.
    • Ignore studies already in the database default ticked: By default OpenREM will attempt to avoid downloading any DICOM objects (RDSRs or images) that have already been imported into the database. Untick this box to override that behaviour and download all suitable objects. See the logic description below for details.
    • Include SR only studies default not ticked: If you have a DICOM store with only the radiation dose structured reports (RDSR) in, or a mix of whole studies and RDSRs without the corresponding study, then tick this box. Any studies with images and RDSRS will be ignored (they can be found without this option). If this box is ticked any modality choices will be ignored.

    When you have finished the query parameters, click Submit

    Review and retrieve

    The progress of the query is reported on the right hand side. If nothing happens, ask the administrator to check if the celery queue is running.

    Once all the responses have been purged of unwanted modalities, study descriptions or study UIDs, the number of studies of each type will be displayed and a button appears. Click Retrieve to request the remote server send the selected objects to your selected Store node. This will be based on your original selection - changing the node on the left hand side at this stage will have no effect.

    The progress of the retrieve is displayed in the same place until the retrieve is complete.

    Query-retrieve using the command line interface

    In a command window/shell, python openrem_qr.py -h should present you with the following output:

    usage: openrem_qr.py [-h] [-ct] [-mg] [-fl] [-dx] [-f yyyy-mm-dd]
                         [-t yyyy-mm-dd] [-e string] [-i string] [-sne string]
                         [-sni string] [-toshiba] [-sr] [-dup]
                         qr_id store_id
    
    Query remote server and retrieve to OpenREM
    
    positional arguments:
      qr_id                 Database ID of the remote QR node
      store_id              Database ID of the local store node
    
    optional arguments:
      -h, --help            show this help message and exit
      -ct                   Query for CT studies. Do not use with -sr
      -mg                   Query for mammography studies. Do not use with -sr
      -fl                   Query for fluoroscopy studies. Do not use with -sr
      -dx                   Query for planar X-ray studies. Do not use with -sr
      -f yyyy-mm-dd, --dfrom yyyy-mm-dd
                            Date from, format yyyy-mm-dd
      -t yyyy-mm-dd, --duntil yyyy-mm-dd
                            Date until, format yyyy-mm-dd
      -e string, --desc_exclude string
                            Terms to exclude in study description, comma
                            separated, quote whole string
      -i string, --desc_include string
                            Terms that must be included in study description,
                            comma separated, quote whole string
      -sne string, --stationname_exclude string
                            Terms to exclude in station name, comma separated,
                            quote whole string
      -sni string, --stationname_include string
                            Terms to include in station name, comma separated,
                            quote whole string
      -toshiba              Advanced: Attempt to retrieve CT dose summary objects
                            and one image from each series
      -sr                   Advanced: Query for structured report only studies.
                            Cannot be used with -ct, -mg, -fl, -dx
      -dup                  Advanced: Retrieve duplicates (studies that are
                            already in database)
    

    As an example, if you wanted to query the PACS for DX images on the 5th April 2010 with any study descriptions including imported excluded, first you need to know the database IDs of the remote node and the local node you want the images sent to. To find these, go to the DICOM Network Configuration page where the database ID is listed among the other details for each node.

    Assuming the PACS database ID is 2, and the store node ID is 1, the command would look something like:

    python openrem_qr.py 2 1 -dx -f 2010-04-05 -t 2010-04-05 -e "imported"
    

    If you want to do this regularly to catch new studies, you might like to use a script something like this on linux:

    #!/bin/bash
    
    . /var/openrem/bin/activate  # activate virtualenv if you are using one, modify or delete this line
    
    ONEHOURAGO=$(date -d "1 hour ago" "+%Y-%m-%d")
    
    python openrem_qr.py 2 1 -dx -f $ONEHOURAGO -t $ONEHOURAGO  -e "Imported"
    

    This script could be run once an hour using a cron job. By asking for the date an hour ago, you shouldn’t miss exams taking place in the last hour of the day.

    A similar script could be created as a batch file or PowerShell script on Windows and run using the scheduler. An example PowerShell script is shown below:

    # Script to obtain all CT studies from a DICOM node on the day prior to the
    # date the script is run and import them into OpenREM.
    # Get yesterday's date
    $dateString = "{0:yyyy-MM-dd}" -f (get-date).AddDays(-1)
    # Run the openrem_qr.py script with yesterday's date as the to and from date
    python D:\Server_Apps\python27\Scripts\openrem_qr.py 2 1 -ct -f $dateString -t $dateString
    

    The above PowerShell script could be run on a regular basis by adding a task to the Windows Task Scheduler that executes the powershell program with an argument of -file C:\\path\\to\\script.ps1.

    Query filtering logic
    Study level query response processing
    • First we query for each modality chosen in turn to get matching responses at study level.
    • If the optional ModalitiesInStudy has been populated in the response, and if you have ticked Include SR only studies, then any studies with anything other than just SR studies is removed from the response list.
    • If any study description or station name filters have been added, and if the StudyDescription and/or StationName tags are returned by the remote server, the study response list is filtered accordingly.
    • For the remaining study level responses, each series is queried.
    • If ModalitiesInStudy was not returned, it is now built from the series level responses.
    • If the remote server returned everything rather than just the modalities we asked for, the study level responses are now filtered against the modalities selected.
    Series level query processing
    • Another attempt is made to exclude or only-include if station name filters have been set

    If mammography exams were requested, and a study has MG in:

    • If one of the series is of type SR, an image level query is done to see if it is an RDSR. If it is, all the other series responses are deleted (i.e. when the move request/’retrieve’ is sent only the RDSR is requested not the images.
    • Otherwise the SR series responses are deleted and all the image series are requested.

    If planar radiographic exams were requested, and a study has DX or CR in:

    • Any SR series are checked at ‘image’ level to see if they are RDSRs. If they are, the other series level responses for that study are deleted.
    • Otherwise the SR series responses are deleted and all the image series are requested.

    If fluoroscopy exams were requested, and a study has RF or XA in:

    • Any SR series are checked at ‘image’ level to see if they are RDSRs or ESRs (Enhanced Structured Reports - not currently used but will be in the future). Any other SR series responses are deleted.
    • All non-SR series responses are deleted.

    If CT exams were requested, and a study has CT in:

    • Any SR series are checked at ‘image’ level to see if they are RDSRs. If they are, all other SR and image series responses are deleted. Otherwise, if it has an ESR series, again all other SR and image series responses are deleted.
    • If there are no RDSR or ESR series, the other series are checked to see if they are Philips ‘Dose info’ series. If there are, other series responses are deleted.
    • If there are no RDSR, ESR or ‘Dose info’ series and the option to get Toshiba images has been selected, then an image level query is performed for the first image in each series. If the image is not a secondary capture, all but the first image are deleted from the image level responses and the image_level_move flag is set. If the image is a secondary capture, the whole series response is kept.
    • If there are no RDSR or ESR, series descriptions aren’t returned and the Toshiba option has been set, the image level query is performed as per the previous point. This process will keep the responses that might have Philips ‘Dose info’ series.
    • If there are no RDSR, ESR, series descriptions aren’t returned and the Toshiba option has not been set, each series with more than five images in is deleted from the series response list - the remaining ones might be Philips ‘Dose info’ series.

    If SR only studies were requested:

    • Each series response is checked at ‘image’ level to see which type of SR it is. If is not RDSR or ESR, the study response is deleted.
    Duplicates processing

    For each remaining study in the query response, the Study Instance UID is checked against the studies already in the OpenREM database.

    If there is a match and the series level modality is SR (from a CT, or RF etc):

    • The image level response will have the SOP Instance UID - this is checked against the SOP Instance UIDs recorded with the matching study. If a match is found, the ‘image’ level response is deleted.

    If there is a match and the series level modality is MG, DX or CR:

    • An image level query is made which will populate the image level responses with SOP Instance UIDs
    • Each image level response is then processed and the SOP Instance UID is checked against the SOP Instance UIDs recorded with the matching study. If a match is found, the ‘image’ level response is deleted.

    Once each series level response is processed:

    • If the series no longer has any image level responses the series level response is deleted.
    • If the study no longer has any series level responses the study level response is deleted.
    Troubleshooting: openrem_qr.log

    If the default logging settings haven’t been changed then there will be a log files to refer to. The default location is within your MEDIAROOT folder:

    This file contains information about the query, the status of the remote node, the C-Find response, the analysis of the response, and the individual C-Move requests.

    The following is an example of the start of the log for the following query which is run once an hour (ie some responses will already have been imported):

    openrem_qr.py 2 1 -dx -f 2016-05-04 -t 2016-05-04 -e "imported"
    
    [04/May/2016 11:30:02] INFO [remapp.netdicom.qrscu:580] qrscu script called
    [04/May/2016 11:30:02] INFO [remapp.netdicom.qrscu:595] Modalities are ['DX']
    [04/May/2016 11:30:02] INFO [remapp.netdicom.qrscu:601] Date from: 2016-05-04
    [04/May/2016 11:30:02] INFO [remapp.netdicom.qrscu:604] Date until: 2016-05-04
    [04/May/2016 11:30:02] INFO [remapp.netdicom.qrscu:610] Study description exclude terms are ['imported']
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:267] Request association with Hospital PACS PACSAET01 (PACSEAT01 104 DICOM_QR_SCP)
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:277] assoc is ... <Association(Thread-7208, started daemon 140538998306560)>
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:280] DICOM Echo ...
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:282] done with status Success
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:284] DICOM FindSCU ...
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:311] Currently querying for DX studies...
    [04/May/2016 11:30:03] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:04] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:04] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:04] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:05] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:05] INFO [remapp.netdicom.qrscu:311] Currently querying for CR studies...
    [04/May/2016 11:30:05] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:05] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:06] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:06] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:06] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:07] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:10] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:10] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:11] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:11] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:12] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:12] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:12] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:12] INFO [remapp.netdicom.qrscu:339] Checking to see if any of the 16 studies are already in the OpenREM database
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:343] Now have 11 studies
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:349] Deleting studies we didn't ask for
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is DX, mod_set is ["CR"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is CR, mod_set is ["CR"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is DX, mod_set is ["PR", "DX"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is CR, mod_set is ["PR", "DX"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is DX, mod_set is ["DX"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is CR, mod_set is ["DX"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is DX, mod_set is ["PR", "CR"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:358] mod is CR, mod_set is ["PR", "CR"]
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:367] Now have 11 studies
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:372] Deleting series we can't use
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:408] Now have 11 studies
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:413] Deleting any studies that match the exclude criteria
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:422] Now have 6 studies after deleting any containing any of [u'imported']
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:438] Release association
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:499] Preparing to start move request
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:504] Requesting move of 6 studies
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:509] Mv: study_no 1
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:515] Mv: study no 1 series no 1
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:528] Requesting move: modality DX, study 1 (of 6) series 1 (of 1). Series contains 1 objects
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:13] INFO [remapp.netdicom.qrscu:44] Move association requested
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:53] Move association released
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:532] _move_req launched
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:509] Mv: study_no 2
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:515] Mv: study no 2 series no 1
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:528] Requesting move: modality DX, study 2 (of 6) series 1 (of 1). Series contains 2 objects
    [04/May/2016 11:30:18] INFO [remapp.netdicom.qrscu:33] Association response received
    [04/May/2016 11:30:19] INFO [remapp.netdicom.qrscu:44] Move association requested
    [04/May/2016 11:30:29] INFO [remapp.netdicom.qrscu:48] gg is Pending
    [04/May/2016 11:30:30] INFO [remapp.netdicom.qrscu:53] Move association released
    ...etc
    

    If you are using an OpenREM native storage node, then you might also like to review Troubleshooting: openrem_store.log

Charts

Chart types

1. Bar chart of average values across a number of categories

Bar chart of mean DLP per acquisition

Figure 1: Bar chart of mean DLP per acquisition

An example of mean DAP per acquisition type is shown in figure 1.

Bar chart sorting options

Figure 2: Bar chart sorting options

Below each bar chart there are options to sort the order of the data. This can be ascending or descending by average value, size of data sample, or alphabetically (figure 2).

Clicking on an entry in the bar chart legend toggles the display of the corresponding series on the chart.

Histogram of acquisition DLP

Figure 3: Histogram of abdomen DLP values

If you have histogram calculation switched on then clicking on an individual data point on a bar chart will take you to a histogram of the data for that point so that you can see the shape of the value’s distribution (figure 3).

Bar charts can be plotted with a series per x-ray system (figure 4). This can be toggled using the Plot a series per system checkbox in the Chart options.

Clicking the left-hand mouse button on the chart background and dragging left or right selects part of the series. Releasing the mouse button zooms in on this selection. A Reset zoom button appears when zoomed in: clicking this resets the chart so that the full series can be seen again. The zoom feature works on both the main series and the histograms. The zooming can be useful when there is a category on the chart that has a very low value compared to others. Zooming in on this category will enable the low values to be seen, as the chart rescales the y-axis after the zoom.

Bar chart of mean DLP (one system per series)

Figure 4: Bar chart of mean DLP (one system per series)

Clicking on the Toggle data table button toggles the display of an HTML table containing the data from the current chart. This button is hidden if you are viewing the chart in full screen mode. An example showing a data table is shown in figure 5.

Bar chart with data table displayed

Figure 5: Bar chart with data table displayed

If the the bar chart that you are viewing shows more than one series then clicking on a category name on the x-axis will take you to a plot that shows multiple histograms: one for each series (figure 6).

If the bar chart that you are viewing shows more than one series then buttons are available to Hide all series, Show all series, and Toggle all series. These provide a quick way to switch which series are being displayed without having to click on individual series in the chart legend.

Histogram of abdomen DLP (one series per system)

Figure 6: Histogram of abdomen DLP values, one series per system

The histogram data can be plotted as absolute values, or be normalised to a value of 1.0 (figure 7). This can be toggled by clicking on the button that is shown below the histogram plots. The normalisation can be useful when trying to compare the shape of several histograms, especially when some histograms have much less data than others.

Normalised histogram of abdomen DLP (one series per system)

Figure 7: Normalised histogram of abdomen DLP, one series per system

Each histogram data point includes a text link that appears when the mouse pointer moves over it. Clicking on this link will filter the displayed studies, showing those that correspond to what is contained in the histogram bin.

Clicking on a legend entry toggles the visibility of the corresponding series.

Only data with non-zero and non-blank dose values are included in the chart data calculations.

2. Pie chart showing the frequency of each item in a category

Pie chart of acquisition frequency

Figure 8: Pie chart of acquisition frequency

Figure 8 shows a pie chart of the number of acquisitions made for every acquisition protocol present in the tabulated data.

Clicking on any of the pie chart segments will filter the displayed studies, showing only the studies that correspond to what is contained in that segment. As for the bar charts, this doesn’t work perfectly, as the category filtering isn’t exact.

Only data with non-zero and non-blank dose values are included in the chart data calculations.

3. Line chart showing how an average value changes over time

Line chart of mean DLP per study type over time

Figure 9: Line chart of mean DLP per study type over time

A line is plotted for each category, with a point calculated every day, week, month or year. This can be a good way of looking at how things have changed over time. For example, the mean DLP of each study type, calculated with a data point per month is shown in figure 9.

Clicking the left-hand mouse button on the chart and dragging left or right across a range of dates and then releasing the mouse button will zoom in on that selection.

Clicking on a legend entry toggles the visibility of the corresponding series.

Only data with non-zero and non-blank dose values are included in the chart data calculations.

4. Pie chart showing the number of events per day of the week

Pie chart of study workload per day of the week

Figure 10: Pie chart of study workload per day of the week

Pie chart of study workload per hour in a day

Figure 11: Pie chart of study workload per hour in a day

Each segment represents a day of the week, and shows the number of events that have taken place on that day (figure 10). Clicking on one of the segments will take you to a pie chart that shows the number of events per on that day (figure 11).

All data, including zero blank dose values are included in the data calculations for this chart type.

5. Scatter plot showing one value vs. another

Scatter plot of average glandular dose vs. compressed thickness

Figure 12: Scatter plot of average glandular dose vs. compressed thickness

Scatter plot of average glandular dose vs. compressed thickness; one series per system

Figure 13: Scatter plot of average glandular dose vs. compressed thickness; one series per system

This plot type shows a data point per event (figure 12). The series name and data values are shown when the mouse cursor is positioned over a data point.

These can be plotted with a series per x-ray system (figure 13). This can be toggled using the Plot a series per system checkbox in the Chart options.

Clicking the left-hand mouse button on the chart and dragging a rectangular region will zoom in on that selection of the chart. A Reset zoom button appears when zoomed in: clicking this resets the chart so that the full series can be seen again.

Clicking on a system’s legend entry toggles the display of the corresponding series on the chart.

Only data with non-zero and non-blank dose values are included in the chart data calculations.

Exporting chart data

An image file of a chart can be saved using the menu in the top-right hand side of any of the charts. The same menu can be used to save the data used to plot a chart: the data can be downloaded in either csv or xls format.

Chart options

OpenREM chart options

Figure 13: OpenREM chart options

Chart options can be configured by choosing the Chart options item from the User options menu on the OpenREM homepage (figure 13).

CT and radiographic plot options can also be set from their respective summary pages.

The first option, Plot charts?, determines whether any plots are shown. This also controls whether the data for the plots is calculated by OpenREM.

Switching Case-insensitive categories on will force chart categories to be lowercase. This can be helpful if several rooms use the same wording but with different capitalisation for acquisition protocol, study description or requested procedure. With this option switched on then all rooms with the same wording, irrespective of capitalisation, will be shown side-by-side under the same single category. With the option switched off there will be a seperate category for each differently capitalised category.

Some plot data is slow to calculate when there is a large amount of data: some users may prefer to leave Plot charts? off for performance reasons. Plot charts? can be switched on and activated with a click of the Submit button after the data has been filtered.

Switching charts off

Figure 14: Switching charts off

The user can also switch off chart plotting by clicking on the Switch charts off link in the User options menu in the navigation bar at the top of any OpenREM page, as shown in figure 14.

The user can choose whether the data displayed on the charts is the mean, median or both by using the drop-down Average to use selection. Only the bar charts can display both mean and median together. Other charts display just median data when this option is selected.

The charts can be sorted by either bar height, frequency or alphabetically by category. The default sorting direction can be set to ascending or descending using the drop-down list near the top of the chart options.

A user’s chart options can also be configured by an administrator via OpenREM’s user administration page.

Chart types - CT

  • Bar chart of average DLP for each acquisition protocol (all systems combined)
  • Bar chart of average DLP for each acquisition protocol (one series per system)
  • Pie chart of the frequency of each acquisition protocol
  • Pie chart showing the number of studies carried on each day of the week
  • Line chart showing the average DLP of each study name over time
  • Bar chart of average CTDIvol for each acquisition protocol
  • Bar chart of average DLP for each study name
  • Pie chart of the frequency of each study name
  • Bar chart of average DLP for each requested procedure
  • Pie chart of the frequency of each requested procedure

Chart types - radiography

  • Bar chart of average DAP for each acquisition protocol
  • Pie chart of the frequency of each acquisition protocol
  • Bar chart of average DAP for each study description
  • Pie chart of the frequency of each study description
  • Bar chart of average number of irradiation events for each study description
  • Bar chart of average DAP for each requested procedure
  • Pie chart of the frequency of each requested procedure
  • Bar chart of average number of irradiation events for each requested procedure
  • Bar chart of average kVp for each acquisition protocol
  • Bar chart of average mAs for each acquisition protocol
  • Pie chart showing the number of studies carried out per weekday
  • Line chart of average DAP of each acquisition protocol over time
  • Line chart of average mAs of each acquisition protocol over time
  • Line chart of average kVp of each acquisition protocol over time

Chart types - fluoroscopy

  • Bar chart of average DAP for each study description
  • Pie chart of the frequency of each study description
  • Pie chart showing the number of studies carried out per weekday

Chart types - mammography

  • Scatter plot of average glandular dose vs. compressed thickness for each acquisition
  • Scatter plot of kVp vs. compressed thickness for each acquisition
  • Scatter plot of mAs vs. compressed thickness for each acquisition
  • Pie chart showing the number of studies carried out per weekday

Performance notes

All chart types

For any study- or request-based charts, filtering using Acquisition protocol forces OpenREM to use a much slower method of querying the database for chart data. Where possible avoid filtering using this field, especially when viewing a large amount of data.

Bar charts

Switching off histogram calculation in Chart options will speed up bar chart data calculation.

Switching off Plot a series per system in the Chart options will speed up data calculation.

Scatter plots

Switching off Plot a series per system in the Chart options will speed up data calculation.

Skin dose maps

Functionality that is available

  • Skin dose map data is calculated to the surface of a simple geometric phantom using the in-built openSkin routines (3D phantom)
  • The calculated doses include kVp-dependent backscatter factors and account for any copper filters using data from this paper for each irradiation event; aluminium or other filters are not considered. Where more than one kVp is stored for an irradiation event the mean kVp is calculated, excluding any zero values.
  • The phantom dimensions are calculated from the height and mass of the patient; defaults of 1.786 m and 73.2 kg are used when patient height and mass are not available
  • Data can be calculated on import to OpenREM, or on demand when a study is viewed. Calculating the skin dose map can take several minutes, so calculating on import may tie up your server for that time, depending on how you have configured the Celery task queue
  • Data is recalculated automatically if the patient height or mass stored in the database differs from the values stored in the skin dose map data file. This is useful when patient size information has been imported in to OpenREM after the initial skin dose map data has been calculated
  • 3D skin dose map data is shown graphically as a 2D image and a 3D model
  • The 3D model can be manipulated using a touch screen
  • The user can change the maximum and minimum displayed dose; alternatively, window level and width can be adjusted
  • A colour dose scale is shown with a selection of colour schemes
  • The skin dose map section can be displayed full-screen
  • The calculated peak skin dose, phantom dimensions, patient height, mass and orientation used for the calculations are shown in the top left hand corner of the skin dose map
  • If skin dose map display is disabled then fluoroscopy study data can be exported in a format suitable for the stand-alone openSkin routines

The phantom consists of a cuboid with one semi-cylinder on each side (see 3D phantom section of phantom design on the openSkin website for details).

2D visualisation of the 3D data

2D visualisation of the 3D skin dose map data

Figure 1: 2D visualisation of the 3D data

This is a 2D view of the whole surface of the 3D phantom, as though the phantom surface has been peeled off and laid out flat (figure 1). The 2D visualisation includes the following features:

  • The skin dose at the mouse pointer is shown as a tool-tip
  • Moving the mouse whilst holding down the left-hand mouse button changes the window level and width of the displayed skin dose map
  • An overlay indicating the phantom regions and orientation can be toggled on and off. This indicates the phantom anterior, left, posterior and right sides, and also shows the superior and inferior ends (figure 2)
2D visualisation of the 3D skin dose map including phantom region overlay

Figure 2: Phantom region overlay

  • The current view can be saved as a png file

3D visualisation

This is a 3D view of the phantom that was used for the calculations, with the skin dose map overlaid onto the surface. The 3D visualisation includes the following features:

  • Moving the mouse whilst holding down the left-hand mouse button rotates the 3D model
  • Using the mouse wheel zooms in and out
3D visualisation of the skin dose map data

Figure 3: 3D visualisation of the data

  • A simple 3D model of a person is displayed in the bottom left corner. This is to enable the viewer to orientate themselves when viewing the 3D skin dose map
  • The current view can be saved as a png file

Skin dose map settings

There are two skin dose map options that can be set by an OpenREM administrator via the Skin dose map settings option in the Config menu:

  • Enable skin dose maps
  • Calculate skin dose maps on import

The first of these sets whether skin dose map data is calculated, and also switches the display of skin dose maps on or off. The second option controls whether the skin dose map data is calculated at the point when a new study is imported into OpenREM, or calculated when a user first views the details of that particular study in the OpenREM interface.

When skin dose maps are enabled:

  • When a user views the details of a fluoroscopy study OpenREM looks for a skin dose map pickle file on the OpenREM server in the skin_maps subfolder of MEDIA_ROOT that corresponds to the study being viewed. If found, the skin dose map data in the pickle file is loaded and displayed. The skin_maps folder is created if it does not exist
  • If a pickle file is not found then OpenREM calculates skin dose map data. These calculations can take some time. They are carried out in the background: an animated graphic is shown during the calculations. On successful calculation of the data the skin dose map is displayed. A pickle file containing the data is saved in the server’s skin_maps subfolder of MEDIA_ROOT. The file name is of the form skin_map_XXXX.p, where XXXX is the database primary key of the study
  • For subsequent views of the same study the data in the pickle file is loaded, rather than re-calculating the data, making the display of the skin dose map much quicker

When calculation on import is enabled:

  • OpenREM calculates the skin dose map data for a fluoroscopy study as soon as it arrives in the system
  • A pickle file containing the data is saved in the skin_maps subfolder of MEDIA_ROOT
  • Users viewing the details of a study won’t have to wait for the skin dose map data to be calculated

Exporting data to openSkin

If skin dose maps are disabled, and the user has export rights, the user is presented with the option of exporting the study data as a csv file that is formatted for use with a stand-alone installation of openSkin. The user must be in the detail view of the study they wish to create the exposure incidence map for, and then click on the link to create the OpenSkin export (figure 4).

Export from OpenREM to openSkin

Figure 4: Export from OpenREM to openSkin

Instructions for openSkin

Download the openSkin repository as a zip file from openSkin downloads. To use openSkin as a stand-alone application you need python 2.x and the pypng python library.

  • Extract the contents of the zip file into a folder on your computer and run python main.py from a command line and answer each question.
  • See phantom design for details of the 2D and 3D phantoms.
  • When asked for the source csv file use the one exported from OpenREM
  • Depending on the number of events in the export and the power of your computer the calculations can take a few minutes

Two files will be produced - a textfile called skin_dose_results.txt and a small image called skin_dose_map.png

Results text file

It should look something like this:

File created    : 04/21/15 17:42:45
Data file       : C:/Users/[...]/exports-2015-04-21-OpenSkinExport20150421-162805246134.csv
Phantom         : 90.0x70.0 3d phantom
Peak dose (Gy)  :                  0.50844405521
Cells > 3 Gy    :                              0
Cells > 5 Gy    :                              0
Cells > 10 Gy   :                              0

The peak dose is the peak incident dose delivered to any one-cm-square area. If any of these 1 cm2 areas (referred to as cells) are above 3 Gy, then the number of cells in this category, or the two higher dose categories, are listed in the table accordingly.

Incidence map image file

The image file will be a small 70x90 px PNG image if you used the 3D phantom, or 150 x 50 px PNG if you used the 2D phantom. With both, the head end of the table is on the left.

The image is scaled so that black is 0 Gy and white is 10 Gy. For most studies, this results in an incidence map that is largely black. However, if you use GIMP or ImageJ or similar to increase the contrast, you will find that the required map is there.

A native and ‘colour equalised’ version of the same export are shown below:

OpenSkin incidence map, unscaled OpenSkin incidence map, scaled

Limitations

Skin dose map calculations do not currently work for all systems. Siemens Artis Zee data is known to work. If skin dose maps do not work for your systems then please let us know via the OpenREM Google Group.

openSkin is yet to be validated independently - if this is something you want to do, please do go ahead and feed back your findings to Jonathan Cole at jacole.

Exporting study information

Exporting to csv and xlsx sheets

If you are logged in as a user in the exportgroup or the admingroup, the export links will be available near the top of the modality filter pages in the OpenREM interface.

For each modality you can export to a single-sheet csv file or a multi-sheet xlsx file. In addition, there is an export tailored to the NHSBSP dose audits requirements.

If you are logged in as a user in the pidgroup you will also have a choice of exporting with patient name and/or patient ID information included in the export (if any is recorded in the database). See Patient identifiable data for more details.

The xlsx export has multiple sheets. The first sheet contains a summary of all the study descriptions, requested procedures and series protocol names contained in the export:

CT export front sheet

This information is useful for seeing what data is in the spreadsheet, and can also be used to prioritise which studies or protocols to analyse based on frequency.

The second sheet of the exported file lists all the studies, with each study taking one line and each series in the study displayed in the columns to the right.

CT export all data sheet

The remainder of the file has one sheet per series protocol name. Each series is listed one per line. If a single study has more than one series with the same protocol name, then the same study will appear on more than one line.

Fluoroscopy exports

Fluoroscopy csv exports only report study level information — this includes summary dose information for fluoroscopy exposures and acquisition exposures, but not information about individual exposures.

Fluoroscopy xlsx exports contain the following sheets:

  • Summary sheet
  • All data sheet with groups of exposures
  • One sheet per acquisition protocol with one row per exposure including all the details of that exposure.

Exposures are considered similar and put in the same group if they are, relative to the first exposure of a group:

  • same plane (for bi-plane systems)
  • same protocol
  • same field size (mag, not collimation)
  • same pulse rate
  • same filter material and thickness
  • within 5° in both directions (primary and secondary)
  • of the same ‘event type’ (Fluoroscopy, Stationary Acquisition, Stepping Acquisition, Rotational Acquisition)

The minimum, maximum and mean of all the remaining factors are presented for each group along with the common factors. Where a factor is not available in the source RDSR, that factor is not considered.

The grouping process for the all data sheet takes a lot of time compared to the other exports. However, we hope that this is a useful way of comprehending the study. Other modalities have all the series for any one study detailed in full on one long row — this is not possible when one study might have 400 exposures!

The majority of systems report kV, mA and pulse width information as a mean value per exposure. Some systems report this information on a per pulse basis instead. In this circumstance, in the web interface you will see the list of pulses, but in the export the mean value (after excluding any zero values) is calculated first and this value is then used.

Exports page

Clicking the link for an export redirects you to the Exports page, which you can also get to using the link at the top right of the navigation bar:

Exports list

Whilst an export is being processed, it will be listed in the first table at the top. The current status is displayed to indicate export progress, and is updated every two seconds. You can stop an export early by using the abort button; you will not be able to download anything in this instance.

Once a study is complete a new table of recently completed exams is created and you will be able to download the file.

When the export is no longer needed, it can be deleted from the server by ticking the delete checkbox and clicking the delete button at the bottom:

Deleting exports

Specific modality export information

NHSBSP dose audits

This export is specific to the UK NHS Breast Screening Programme and generates the source data in the format required for the dose audit database developed by the National Co-ordinating Centre for the Physics of Mammography.

It has been modified to clean up the data to remove exposures that are unlikely to be wanted in the submitted data, such as exposures with any of the following in the protocol name:

scout, postclip, prefire, biopsy, postfire, stereo, specimin, artefact

The view codes have been modified to match the NCCPM convention, i.e. medio-lateral oblique is recorded as OB instead of MLO. The other codes are mapped to the ACR MQCM 1999 Equivalent code.

Each patient is numbered from starting from 1. Each view for any one patient has a unique view code, so if a second cranio-caudal exposure is made to the left breast the view codes will be LCC and LCC2.

The survey number is left as 1. This needs to be modified as appropriate. The easiest way to do this in Excel is to change the first two or three rows, select those cells that have been changed, then double click on the bottom-right corner of the selection box to copy-down the value to all the remaining cells below.

The data can then be copied and pasted into the NCCPM database.

If there are a mixture of 2D and tomography exposures, providing you can separate them by virtue of the filter used, then you should further prepare the data as follows:

  1. Copy the sheet to a new sheet
  2. In the first sheet, filter for the target and filter combination used for used for the tomographic exposures and delete those rows.
  3. In the second sheet, filter for the target and filter combinations used for 2D exposures and delete those rows.
  4. Change the survey number on the 2D sheet and the the survey number on the tomographic sheet as appropriate, with the tomographic survey number bing one more than the 2D survey number.

Where patients have had both 2D and tomographic exposures in the same study, NCCPM will be able to match them up as they will have the same patient number in both surveys.

Opening csv exports in Excel

If the export contains non-ASCII characters, then Microsoft Excel is unlikely to display them correctly by default. This issue does not occur with Libre Office which defaults to UTF-8 – behaviour with other applications will vary.

To correctly render characters in csv files with Excel, you will need to follow the following procedure:

  1. Open Excel.
  2. On the Data tab of the ribbon interface, select From Text in the Get External Data section.
  3. Select your exported csv file and click Import
  4. Ensure that Data Type Fixed width is selected.
  5. Change the File origin from Windows (ANSI) to 65001 : Unicode (UTF-8) – the easiest way to find it is to scroll right to the bottom of the list, then move up one.
  6. Click Next >
  7. Change the delimiter to just Comma
  8. Either click Finish or Next > if you want to further customise the import.

Troubleshooting

Server 500 errors

Turn on debug mode

Locate and edit your local_settings file

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/openremproject/local_settings.py

  • Other linux: /usr/lib/python2.7/site-packages/openrem/openremproject/local_settings.py

  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/openremproject/local_settings.py

  • Windows: C:\Python27\Lib\site-packages\openrem\openremproject\local_settings.py

  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\openremproject\local_settings.py

  • Change the line:

    # DEBUG = True
    
  • to:

    DEBUG = True
    

This will render a debug report in the browser - usually revealing the problem.

Once the problem is fixed, change DEBUG to False, or comment it again using a #. If you leave debug mode in place, the system is likely to run out of memory as database queries are cached.

Unknown encoding errors

Background

OpenREM versions 0.8 and earlier rely on pydicom 0.9.9 which has recently had a new release, version 1.0. A replacement of pynetdicom is also being worked on and will be called pynetdicom3. When pynetdicom3 is released, OpenREM will be revised to work the two new packages, but as they are both backwards-incompatible with the old versions you must keep using the versions specified by the installation instructions until this happens. (The correct version of pydicom is installed automatically when you install OpenREM).

The function that deals with character set encoding in pydicom 0.9.9 has not been edited since 2013 and is missing a lot of encodings which can cause imports and query-retrieve functions to fail if those encodings are encountered.

As an interim work-around, users that encounter this issue can take the following step to work with those files in OpenREM 0.8.

Work-around

Find and edit the pydicom file charset.py. It should be at one of the following paths:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/dicom/charset.py
  • Other linux: /usr/lib/python2.7/site-packages/dicom/charset.py
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/dicom/charset.py
  • Windows: C:\Python27\Lib\site-packages\dicom\charset.py
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\dicom\charset.py

Replace the dictionary python_encoding that is declared at the start with the following dictionary:

# Map DICOM Specific Character Set to python equivalent
python_encoding = {

    # default character set for DICOM
    '': 'iso8859',

    # alias for latin_1 too (iso_ir_6 exists as an alias to 'ascii')
    'ISO_IR 6': 'iso8859',
    'ISO_IR 13': 'shift_jis',

    # these also have iso_ir_1XX aliases in python 2.7
    'ISO_IR 100': 'latin_1',
    'ISO_IR 101': 'iso8859_2',
    'ISO_IR 109': 'iso8859_3',
    'ISO_IR 110': 'iso8859_4',
    'ISO_IR 126': 'iso_ir_126',  # Greek
    'ISO_IR 127': 'iso_ir_127',  # Arabic
    'ISO_IR 138': 'iso_ir_138',  # Hebrew
    'ISO_IR 144': 'iso_ir_144',  # Russian
    'ISO_IR 148': 'iso_ir_148',  # Turkish
    'ISO_IR 166': 'iso_ir_166',  # Thai
    'ISO 2022 IR 6': 'iso8859',  # alias for latin_1 too
    'ISO 2022 IR 13': 'shift_jis',
    'ISO 2022 IR 87': 'iso2022_jp',
    'ISO 2022 IR 100': 'latin_1',
    'ISO 2022 IR 101': 'iso8859_2',
    'ISO 2022 IR 109': 'iso8859_3',
    'ISO 2022 IR 110': 'iso8859_4',
    'ISO 2022 IR 126': 'iso_ir_126',
    'ISO 2022 IR 127': 'iso_ir_127',
    'ISO 2022 IR 138': 'iso_ir_138',
    'ISO 2022 IR 144': 'iso_ir_144',
    'ISO 2022 IR 148': 'iso_ir_148',
    'ISO 2022 IR 149': 'euc_kr',  # needs cleanup via clean_escseq()
    'ISO 2022 IR 159': 'iso-2022-jp',
    'ISO 2022 IR 166': 'iso_ir_166',
    'ISO 2022 IR 58': 'iso_ir_58',
    'ISO_IR 192': 'UTF8',  # from Chinese example, 2008 PS3.5 Annex J p1-4
    'GB18030': 'GB18030',
    'ISO 2022 GBK': 'GBK',  # from DICOM correction CP1234
    'ISO 2022 58': 'GB2312',  # from DICOM correction CP1234
    'GBK': 'GBK',  # from DICOM correction CP1234
}

Leave the rest of the file as it was and save. You should now be able to work with DICOM encoded with a much wider range of character sets.

Invalid tag errors

Background

OpenREM versions 0.8 and earlier rely on pydicom 0.9.9 which has recently had a new release, version 1.0. A replacement of pynetdicom is also being worked on and will be called pynetdicom3. When pynetdicom3 is released, OpenREM will be revised to work the two new packages, but as they are both backwards-incompatible with the old versions you must keep using the versions specified by the installation instructions until this happens. (The correct version of pydicom is installed automatically when you install OpenREM).

The DICOM dictionary (the list of all the tags) supplied with pydicom 0.9.9 is old and doesn’t have some of the newer tags that some modalities are starting to include. This can lead to errors when you attempt to import a file with one of these tags, and the file will not be imported.

Error message

The error message will look something like this. The actual tag can be different, this is an example.

Error

KeyError: ‘Invalid tag (0018, 9559): “Unknown DICOM tag (0018, 9559) - can’t look up VR”’

Work-around

The workaround is to replace the _dicom_dict.py with the version in the current master of pydicom.

First, download the current file (right click ‘Save link as…’ or similar): _dicom_dict.py

Next, find the location of your pydicom install – it should be at one of the following paths:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/dicom/
  • Other linux: /usr/lib/python2.7/site-packages/dicom/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/dicom/
  • Windows: C:\Python27\Lib\site-packages\dicom\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\dicom\

Replace the existing _dicom_dict.py file with the one you have downloaded.

Try and import the file again - it should now work.

Fixing accumulated AGD and laterality for Hologic DBT

The code for extracting dose related information from Hologic digital breast tomosynthesis proprietary projection images object used an incorrect tag to extract the laterality of the image. As a result the accumulated AGD code didn’t work, so the accumulated AGD cell on the mammography summary sheets remained blank.

The code below allows the laterality to be derived automatically from the acquisition protocol name, if the protocol name has R or L as the first letter. When this information is derived, the accumulated AGD is also calculated per breast.

Back up your database

It is always best practice to backup your database before running code to edit your database:

  • For PostgreSQL you can refer to Backup the database
  • For a non-production SQLite3 database, simply make a copy of the database file

How to use the code

Create a new file in your Python OpenREM folder (the folder manage.py is in). For example it could be called fix_dbt_laterality.py:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/fix_dbt_laterality.py
  • Other linux: /usr/lib/python2.7/site-packages/openrem/fix_dbt_laterality.py
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/fix_dbt_laterality.py
  • Windows: C:\Python27\Lib\site-packages\openrem\fix_dbt_laterality.py
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\fix_dbt_laterality.py

Copy and paste into the new file the code from below. You will need to edit the DISPLAY_NAME to match the display name you have configured for the Hologic DBT system that needs to be modified. If you are not sure what this is, go to the home page of your OpenREM installation and see how the Hologic unit is listed there. You will not be able to copy and paste from there as if you click on it it will load that page, and likewise on the summary list page. If you click through to the study detail page, you will find the display name listed in the details there. See the Display names and user-defined modalities documentation for more information.

If you are working on Linux, you may like to look at the brief tips on using nano on the Nginx Troubleshooting and tips section.

# -*- coding: utf-8 -*-
# Routine to fix missing laterality details for exposures extracted from Hologic DBT proprietary projection data objects
# Also retrospectively updates the accumulated average glandular dose fields that were previously missing due to lack
# of laterality.
#
# Contains hard coded DISPLAY_NAME that must be changed appropriately to identify the Hologic DBT system in your
# database.
# See https://bitbucket.org/openrem/openrem/issues/411

import os
import sys
import django
import logging
from django.core.exceptions import ObjectDoesNotExist

logger = logging.getLogger(__name__)

# setup django/OpenREM
basepath = os.path.dirname(__file__)
projectpath = os.path.abspath(os.path.join(basepath, ))
if projectpath not in sys.path:
    sys.path.insert(1, projectpath)
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'openremproject.settings')
django.setup()

from remapp.models import IrradEventXRayData
from remapp.tools.get_values import get_or_create_cid

# ******************************************************
# Update the DISPLAY_NAME
DISPLAY_NAME = "Display name of my Hologic"
# ******************************************************

def _accumulatedxraydose(proj):
    """
    Borrowed from mam.py
    Creates an AccumXRayDose table entry
    :param proj: Projection x-ray radiation dose database entry
    :return: Nothing
    """
    from remapp.models import AccumXRayDose, AccumMammographyXRayDose
    from remapp.tools.get_values import get_or_create_cid
    accum = AccumXRayDose.objects.create(projection_xray_radiation_dose=proj)
    accum.acquisition_plane = get_or_create_cid('113622', 'Single Plane')
    accum.save()
    accummam = AccumMammographyXRayDose.objects.create(accumulated_xray_dose=accum)
    accummam.accumulated_average_glandular_dose = 0.0
    accummam.save()


def _accumulatedmammo_update(event):  # TID 10005
    """
    Borrowed from mam.py
    Updates the accumulated average glandular dose tables on a per-breast basis
    :param event: Irradiation event database entry
    :return: Nothing
    """
    from remapp.tools.get_values import get_or_create_cid
    from remapp.models import AccumMammographyXRayDose
    try:
        accum = event.projection_xray_radiation_dose.accumxraydose_set.get()
    except ObjectDoesNotExist:
        print("No accumxraydose for event occurring at {0} in study no {1}".format(event.date_time_started,
            event.projection_xray_radiation_dose.general_study_module_attributes.id))
        _accumulatedxraydose(event.projection_xray_radiation_dose)
        accum = event.projection_xray_radiation_dose.accumxraydose_set.get()
    accummams = accum.accummammographyxraydose_set.all()
    event_added = False
    for accummam in accummams:
        if not accummam.laterality:
            if event.laterality.code_meaning == 'Right':
                accummam.laterality = get_or_create_cid('T-04020', 'Right breast')
            elif event.laterality.code_meaning == 'Left':
                accummam.laterality = get_or_create_cid('T-04030', 'Left breast')
            accummam.accumulated_average_glandular_dose += event.irradeventxraysourcedata_set.get(
                ).average_glandular_dose
            accummam.save()
            event_added = True
        elif event.laterality.code_meaning in accummam.laterality.code_meaning:
            accummam.accumulated_average_glandular_dose += event.irradeventxraysourcedata_set.get(
                ).average_glandular_dose
            accummam.save()
            event_added = True
    if not event_added:
        accummam = AccumMammographyXRayDose.objects.create(accumulated_xray_dose=accum)
        if event.laterality.code_meaning == 'Right':
            accummam.laterality = get_or_create_cid('T-04020', 'Right breast')
        elif event.laterality.code_meaning == 'Left':
            accummam.laterality = get_or_create_cid('T-04030', 'Left breast')
        accummam.accumulated_average_glandular_dose = event.irradeventxraysourcedata_set.get().average_glandular_dose
        accummam.save()
    accummam.save()


events = IrradEventXRayData.objects.filter(
    projection_xray_radiation_dose__general_study_module_attributes__generalequipmentmoduleattr__unique_equipment_name__display_name__exact=DISPLAY_NAME)

events_r = events.filter(laterality__code_meaning__exact=u"Right")
events_l = events.filter(laterality__code_meaning__exact=u"Left")
events_n = events.filter(laterality__isnull=True)
print(u"Total events is {0}, of which {1} are Right, {2} are Left and {3} are null (remainder {4})".format(
    events.count(), events_r.count(), events_l.count(), events_n.count(),
    events.count() - events_r.count() - events_l.count() - events_n.count()))

for event in events_n:
    if event.acquisition_protocol[0] == u'R':
        event.laterality = get_or_create_cid('G-A100', 'Right')
        event.save()
        _accumulatedmammo_update(event)
    elif event.acquisition_protocol[0] == u'L':
        event.laterality = get_or_create_cid('G-A101', 'Left')
        event.save()
        _accumulatedmammo_update(event)
    else:
        print("Event acquisition protocol is {0} so we couldn't assign it left or right. Exam ID is {1}".format(
            event.acquisition_protocol, event.projection_xray_radiation_dose.general_study_module_attributes.id))

events_r = events.filter(laterality__code_meaning__exact=u"Right")
events_l = events.filter(laterality__code_meaning__exact=u"Left")
events_n = events.filter(laterality__isnull=True)
print(u"Post update, total events is {0}, of which {1} are Right, {2} are Left and {3} are null (remainder {4})".format(
    events.count(), events_r.count(), events_l.count(), events_n.count(),
    events.count() - events_r.count() - events_l.count() - events_n.count()))

Run the fix

In a shell/command window, activate your virtualenv if you are using one, and change directory to the openrem folder:

  • Ubuntu linux: cd /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: cd /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: cd virtualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: cd C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: cd virtualenvfolder\Lib\site-packages\openrem\

Then:

python fix_dbt_laterality.py

This should generate the following response, with one message for each event that can’t be assigned laterality due to the acquisition protocol name not starting with L or R:

Total events is 46410, of which 0 are Right, 0 are Left and 46410 are null (remainder 0)
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 184466
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 75963
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 75919
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 76004
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 83784
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 83784
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 84912
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 100765
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 110471
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 121500
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 121588
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 123462
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 137145
Event acquisition protocol is ACR Phantom Combo so we couldn't assign it left or right. Exam ID is 140563
Event acquisition protocol is Flat Field Tomo so we couldn't assign it left or right. Exam ID is 156826
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 165131
Event acquisition protocol is Flat Field Combo so we couldn't assign it left or right. Exam ID is 165486
Post update, total events is 46410, of which 23323 are Right, 23070 are Left and 17 are null (remainder 0)

The Exam ID referred to is the database ID, so if you look at a mammography exam in the web interface, you can change the Exam ID in the URL if you want to review that study.

Multiple Hologic units

If you have more than one unit that has studies that need fixing, simply change the DISPLAY_NAME and run the code again.

If you have a modality where every study has one event (usually CT), review

If planar X-ray studies are appearing in fluoroscopy or vice-versa, review

For DICOM networking:

Log files

Log file location, naming and verbosity were configured in the local_settings.py configuration - see the Log file configuration docs for details.

If the defaults have not been modified, then there will be three log files in your MEDIAROOT folder which you configured at installation. See the install config section on Location for imports and exports for details.

The openrem.log has general logging information, the other two are specific to the DICOM store and DICOM query-retrieve functions if you are making use of them.

You can increase the verbosity of the log files by changing the log ‘level’ to DEBUG, or you can decrease the verbosity to WARNING, ERROR, or CRITICAL. The default is INFO.

Starting again!

If for any reason you want to start again with the database, then this is how you might do it:

SLQite3 database

  • Delete or rename your existing database file (location will be described in your local_settings.py file)
  • Create the database

Any database

These instructions will also allow you to keep any user settings if you use an SQLite3 database.

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: cd /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: cd /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: cd virtualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: cd C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: cd virtualenvfolder\Lib\site-packages\openrem\

Run the django python shell:

python manage.py shell

from remapp.models import GeneralStudyModuleAttr
a = GeneralStudyModuleAttr.objects.all()
a.count()  # Just to see that we are doing something!
a.delete()
a.count()
exit()

Developers

Contents:

DICOM import modules

RDSR module

Ultimately this should be the only module required as it deals with all Radiation Dose Structured Reports. This is used for CT, fluoroscopy, mammography and digital radiography.

(task)openrem.remapp.extractors.rdsr.rdsr(rdsr_file)

Extract radiation dose related data from DICOM Radiation SR objects.

Parameters:filename (str.) – relative or absolute path to Radiation Dose Structured Report.

Mammography module

Mammography is interesting in that all the information required for dose audit is contained in the image header, including patient ‘size’, ie thickness. However the disadvantage over an RSDR is the requirement to process each individual image rather than a single report for the study, which would also capture any rejected images.

(task)openrem.remapp.extractors.mam.mam(mg_file)

Extract radiation dose structured report related data from mammography images

Parameters:mg_file (str.) – relative or absolute path to mammography DICOM image file.

CR and DR module

In practice this is only useful for DR modalities, but most of them use the CR IOD instead of the DX one, so both are catered for. This module makes use of the image headers much like the mammography module.

(task)openrem.remapp.extractors.dx.dx(dig_file)

Extract radiation dose structured report related data from DX radiographic images

Parameters:filename (str.) – relative or absolute path to DICOM DX radiographic image file.

CT non-standard modules

Philips CT dose info reports

These have all the information that could be derived from the images also held in the DICOM header information, making harvesting relatively easy. Used where RDSR is not available from older Philips systems.

(task)openrem.remapp.extractors.ct_philips.ct_philips(philips_file)

Extract radiation dose structured report related data from Philips CT dose report images

Parameters:filename (str.) – relative or absolute path to Philips CT dose report DICOM image file.
Tested with:
  • Philips Gemini TF PET-CT v2.3.0
  • Brilliance BigBore v3.5.4.17001.
Toshiba dose summary and images

OpenREM can harvest information from older Toshiba CT systems that create dose summary images but cannot create RDSR objects by using a combination of tools to create an RDSR that can then be imported in the normal manner. This extractor requires that the Offis DICOM toolkit, java.exe and pixelmed.jar are available to the system.

(task)openrem.remapp.extractors.ct_toshiba.ct_toshiba(folder_name)

Function to create radiation dose structured reports from a folder of dose images.

Parameters:folder_name – Path to folder containing Toshiba DICOM objects - dose summary and images

Non-DICOM import modules

Patient height and weight csv import module

This module enables a csv file to be parsed and the height and weight information extracted and added to existing studies in the OpenREM database. An example may be a csv extract from a RIS or EPR system.

There needs to be a common unique identifier for the exam - currently this is limited to accession number or study instance UID.

remapp.extractors.ptsizecsv2db.csv2db(*args, **kwargs)

Import patient height and weight data from csv RIS exports. Can be called from openrem_ptsizecsv.py script

Parameters:
  • --si-uid (bool) – Use Study Instance UID instead of Accession Number. Short form -s.
  • csvfile (str) – relative or absolute path to csv file
  • id (str) – Accession number column header or header if -u or –si-uid is set. Quote if necessary.
  • height (str) – Patient height column header. Create if necessary, quote if necessary. Values in cm.
  • weight (str) – Patient weight column header. Create if necessary, quote if necessary. Values in kg.

Example:

openrem_ptsizecsv.py -s MyRISExport.csv StudyInstanceUID height weight
(task)remapp.extractors.ptsizecsv2db.websizeimport(csv_pk=None, *args, **kwargs)

Task to import patient size data from the OpenREM web interface.

Parameters:csv_pk – Database index key for the import record, containing the path to the import csv file and the field header details.

Export from database

Multi-sheet Microsoft Excel XLSX exports

This export has a summary sheet of all the requested and performed protocols and the series protocols. The next sheet has all studies on, one study per line, with the series stretching off to the right. The remaining sheets are specific to each series protocol, in alphabetical order, with one series per line. If one study has three series with the same protocol name, each one has a line of its own.

(task)remapp.exports.rf_export.rfxlsx(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered RF database data to multi-sheet Microsoft XSLX files.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves xlsx file into Media directory for user to download

(task)remapp.exports.ct_export.ctxlsx(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered CT database data to multi-sheet Microsoft XSLX files

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves xlsx file into Media directory for user to download

(task)remapp.exports.dx_export.dxxlsx(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered DX and CR database data to multi-sheet Microsoft XSLX files.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves xlsx file into Media directory for user to download

(task)remapp.exports.mg_export.exportMG2excel(filterdict, pid=False, name=None, patid=None, user=None, xlsx=False)

Export filtered mammography database data to a single-sheet CSV file or a multi sheet xlsx file.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
  • xlsx – Whether to export a single sheet csv or a multi sheet xlsx
Returns:

Saves csv file into Media directory for user to download

Single sheet CSV exports

(task)remapp.exports.rf_export.exportFL2excel(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered fluoro database data to a single-sheet CSV file.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves csv file into Media directory for user to download

(task)remapp.exports.ct_export.ct_csv(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered CT database data to a single-sheet CSV file.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves csv file into Media directory for user to download

(task)remapp.exports.mg_export.exportMG2excel(filterdict, pid=False, name=None, patid=None, user=None, xlsx=False)

Export filtered mammography database data to a single-sheet CSV file or a multi sheet xlsx file.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
  • xlsx – Whether to export a single sheet csv or a multi sheet xlsx
Returns:

Saves csv file into Media directory for user to download

(task)remapp.exports.dx_export.exportDX2excel(filterdict, pid=False, name=None, patid=None, user=None)

Export filtered DX database data to a single-sheet CSV file.

Parameters:
  • filterdict – Queryset of studies to export
  • pid – does the user have patient identifiable data permission
  • name – has patient name been selected for export
  • patid – has patient ID been selected for export
  • user – User that has started the export
Returns:

Saves csv file into Media directory for user to download

Specialised csv exports - NHSBSP formatted mammography export
(task)remapp.exports.mg_csv_nhsbsp.mg_csv_nhsbsp(filterdict, user=None)

Export filtered mammography database data to a NHSBSP formatted single-sheet CSV file.

Parameters:filterdict (dict) – Dictionary of query parameters from the mammo filtered page URL.
Returns:None - file is saved to disk and location is stored in database

Tools and helper modules

OpenREM settings

Administrative module to define the name of the project and to add it to the Python path

remapp.extractors.openrem_settings.add_project_to_path()

Add project to path, assuming this file is within project

Get values

Tiny modules to reduce repetition in the main code when extracting information from DICOM headers using pydicom.

remapp.tools.get_values.get_value_kw(tag, dataset)

Get DICOM value by keyword reference.

Parameters:
  • tag (str.) – DICOM keyword, no spaces or plural as per dictionary.
  • dataset (dataset) – The DICOM dataset containing the tag.
Returns:

str. – value

remapp.tools.get_values.get_value_num(tag, dataset)

Get DICOM value by tag group and element number.

Always use get_value_kw by preference for readability. This module can be required when reading private elements.

Parameters:
  • tag (hex) – DICOM group and element number as a single hexadecimal number (prefix 0x).
  • dataset (dataset) – The DICOM dataset containing the tag.
Returns:

str. – value

remapp.tools.get_values.get_seq_code_value(sequence, dataset)

From a DICOM sequence, get the code value.

Parameters:
  • sequence (DICOM keyword, no spaces or plural as per dictionary.) – DICOM sequence name.
  • dataset (DICOM dataset) – The DICOM dataset containing the sequence.
Returns:

int. – code value

remapp.tools.get_values.get_seq_code_meaning(sequence, dataset)

From a DICOM sequence, get the code meaning.

Parameters:
  • sequence (DICOM keyword, no spaces or plural as per dictionary.) – DICOM sequence name.
  • dataset (DICOM dataset) – The DICOM dataset containing the sequence.
Returns:

str. – code meaning

remapp.tools.get_values.get_or_create_cid(codevalue, codemeaning)

Create a code_value code_meaning pair entry in the ContextID table if it doesn’t already exist.

Parameters:
  • codevalue (int.) – Code value as defined in the DICOM standard part 16
  • codemeaning – Code meaning as defined in the DICOM standard part 16
Returns:

ContextID entry for code value passed

remapp.tools.get_values.return_for_export(model, field)

Prevent errors due to missing data in models :param model: database table :param field: database field :return: value or None

remapp.tools.get_values.export_csv_prep(unicode_string)

Built-in csv module can’t deal with unicode strings without specifying an encoding. Hence encode to utf-8 before writing. :param unicode_string: String to encode as utf-8 :return: UTF-8 encoded string with no commas or semicolons.

remapp.tools.get_values.list_to_string(dicom_value)

Turn multivalue names into a single string for correct encoding and pretty reproduction :param dicom_value: returned DICOM value, usually a name field. Might be single (string) or multivalue (list) :return: string of name(s)

Check if UID exists

Small module to check if UID already exists in the database.

remapp.tools.check_uid.check_uid(uid, level='Study')

Check if UID already exists in database.

Parameters:uid (str.) – Study UID.
Returns:1 if it does exist, 0 otherwise
remapp.tools.check_uid.record_sop_instance_uid(study, sop_instance_uid)

Record the object’s SOP Instance UID so we can ignore it next time. If an object does need to be imported again, the original one needs to be deleted first.

Parameters:
  • study – GeneralStudyModuleAttr database object
  • sop_instance_uid – SOP Instance UID of object being imported
Returns:

DICOM time and date values

Module to convert betweeen DICOM and Python dates and times.

remapp.tools.dcmdatetime.get_date(tag, dataset)

Get DICOM date string and return Python date.

Parameters:
  • tag (str.) – DICOM keyword, no spaces or plural as per dictionary.
  • dataset (dataset) – The DICOM dataset containing the tag.
Returns:

Python date value

remapp.tools.dcmdatetime.get_time(tag, dataset)

Get DICOM time string and return Python time.

Parameters:
  • tag (str.) – DICOM keyword, no spaces or plural as per dictionary.
  • dataset (dataset) – The DICOM dataset containing the tag.
Returns:

python time value

remapp.tools.dcmdatetime.get_date_time(tag, dataset)

Get DICOM date time string and return Python date time.

Parameters:
  • tag (str.) – DICOM keyword, no spaces or plural as per dictionary.
  • dataset (dataset) – The DICOM dataset containing the tag.
Returns:

Python date time value

remapp.tools.dcmdatetime.make_date(dicomdate)

Given a DICOM date, return a Python date.

Parameters:dicomdate (str.) – DICOM style date.
Returns:Python date value
remapp.tools.dcmdatetime.make_time(dicomtime)

Given a DICOM time, return a Python time.

Parameters:dicomdate (str.) – DICOM style time.
Returns:Python time value
remapp.tools.dcmdatetime.make_date_time(dicomdatetime)

Given a DICOM date time, return a Python date time.

Parameters:dicomdate (str.) – DICOM style date time.
Returns:Python date time value
remapp.tools.dcmdatetime.make_dcm_date(pythondate)

Given a Python date, return a DICOM date :param pythondate: Date :type pythondate: Python date object :returns: DICOM date as string

remapp.tools.dcmdatetime.make_dcm_date_range(date1=None, date2=None)

Given one or two dates of the form yyyy-mm-dd, return a DICOM date range :param: date1, date2: One or two yyyy-mm-dd dates :type date1, date2: String :returns: DICOM date range as string

Test for QA or other non-patient related studies

remapp.tools.not_patient_indicators.get_not_pt(dataset)

Looks for indications that a study might be a test or QA study.

Some values that might indicate a study was for QA or similar purposes are not recorded in the database, for example patient name. Therefore this module attempts to find such indications and creates an xml style string that can be recorded in the database on study import.

Parameters:dataset (dataset) – The DICOM dataset.
Returns:str. – xml style string if any trigger values are found.

Models

class remapp.models.AdminTaskQuestions(*args, **kwargs)

Record if admin tasks have been dealt with

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.NotPatientIndicatorsID(*args, **kwargs)

Table to record strings that indicate a patient ID is really a test or QA ID

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.NotPatientIndicatorsName(*args, **kwargs)

Table to record strings that indicate a patient name is really a test or QA name

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.SkinDoseMapCalcSettings(id, enable_skin_dose_maps, calc_on_import)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomDeleteSettings(id, del_no_match, del_rdsr, del_mg_im, del_dx_im, del_ct_phil)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.PatientIDSettings(id, name_stored, name_hashed, id_stored, id_hashed, accession_hashed, dob_stored)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomStoreSCP(id, name, aetitle, port, task_id, status, run, keep_alive, controlled)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomRemoteQR(id, name, aetitle, port, ip, hostname, callingaet, enabled)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomQuery(id, complete, query_id, failed, message, stage, qr_scp_fk, store_scp_fk, move_complete)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomQRRspStudy(id, dicom_query, query_id, study_instance_uid, modality, modalities_in_study, study_description, number_of_study_related_series, sop_classes_in_study, station_name)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomQRRspSeries(id, dicom_qr_rsp_study, query_id, series_instance_uid, series_number, series_time, modality, series_description, number_of_series_related_instances, station_name, sop_class_in_series, image_level_move)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DicomQRRspImage(id, dicom_qr_rsp_series, query_id, sop_instance_uid, instance_number, sop_class_uid)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.UserProfile(id, itemsPerPage, user, median_available, plotAverageChoice, plotInitialSortingDirection, plotCharts, plotDXAcquisitionMeanDAP, plotDXAcquisitionFreq, plotDXStudyMeanDAP, plotDXStudyFreq, plotDXRequestMeanDAP, plotDXRequestFreq, plotDXAcquisitionMeankVp, plotDXAcquisitionMeanmAs, plotDXStudyPerDayAndHour, plotDXAcquisitionMeankVpOverTime, plotDXAcquisitionMeanmAsOverTime, plotDXAcquisitionMeanDAPOverTime, plotDXAcquisitionMeanDAPOverTimePeriod, plotDXInitialSortingChoice, plotCTAcquisitionMeanDLP, plotCTAcquisitionMeanCTDI, plotCTAcquisitionFreq, plotCTStudyMeanDLP, plotCTStudyMeanCTDI, plotCTStudyFreq, plotCTStudyNumEvents, plotCTRequestMeanDLP, plotCTRequestFreq, plotCTRequestNumEvents, plotCTStudyPerDayAndHour, plotCTStudyMeanDLPOverTime, plotCTStudyMeanDLPOverTimePeriod, plotCTInitialSortingChoice, plotRFStudyPerDayAndHour, plotRFStudyFreq, plotRFStudyDAP, plotRFInitialSortingChoice, plotMGStudyPerDayAndHour, plotMGAGDvsThickness, plotMGkVpvsThickness, plotMGmAsvsThickness, displayCT, displayRF, displayMG, displayDX, plotSeriesPerSystem, plotHistogramBins, plotHistograms, plotCaseInsensitiveCategories)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.UniqueEquipmentNames(id, manufacturer, manufacturer_hash, institution_name, institution_name_hash, station_name, station_name_hash, institutional_department_name, institutional_department_name_hash, manufacturer_model_name, manufacturer_model_name_hash, device_serial_number, device_serial_number_hash, software_versions, software_versions_hash, gantry_id, gantry_id_hash, display_name, user_defined_modality, hash_generated)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.SizeUpload(id, sizefile, height_field, weight_field, id_field, id_type, task_id, status, progress, num_records, logfile, import_date, processtime)
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.Exports(*args, **kwargs)

Table to hold the export status and filenames

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ContextID(*args, **kwargs)

Table to hold all the context ID code values and code meanings.

  • Could be prefilled from the tables in DICOM 3.16, but is actually populated as the codes occur. This assumes they are used correctly.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.GeneralStudyModuleAttr(*args, **kwargs)

General Study Module C.7.2.1

Specifies the Attributes that describe and identify the Study performed upon the Patient. From DICOM Part 3: Information Object Definitions Table C.7-3

Additional to the module definition:
  • performing_physician_name
  • operator_name
  • modality_type
  • procedure_code_value_and_meaning
  • requested_procedure_code_value_and_meaning
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ObjectUIDsProcessed(*args, **kwargs)

Table to hold the SOP Instance UIDs of the objects that have been processed against this study to enable duplicate sorting.

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ProjectionXRayRadiationDose(*args, **kwargs)

Projection X-Ray Radiation Dose template TID 10001

From DICOM Part 16:
This template defines a container (the root) with subsidiary content items, each of which represents a single projection X-Ray irradiation event entry or plane-specific dose accumulations. There is a defined recording observer (the system or person responsible for recording the log, generally the system). A Biplane irradiation event will be recorded as two individual events, one for each plane. Accumulated values will be kept separate for each plane.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.AccumXRayDose(*args, **kwargs)

Accumulated X-Ray Dose TID 10002

From DICOM Part 16:
This general template provides detailed information on projection X-Ray dose value accumulations over several irradiation events from the same equipment (typically a study or a performed procedure step).
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.Calibration(*args, **kwargs)

Table to hold the calibration information

  • Container in TID 10002 Accumulated X-ray dose
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.IrradEventXRayData(*args, **kwargs)

Irradiation Event X-Ray Data TID 10003

From DICOM part 16:
This template conveys the dose and equipment parameters of a single irradiation event.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ImageViewModifier(*args, **kwargs)

Table to hold image view modifiers for the irradiation event x-ray data table

From DICOM Part 16 Annex D DICOM controlled Terminology Definitions
  • Code Value 111032
  • Code Meaning Image View Modifier
  • Code Definition Modifier for image view
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.IrradEventXRayDetectorData(*args, **kwargs)

Irradiation Event X-Ray Detector Data TID 10003a

From DICOM Part 16 Correction Proposal CP-1077:
This template contains data which is expected to be available to the X-ray detector or plate reader component of the equipment.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.IrradEventXRaySourceData(*args, **kwargs)

Irradiation Event X-Ray Source Data TID 10003b

From DICOM Part 16 Correction Proposal CP-1077:
This template contains data which is expected to be available to the X-ray source component of the equipment.
Additional to the template:
  • ii_field_size
  • exposure_control_mode
  • grid information over and above grid type
convert_gy_to_mgy()

Converts Gy to mGy for display in web interface

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.XrayGrid(*args, **kwargs)

Content ID 10017 X-Ray Grid

From DICOM Part 16

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.PulseWidth(*args, **kwargs)

In TID 10003b. Code value 113793 (ms)

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.Kvp(*args, **kwargs)

In TID 10003b. Code value 113733 (kV)

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.XrayTubeCurrent(*args, **kwargs)

In TID 10003b. Code value 113734 (mA)

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.Exposure(*args, **kwargs)

In TID 10003b. Code value 113736 (uAs)

convert_uAs_to_mAs()

Converts uAs to mAs for display in web interface

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.XrayFilters(*args, **kwargs)

Container in TID 10003b. Code value 113771

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.IrradEventXRayMechanicalData(*args, **kwargs)

Irradiation Event X-Ray Mechanical Data TID 10003c

From DICOM Part 16 Correction Proposal CP-1077:
This template contains data which is expected to be available to the gantry or mechanical component of the equipment.
Additional to the template:
  • compression_force
  • magnification_factor
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DoseRelatedDistanceMeasurements(*args, **kwargs)

Dose Related Distance Measurements Context ID 10008

Called from TID 10003c

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.AccumProjXRayDose(*args, **kwargs)

Accumulated Fluoroscopy and Acquisition Projection X-Ray Dose TID 10004

From DICOM Part 16:
This general template provides detailed information on projection X-Ray dose value accumulations over several irradiation events from the same equipment (typically a study or a performed procedure step).
fluoro_gym2_to_cgycm2()

Converts fluoroscopy DAP total from Gy.m2 to cGy.cm2 for display in web interface

acq_gym2_to_cgycm2()

Converts acquisition DAP total from Gy.m2 to cGy.cm2 for display in web interface

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.AccumMammographyXRayDose(*args, **kwargs)

Accumulated Mammography X-Ray Dose TID 10005

From DICOM Part 16:
This modality specific template provides detailed information on mammography X-Ray dose value accumulations over several irradiation events from the same equipment (typically a study or a performed procedure step).
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.AccumCassetteBsdProjRadiogDose(*args, **kwargs)

Accumulated Cassette-based Projection Radiography Dose TID 10006

From DICOM Part 16 Correction Proposal CP-1077:
This template provides information on Projection Radiography dose values accumulated on Cassette- based systems over one or more irradiation events (typically a study or a performed procedure step) from the same equipment.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.AccumIntegratedProjRadiogDose(*args, **kwargs)

Accumulated Integrated Projection Radiography Dose TID 10007

From DICOM Part 16 Correction Proposal CP-1077:
This template provides information on Projection Radiography dose values accumulated on Integrated systems over one or more irradiation events (typically a study or a performed procedure step) from the same equipment.
convert_gym2_to_cgycm2()

Converts Gy.m2 to cGy.cm2 for display in web interface

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.PatientModuleAttr(*args, **kwargs)

Patient Module C.7.1.1

From DICOM Part 3: Information Object Definitions Table C.7-1:
Specifies the Attributes of the Patient that describe and identify the Patient who is the subject of a diagnostic Study. This Module contains Attributes of the patient that are needed for diagnostic interpretation of the Image and are common for all studies performed on the patient. It contains Attributes that are also included in the Patient Modules in Section C.2.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.PatientStudyModuleAttr(*args, **kwargs)

Patient Study Module C.7.2.2

From DICOM Part 3: Information Object Definitions Table C.7-4a:
Defines Attributes that provide information about the Patient at the time the Study started.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.GeneralEquipmentModuleAttr(*args, **kwargs)

General Equipment Module C.7.5.1

From DICOM Part 3: Information Object Definitions Table C.7-8:
Specifies the Attributes that identify and describe the piece of equipment that produced a Series of Composite Instances.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtRadiationDose(*args, **kwargs)

CT Radiation Dose TID 10011

From DICOM Part 16:
This template defines a container (the root) with subsidiary content items, each of which corresponds to a single CT X-Ray irradiation event entry. There is a defined recording observer (the system or person responsible for recording the log, generally the system). Accumulated values shall be kept for a whole Study or at least a part of a Study, if the Study is divided in the workflow of the examination, or a performed procedure step. Multiple CT Radiation Dose objects may be created for one Study.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.SourceOfCTDoseInformation(*args, **kwargs)

Source of CT Dose Information

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtAccumulatedDoseData(*args, **kwargs)

CT Accumulated Dose Data

From DICOM Part 16:
This general template provides detailed information on CT X-Ray dose value accumulations over several irradiation events from the same equipment and over the scope of accumulation specified for the report (typically a Study or a Performed Procedure Step).
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtIrradiationEventData(*args, **kwargs)

CT Irradiation Event Data TID 10013

From DICOM Part 16:
This template conveys the dose and equipment parameters of a single irradiation event.
Additional to the template:
  • date_time_started
  • series_description
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtReconstructionAlgorithm(*args, **kwargs)

Container in TID 10013 to hold CT reconstruction methods

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtXRaySourceParameters(*args, **kwargs)

Container in TID 10013 to hold CT x-ray source parameters

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ScanningLength(*args, **kwargs)

Scanning Length TID 10014

From DICOM Part 16:
No description
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.SizeSpecificDoseEstimation(*args, **kwargs)

Container in TID 10013 to hold size specific dose estimation details

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.WEDSeriesOrInstances(*args, **kwargs)

From TID 10013 Series or Instance used for Water Equivalent Diameter estimation

exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.CtDoseCheckDetails(*args, **kwargs)

CT Dose Check Details TID 10015

From DICOM Part 16:
This template records details related to the use of the NEMA Dose Check Standard (NEMA XR-25-2010).
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.ObserverContext(*args, **kwargs)

Observer Context TID 1002

From DICOM Part 16:
The observer (person or device) that created the Content Items to which this context applies.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.DeviceParticipant(*args, **kwargs)

Device Participant TID 1021

From DICOM Part 16:
This template describes a device participating in an activity as other than an observer or subject. E.g. for a dose report documenting an irradiating procedure, participants include the irradiating device.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.PersonParticipant(*args, **kwargs)

Person Participant TID 1020

From DICOM Part 16:
This template describes a person participating in an activity as other than an observer or subject. E.g. for a dose report documenting an irradiating procedure, participants include the person administering the irradiation and the person authorizing the irradiation.
exception DoesNotExist
exception MultipleObjectsReturned
class remapp.models.MedianSQL(col, source=None, is_summary=False, **extra)
class remapp.models.Median(*expressions, **extra)
sql

alias of MedianSQL

Filtering code

class remapp.interface.mod_filters.RFSummaryListFilter(data=None, queryset=None, prefix=None, strict=None)

Filter for fluoroscopy studies to display in web interface.

class remapp.interface.mod_filters.RFFilterPlusPid(*args, **kwargs)
class remapp.interface.mod_filters.CTSummaryListFilter(data=None, queryset=None, prefix=None, strict=None)

Filter for CT studies to display in web interface.

class remapp.interface.mod_filters.CTFilterPlusPid(*args, **kwargs)
class remapp.interface.mod_filters.MGSummaryListFilter(data=None, queryset=None, prefix=None, strict=None)

Filter for mammography studies to display in web interface.

class remapp.interface.mod_filters.MGFilterPlusPid(*args, **kwargs)
class remapp.interface.mod_filters.DXSummaryListFilter(data=None, queryset=None, prefix=None, strict=None)

Filter for DX studies to display in web interface.

class remapp.interface.mod_filters.DXFilterPlusPid(*args, **kwargs)

Views

Will be documented in future release

Export Views

remapp.exports.exportviews.include_pid(request, name, pat_id)

Check if user is allowed to export PID, then check if they have asked to. :param request: request so we can determine the user and therefore groups :param name: string, 0 or 1 from URL indicating if names should be exported :param pat_id: string, 0 or 1 from URL indicating if patient ID should be exported :return: dict, with pidgroup, include_names and include_pat_id as bools

remapp.exports.exportviews.ctcsv1(*args, **kwargs)

View to launch celery task to export CT studies to csv file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • pat_id – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.ctxlsx1(*args, **kwargs)

View to launch celery task to export CT studies to xlsx file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • pat_id – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.dxcsv1(*args, **kwargs)

View to launch celery task to export DX and CR studies to csv file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • pat_id – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.dxxlsx1(*args, **kwargs)

View to launch celery task to export DX and CR studies to xlsx file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • pat_id – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.flcsv1(*args, **kwargs)

View to launch celery task to export fluoroscopy studies to csv file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • patid – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.rfxlsx1(*args, **kwargs)

View to launch celery task to export fluoroscopy studies to xlsx file

Parameters:
  • request (GET) – Contains the database filtering parameters. Also used to get user group.
  • name – string, 0 or 1 from URL indicating if names should be exported
  • patid – string, 0 or 1 from URL indicating if patient ID should be exported
remapp.exports.exportviews.rfopenskin(*args, **kwargs)

Create csv export suitable for import to standalone openSkin :param request: request object :param pk: primary key of study in GeneralStudyModuleAttr table

remapp.exports.exportviews.mgcsv1(*args, **kwargs)

Launches export of mammo data to CSV :param request: Contains the database filtering parameters. Also used to get user group. :param name: string, 0 or 1 from URL indicating if names should be exported :param patid: string, 0 or 1 from URL indicating if patient ID should be exported :return:

remapp.exports.exportviews.mgxlsx1(*args, **kwargs)

Launches export of mammo data to xlsx :param request: Contains the database filtering parameters. Also used to get user group. :param name: string, 0 or 1 from URL indicating if names should be exported :param patid: string, 0 or 1 from URL indicating if patient ID should be exported :return:

remapp.exports.exportviews.mgnhsbsp(*args, **kwargs)

View to launch celery task to export mammography studies to csv file using a NHSBSP template

Parameters:request (GET) – Contains the database filtering parameters. Also used to get user group.
remapp.exports.exportviews.export(*args, **kwargs)

View to list current and completed exports to track progress, download and delete

Parameters:request – Used to get user group.
remapp.exports.exportviews.download(request, *args, **kwargs)

View to handle downloads of files from the server

Originally used for download of the export spreadsheets, now also used for downloading the patient size import logfiles.

Parameters:
  • request – Used to get user group.
  • task_id – ID of the export or logfile
remapp.exports.exportviews.deletefile(*args, **kwargs)

View to delete export files from the server

Parameters:request (POST) – Contains the task ID
remapp.exports.exportviews.export_abort(request, *args, **kwargs)

View to abort current export job

Parameters:request (POST) – Contains the task primary key
remapp.exports.exportviews.update_active(*args, **kwargs)

AJAX function to return active exports

Parameters:request – Request object
Returns:HTML table of active exports
remapp.exports.exportviews.update_error(*args, **kwargs)

AJAX function to return exports in error state

Parameters:request – Request object
Returns:HTML table of exports in error state
remapp.exports.exportviews.update_complete(*args, **kwargs)

AJAX function to return recently completed exports

Parameters:request – Request object, including pk of latest complete export at initial page load
Returns:HTML table of completed exports

Forms

class remapp.forms.SizeUploadForm(data=None, files=None, auto_id=u'id_%s', prefix=None, initial=None, error_class=<class 'django.forms.utils.ErrorList'>, label_suffix=None, empty_permitted=False)

Form for patient size csv file upload

class remapp.forms.SizeHeadersForm(my_choice=None, **kwargs)

Form for csv column header patient size imports through the web interface

class remapp.forms.DicomQueryForm(*args, **kwargs)

Form for launching DICOM Query

clean()

Validate the form data to clear modality selections if sr_only is selected. :return: Form with modalities _or_ sr_only selected

class date

date(year, month, day) –> date object

ctime()

Return ctime() style string.

fromordinal()

int -> date corresponding to a proleptic Gregorian ordinal.

fromtimestamp()

timestamp -> local date from a POSIX timestamp (like time.time()).

isocalendar()

Return a 3-tuple containing ISO year, week number, and weekday.

isoformat()

Return string in ISO 8601 format, YYYY-MM-DD.

isoweekday()

Return the day of the week represented by the date. Monday == 1 … Sunday == 7

replace()

Return date with new specified fields.

strftime()

format -> strftime() style string.

timetuple()

Return time tuple, compatible with time.localtime().

today()

Current date or datetime: same as self.__class__.fromtimestamp(time.time()).

toordinal()

Return proleptic Gregorian ordinal. January 1 of year 1 is day 1.

weekday()

Return the day of the week represented by the date. Monday == 0 … Sunday == 6

class remapp.forms.DicomDeleteSettingsForm(*args, **kwargs)

Form for configuring whether DICOM objects are stored or deleted once processed

class remapp.forms.DicomQRForm(*args, **kwargs)

Form for configuring remote Query Retrieve nodes

class remapp.forms.DicomStoreForm(*args, **kwargs)

Form for configuring local Store nodes

class remapp.forms.SkinDoseMapCalcSettingsForm(*args, **kwargs)

Form for configuring whether skin dose maps are shown / calculated

class remapp.forms.NotPatientNameForm(*args, **kwargs)

Form for configuring not-patient name patterns

class remapp.forms.NotPatientIDForm(*args, **kwargs)

Form for configuring not-patient ID patterns

DICOM networking modules

Query-retrieve module

Query function
(task)openrem.remapp.netdicom.qrscu.qrscu(qr_scp_pk=None, store_scp_pk=None, implicit=False, explicit=False, move=False, query_id=None, date_from=None, date_until=None, modalities=None, inc_sr=False, remove_duplicates=True, filters=None, get_toshiba_images=False, *args, **kwargs)

Query retrieve service class user function

Queries a pre-configured remote query retrieve service class provider for dose metric related objects, making use of the filter parameters provided. Can automatically trigger a c-move (retrieve) operation.

Parameters:
  • qr_scp_pk (int, optional) – Database ID/pk of the remote QR SCP (Default value = None)
  • store_scp_pk (int, optional) – Database ID/pk of the local store SCP (Default value = None)
  • implicit (bool, optional) – Prefer implicit transfer syntax (preference possibly not implemented) (Default value = False)
  • explicit (bool, optional) – Prefer explicit transfer syntax (preference possibly not implemented) (Default value = False)
  • move (bool, optional) – Automatically trigger move request when query is complete (Default value = False)
  • query_id (str, optional) – UID of query if generated by web interface (Default value = None)
  • date_from (str, optional) – Date to search from, format yyyy-mm-dd (Default value = None)
  • date_until (str, optional) – Date to search until, format yyyy-mm-dd (Default value = None)
  • modalities (list, optional) – Modalities to search for, options are CT, MG, DX and FL (Default value = None)
  • inc_sr (bool, optional) – Only include studies that only have structured reports in (unknown modality) (Default value = False)
  • remove_duplicates (bool, optional) – If True, studies that already exist in the database are removed from the query results (Default value = True)
  • filters (dictionary list, optional) – include and exclude lists for StationName and StudyDescription (Default value = None)
  • get_toshiba_images (bool, optional) – Whether to try to get Toshiba dose summary images
  • *args
  • **kwargs
Returns:

Series Instance UIDs are stored as rows in the database to be used by a move request. Move request is optionally triggered automatically.

Move function
(task)openrem.remapp.netdicom.qrscu.movescu(query_id)

C-Move request element of query-retrieve service class user :param query_id: UUID of query in the DicomQuery table :return: None

openrem_qr.py script

Query remote server and retrieve to OpenREM

usage: openrem_qr.py [-h] [-ct] [-mg] [-fl] [-dx] [-f yyyy-mm-dd]
                     [-t yyyy-mm-dd] [-e string] [-i string] [-sne string]
                     [-sni string] [-toshiba] [-sr] [-dup]
                     qr_id store_id
Positional Arguments
qr_id Database ID of the remote QR node
store_id Database ID of the local store node
Named Arguments
-ct

Query for CT studies. Do not use with -sr

Default: False

-mg

Query for mammography studies. Do not use with -sr

Default: False

-fl

Query for fluoroscopy studies. Do not use with -sr

Default: False

-dx

Query for planar X-ray studies. Do not use with -sr

Default: False

-f, --dfrom Date from, format yyyy-mm-dd
-t, --duntil Date until, format yyyy-mm-dd
-e, --desc_exclude
 Terms to exclude in study description, comma separated, quote whole string
-i, --desc_include
 Terms that must be included in study description, comma separated, quote whole string
-sne, --stationname_exclude
 Terms to exclude in station name, comma separated, quote whole string
-sni, --stationname_include
 Terms to include in station name, comma separated, quote whole string
-toshiba

Advanced: Attempt to retrieve CT dose summary objects and one image from each series

Default: False

-sr

Advanced: Use if store has RDSRs only, no images. Cannot be used with -ct, -mg, -fl, -dx

Default: False

-dup

Advanced: Retrieve duplicates (objects that have been processed before)

Default: False

NetDICOM common functions

openrem.remapp.netdicom.tools.echoscu(scp_pk=None, store_scp=False, qr_scp=False, *args, **kwargs)

Function to check if built-in Store SCP or remote Query-Retrieve SCP returns a DICOM echo :param scp_pk: Primary key if either Store or QR SCP in database :param store_scp: True if checking Store SCP :param qr_scp: True if checking QR SCP :param args: :param kwargs: :return: ‘AssocFail’, Success or ?

openrem.remapp.netdicom.tools.create_ae(aet, port=None, sop_scu=None, sop_scp=None, transfer_syntax=None)

Function to create an application entity :param aet: string, AE Title :param sop_classes: list of supported SOP classes from netdicom.SOPclass to override default set :param transfer_syntax: list of supported transfer syntax from dicom.UID to override default set :return: application entity object ready to be started

Adding new charts

To add a new chart several files need to be updated:

  • models.py
  • forms.py
  • views.py
  • xxfiltered.html
  • xxChartAjax.js
  • displaychartoptions.html

Where xx is one of ct, dx, mg or rf

The additions to the files add:

  • database fields in the user profile to control whether the new charts are plotted (models.py)
  • new options on the chart plotting forms (forms.py, displaychartoptions.html)
  • extra code to calculate the data for the new charts if they are switched on (views.py)
  • a section of html and JavaScript to contain the charts (xxfiltered.html)
  • a section of JavaScript to pass the data calculated by views.py to xxfiltered.html

The process is probably best illustrated with an example. What follows is a description of how to add a new chart that displays study workload for fluoroscopy, and a pie chart of study description frequency.

Additions to models.py

A field per chart needs to be added to the UserProfile section in models.py to control whether the new charts should be plotted. There is a section of this file that looks like the following:

plotCTAcquisitionMeanDLP = models.BooleanField(default=True)
plotCTAcquisitionMeanCTDI = models.BooleanField(default=True)
plotCTAcquisitionFreq = models.BooleanField(default=False)
plotCTStudyMeanDLP = models.BooleanField(default=True)
plotCTStudyFreq = models.BooleanField(default=False)
plotCTRequestMeanDLP = models.BooleanField(default=False)
plotCTRequestFreq = models.BooleanField(default=False)
plotCTStudyPerDayAndHour = models.BooleanField(default=False)
plotCTStudyMeanDLPOverTime = models.BooleanField(default=False)
plotCTStudyMeanDLPOverTimePeriod = models.CharField(max_length=6,
                                                    choices=TIME_PERIOD,
                                                    default=MONTHS)
plotCTInitialSortingChoice = models.CharField(max_length=4,
                                              choices=SORTING_CHOICES_CT,
                                              default=FREQ)

Two new lines needs to be added to this section, using appropriate names such as:

plotRFStudyPerDayAndHour = models.BooleanField(default=False)
plotRFStudyFreq = models.BooleanField(default=False)

Adding new fields to models.py requires that a database migration is carried out to add the fields to the database. This is done via the command line:

python manage.py makemigrations remapp
python manage.py migrate remapp

The first command should result in a response similar to:

Migrations for 'remapp':
  0004_auto_20160424_1116.py:
    - Add field plotRFAcquisitionCTDIOverTime to userprofile
    - Add field plotRFStudyFreq to userprofile

The second command should result in a response similar to:

Operations to perform:
  Apply all migrations: remapp
Running migrations:
  Rendering model states... DONE
  Applying remapp.0004_auto_20160424_1116... OK

That’s the end of the changes required in models.py

Additions to forms.py

Two additional lines need to be added to the XXChartOptionsForm and XXChartOptionsDisplayForm methods in forms.py, where XX is one of CT, DX, MG or RF.

For our new charts the following lines needs to be added to both RFChartOptionsForm and RFChartOptionsDisplayForm:

plotRFStudyPerDayAndHour = forms.BooleanField(label='Study workload', required=False)
plotRFStudyFreq = forms.BooleanField(label='Study frequency', required=False)

In addition, a new method needs to be added so that the RF chart options are shown when the user goes to Config -> Chart options:

class RFChartOptionsDisplayForm(forms.Form):
    plotRFStudyPerDayAndHour = forms.BooleanField(label='Study workload', required=False)
    plotRFStudyFreq = forms.BooleanField(label='Study frequency', required=False)

That’s the end of the changes required in forms.py

Additions to views.py

Four methods in this file need to be updated.

xx_summary_list_filter additions

Some additions need to be made to the xx_summary_list_filter method in views.py, where xx is one of ct, dx, mg or rf. As we’re adding new RF charts, we need to edit rf_summary_list_filter.

A section of this method examines the user’s chart plotting preferences. Code must be added to include the new chart in these checks. An abbreviated version of the section is shown below.

# Obtain the chart options from the request
chart_options_form = RFChartOptionsForm(request.GET)
# Check whether the form data is valid
if chart_options_form.is_valid():
    # Use the form data if the user clicked on the submit button
    if "submit" in request.GET:
        # process the data in form.cleaned_data as required
        user_profile.plotCharts = chart_options_form.cleaned_data['plotCharts']
        if median_available:
            user_profile.plotAverageChoice = chart_options_form.cleaned_data['plotMeanMedianOrBoth']
        user_profile.save()

    else:
        form_data = {'plotCharts': user_profile.plotCharts,
                     'plotMeanMedianOrBoth': user_profile.plotAverageChoice}
        chart_options_form = RFChartOptionsForm(form_data)

Two new lines needs to be inserted into the if and else sections for the new chart:

# Obtain the chart options from the request
chart_options_form = RFChartOptionsForm(request.GET)
# Check whether the form data is valid
if chart_options_form.is_valid():
    # Use the form data if the user clicked on the submit button
    if "submit" in request.GET:
        # process the data in form.cleaned_data as required
        user_profile.plotCharts = chart_options_form.cleaned_data['plotCharts']
        user_profile.plotRFStudyPerDayAndHour = chart_options_form.cleaned_data['plotRFStudyPerDayAndHour']
        user_profile.plotRFStudyFreq = chart_options_form.cleaned_data['plotRFStudyFreq']
        if median_available:
            user_profile.plotAverageChoice = chart_options_form.cleaned_data['plotMeanMedianOrBoth']
        user_profile.save()

    else:
        form_data = {'plotCharts': user_profile.plotCharts,
                     'plotRFStudyPerDayAndHour': user_profile.plotRFStudyPerDayAndHour,
                     'plotRFStudyFreq': user_profile.plotRFStudyFreq,
                     'plotMeanMedianOrBoth': user_profile.plotAverageChoice}
        chart_options_form = RFChartOptionsForm(form_data)
xx_summary_chart_data additions

The return_structure variable needs the new user_profile fields adding.

Before:

return_structure =\
    rf_plot_calculations(f, request_results, median_available, user_profile.plotAverageChoice,
                         user_profile.plotSeriesPerSystem, user_profile.plotHistogramBins,
                         user_profile.plotHistograms)

After:

return_structure =\
    rf_plot_calculations(f, request_results, median_available, user_profile.plotAverageChoice,
                         user_profile.plotSeriesPerSystem, user_profile.plotHistogramBins,
                         user_profile.plotRFStudyPerDayAndHour,  user_profile.plotRFStudyFreq,
                         user_profile.plotHistograms)
xx_plot_calculations additions

Two items needs to be added to this method’s parameters.

Before:

def rf_plot_calculations(f, request_results, median_available, plot_average_choice, plot_series_per_systems,
                         plot_histogram_bins, plot_histograms):

After:

def rf_plot_calculations(f, request_results, median_available, plot_average_choice, plot_series_per_systems,
                         plot_histogram_bins, plot_study_per_day_and_hour, plot_study_freq, plot_histograms):

Our new charts makes use of study_events (rather than acquisition_events or request_events). We therefore need to ensure that study_events are available if the user has chosen to show the new chart.

After additions:

if plot_study_per_day_and_hour:
    study_events = f.qs

We now need to add code that will calculate the data for the new charts. This uses one of the methods in the chart_functions.py file, located in the interface folder of the OpenREM project.

if plot_study_per_day_and_hour:
    result = workload_chart_data(study_events)
    return_structure['studiesPerHourInWeekdays'] = result['workload']

if plot_study_freq:
    result = average_chart_inc_histogram_data(study_events,
                                              'generalequipmentmoduleattr__unique_equipment_name_id__display_name',
                                              'study_description',
                                              'projectionxrayradiationdose__accumxraydose__accumintegratedprojradiogdose__dose_area_product_total',
                                              1000000,
                                              plot_study_dap, plot_study_freq,
                                              plot_series_per_systems, plot_average_choice,
                                              median_available, plot_histogram_bins,
                                              calculate_histograms=plot_histograms)

    return_structure['studySystemList'] = result['system_list']
    return_structure['studyNameList'] = result['series_names']
    return_structure['studySummary'] = result['summary']

The data in return_structure will now be available to the browser via JavaScript, and can be used to populate the charts themselves.

chart_options_view additions

The RF options form need to be imported

Before:

from remapp.forms import GeneralChartOptionsDisplayForm, DXChartOptionsDisplayForm, CTChartOptionsDisplayForm

After:

from remapp.forms import GeneralChartOptionsDisplayForm, DXChartOptionsDisplayForm, CTChartOptionsDisplayForm,\
    RFChartOptionsDisplayForm

The RF form items need to be included

Before (abbreviated):

if request.method == 'POST':
    general_form = GeneralChartOptionsDisplayForm(request.POST)
    ct_form = CTChartOptionsDisplayForm(request.POST)
    dx_form = DXChartOptionsDisplayForm(request.POST)
    if general_form.is_valid() and ct_form.is_valid() and dx_form.is_valid() and rf_form.is_valid():
        try:
            # See if the user has plot settings in userprofile
            user_profile = request.user.userprofile
        except:
            # Create a default userprofile for the user if one doesn't exist
            create_user_profile(sender=request.user, instance=request.user, created=True)
            user_profile = request.user.userprofile

        user_profile.plotCharts = general_form.cleaned_data['plotCharts']
        ...
        ...
        user_profile.plotHistogramBins = general_form.cleaned_data['plotHistogramBins']

        user_profile.plotCTAcquisitionMeanDLP = ct_form.cleaned_data['plotCTAcquisitionMeanDLP']
        ...
        ...
        user_profile.plotCTInitialSortingChoice = ct_form.cleaned_data['plotCTInitialSortingChoice']

        user_profile.plotDXAcquisitionMeanDAP = dx_form.cleaned_data['plotDXAcquisitionMeanDAP']
        ...
        ...
        user_profile.plotDXInitialSortingChoice = dx_form.cleaned_data['plotDXInitialSortingChoice']

        user_profile.save()

    messages.success(request, "Chart options have been updated")

...
...

general_form_data = {'plotCharts': user_profile.plotCharts,
                     'plotMeanMedianOrBoth': user_profile.plotAverageChoice,
                     'plotInitialSortingDirection': user_profile.plotInitialSortingDirection,
                     'plotSeriesPerSystem': user_profile.plotSeriesPerSystem,
                     'plotHistogramBins': user_profile.plotHistogramBins}

ct_form_data = {'plotCTAcquisitionMeanDLP': user_profile.plotCTAcquisitionMeanDLP,
                ...
                ...
                'plotCTInitialSortingChoice': user_profile.plotCTInitialSortingChoice}

dx_form_data = {'plotDXAcquisitionMeanDAP': user_profile.plotDXAcquisitionMeanDAP,
                ...
                ...
                'plotDXInitialSortingChoice': user_profile.plotDXInitialSortingChoice}


general_chart_options_form = GeneralChartOptionsDisplayForm(general_form_data)
ct_chart_options_form = CTChartOptionsDisplayForm(ct_form_data)
dx_chart_options_form = DXChartOptionsDisplayForm(dx_form_data)

return_structure = {'admin': admin,
                    'GeneralChartOptionsForm': general_chart_options_form,
                    'CTChartOptionsForm': ct_chart_options_form,
                    'DXChartOptionsForm': dx_chart_options_form
                    }

After (abbreviated):

if request.method == 'POST':
    general_form = GeneralChartOptionsDisplayForm(request.POST)
    ct_form = CTChartOptionsDisplayForm(request.POST)
    dx_form = DXChartOptionsDisplayForm(request.POST)
    rf_form = RFChartOptionsDisplayForm(request.POST)
    if general_form.is_valid() and ct_form.is_valid() and dx_form.is_valid() and rf_form.is_valid():
        try:
            # See if the user has plot settings in userprofile
            user_profile = request.user.userprofile
        except:
            # Create a default userprofile for the user if one doesn't exist
            create_user_profile(sender=request.user, instance=request.user, created=True)
            user_profile = request.user.userprofile

        user_profile.plotCharts = general_form.cleaned_data['plotCharts']
        ...
        ...
        user_profile.plotHistogramBins = general_form.cleaned_data['plotHistogramBins']

        user_profile.plotCTAcquisitionMeanDLP = ct_form.cleaned_data['plotCTAcquisitionMeanDLP']
        ...
        ...
        user_profile.plotCTInitialSortingChoice = ct_form.cleaned_data['plotCTInitialSortingChoice']

        user_profile.plotDXAcquisitionMeanDAP = dx_form.cleaned_data['plotDXAcquisitionMeanDAP']
        ...
        ...
        user_profile.plotDXInitialSortingChoice = dx_form.cleaned_data['plotDXInitialSortingChoice']

        user_profile.plotRFStudyPerDayAndHour = rf_form.cleaned_data['plotRFStudyPerDayAndHour']
        user_profile.plotRFStudyFreq = rf_form.cleaned_data['plotRFStudyFreq']

        user_profile.save()

    messages.success(request, "Chart options have been updated")

...
...

general_form_data = {'plotCharts': user_profile.plotCharts,
                     ...
                     ...
                     'plotHistogramBins': user_profile.plotHistogramBins}

ct_form_data = {'plotCTAcquisitionMeanDLP': user_profile.plotCTAcquisitionMeanDLP,
                ...
                ...
                'plotCTInitialSortingChoice': user_profile.plotCTInitialSortingChoice}

dx_form_data = {'plotDXAcquisitionMeanDAP': user_profile.plotDXAcquisitionMeanDAP,
                ...
                ...
                'plotDXInitialSortingChoice': user_profile.plotDXInitialSortingChoice}

rf_form_data = {'plotDXStudyPerDayAndHour': user_profile.plotDXStudyPerDayAndHour,
                'plotRFStudyFreq': user_profile.plotRFStudyFreq}

general_chart_options_form = GeneralChartOptionsDisplayForm(general_form_data)
ct_chart_options_form = CTChartOptionsDisplayForm(ct_form_data)
dx_chart_options_form = DXChartOptionsDisplayForm(dx_form_data)
rf_chart_options_form = RFChartOptionsDisplayForm(rf_form_data)

return_structure = {'admin': admin,
                    'GeneralChartOptionsForm': general_chart_options_form,
                    'CTChartOptionsForm': ct_chart_options_form,
                    'DXChartOptionsForm': dx_chart_options_form,
                    'RFChartOptionsForm': rf_chart_options_form,
                    }

Additions to displaychartoptions.html

A new div needs to be added for the fluoroscopy chart options:

<div class="panel-heading">
  <h3 class="panel-title">Fluoroscopy chart options</h3>
</div>
<div class="panel-body">
  <table>
    {% csrf_token %}
    {{ RFChartOptionsForm }}
  </table>
  <input class="btn btn-default" name="submit" type="submit" />
</div>

Additions to rffiltered.html

A section of this file sets a JavaScript variable per chart. Two new ones needs to be added.

Additions:

{% if request.user.userprofile.plotRFStudyPerDayAndHour %}
    <script>
        var plotRFStudyPerDayAndHour = true;
        result = chartWorkload('piechartStudyWorkloadDIV', 'Studies');
    </script>
{% endif %}

{% if request.user.userprofile.plotRFStudyFreq %}
    <script>
        var plotRFStudyFreq = true;
        var urlStartStudy = '/openrem/rf/?{% for field in filter.form %}{% if field.name != 'study_description' and field.name != 'o' and field.value %}&{{ field.name }}={{ field.value }}{% endif %}{% endfor %}&study_description=';
        result = chartFrequency('piechartStudyDIV', 'Study description frequency');
    </script>
{% endif %}

A second section of code needs to be added to rffiltered.html to include a DIV for the new charts:

{% if request.user.userprofile.plotRFStudyPerDayAndHour %}
    <!-- HTML to include div container for study workload -->

    <script>
        $(window).resize(function() {
            chartSetExportSize('piechartStudyWorkloadDIV');
            fitChartToDiv('piechartStudyWorkloadDIV');
        });
    </script>

    <div class="panel-group" id="accordion5">
        <div class="panel panel-default">
            <div class="panel-heading">
                <h4 class="panel-title">
                    <a data-toggle="collapse" data-parent="#accordion5" href="#collapseStudyWorkloadPieChart" onclick="setTimeout(function() {$(document).resize();}, 0);">
                        Pie chart showing a breakdown of number of studies per weekday.
                    </a>
                </h4>
            </div>
            <div id="collapseStudyWorkloadPieChart" class="panel-collapse collapse">
                <div class="panel-body">
                    <div id="piechartStudyWorkloadDIV" style="height: auto; margin: 0 0"></div>
                    <p>Click on a segment to be taken to a pie chart showing the breakdown per hour for that weekday.</p>
                    <a onclick="enterFullScreen('collapseStudyWorkloadPieChart', 'piechartStudyWorkloadDIV')" class="btn btn-default btn-sm" role="button">Toggle fullscreen</a>
                </div>
            </div>
        </div>
    </div>
    <!-- End of HTML to include div container for studies per week day pie chart -->
{% endif %}

{% if request.user.userprofile.plotRFStudyFreq %}
    <!-- HTML to include div container for study name pie chart -->

    <script>
        $(window).resize(function() {
            chartSetExportSize('piechartStudyDIV');
            fitChartToDiv('piechartStudyDIV');
        });
    </script>

    <div class="panel-group" id="accordionPiechartStudy">
        <div class="panel panel-default">
            <div class="panel-heading">
                <h4 class="panel-title">
                    <a data-toggle="collapse" data-parent="#accordionPiechartStudy" href="#collapseStudyPieChart" onclick="setTimeout(function() {$(document).resize();}, 0);">
                        Pie chart showing a breakdown of study name frequency.
                    </a>
                </h4>
            </div>
            <div id="collapseStudyPieChart" class="panel-collapse collapse">
                <div class="panel-body">
                    <div id="piechartStudyDIV" style="height: auto; margin: 0 0"></div>
                    <a onclick="enterFullScreen('collapseStudyPieChart', 'piechartStudyDIV')" class="btn btn-default btn-sm" role="button">Toggle fullscreen</a>
                </div>
            </div>
        </div>
    </div>
    <!-- End of HTML to include div container for study name pie chart -->
{% endif %}

Additions to rfChartAjax.js

This file needs to update the skeleton chart with the data that has been provided by views.py. It does this via the appropriate routines contained in the chartUpdateData.js file. In this case, updateWorkloadChart and updateFrequencyChart:

// Study workload chart data
if(typeof plotRFStudyPerDayAndHour !== 'undefined') {
    updateWorkloadChart(json.studiesPerHourInWeekdays, 'piechartStudyWorkloadDIV', colour_scale);
}

// Study description frequency chart data start
if(typeof plotRFStudyFreq !== 'undefined') {
    updateFrequencyChart(json.studyNameList, json.studySystemList, json.studySummary, urlStartStudy, 'piechartStudyDIV', colour_scale);
}

That’s it - you should now have two new charts visible in the fluoroscopy filtered page.

Running the test suite

Preparation

Install the dependencies and OpenREM

OpenREM is a Django application, and therefore we use Django’s test-execution framework to test OpenREM.

The first thing to do is to create a local copy of the git repository, then install all of OpenREM’s dependencies in a virtualenv.

You will need python, pip, git and virtualenv installed - see the links on the Pre-installation preparations docs for the latter, but you might try pip install virtualenv.

mkdir openremrepo
git clone https://bitbucket.org/openrem/openrem.git openremrepo

Now create the virtualenv:

mkdir veOpenREM
virtualenv veOpenREM
. veOpenREM/bin/activate  # Linux
veOpenREM\Scripts\activate  # Windows

At this stage there should be a (veOpenREM) prefix to our prompt telling us the virtualenv is activated.

Now install the dependencies:

pip install -e openremrepo/
pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

In the future it might be necessary to install numpy too for testing.

Configure OpenREM

Rename and configure openremproject/local_settings.py.example and openremproject/wsgi.py.example as per the Installing OpenREM docs.

Create a database following the same Installing OpenREM instructions.

Run the tests!

Making sure the virtualenv is activated, move to openremrepo/openrem and run:

python manage.py test remapp

All the tests that exit in openrem/remapp/tests/ will now be run.

Creating test versions of production systems

If you wish to create a duplicate install to test upgrades etc, refer to Restore the database and the preceding text regarding making backups.

Indices and tables

Previous Release Notes and Change Log

Version history change log

OpenREM version history

0.8.0 (2018-06-11)
  • #635 Documentation: added Orthanc as preferred third party DICOM Store service
  • #634 Documentation: updated docs for import and query-retrieve duplicates processing
  • #633 Charts: fixed issue where charts failed if bar chart series name was null
  • #632 DICOM: move requests for queries that don’t exist now fail gracefully
  • #631 Skin dose maps: bug fixed that prevented message from displaying on screen when skin dose map cannot be calculated
  • #630 Documentation: improved installation instructions
  • #628 Imports: fixed code for importing when there are duplicate DX or MG studies in the database
  • #626 DICOM: isolated the generate modalities in study function and added testing
  • #625 Imports: now using event level UIDs to process continued, cumulative and duplicate RDSRs
  • #624 Charts: removed filter link on number of events histogram as it was not functioning correctly
  • #623 Imports: changed name of Toshiba image based extractor routine
  • #621 Documentation: reversed install order of openrem and pynetdicom due to new pydicom release
  • #619 Documentation: added workaround for outdated dictionary issues
  • #618 DICOM: fixed image level query that prevented RDSRs from being found
  • #617 Imports: fixed issue with multi study exams crashing the Toshiba extractor
  • #616 Documentation: added information for pip download -d
  • #615 Exports: added Target Exposure Index and Deviation Index to radiographic exports
  • #614 Exports: handle error when study is deleted during sheet creation for exports
  • #613 Imports: fixed dual modality type imports after ‘dual’ designation from ref #580
  • #612 Imports: prevented crash when RDSR was imported with AcquisitionProtocol sequence with no TextValue
  • #610 DICOM: query-retrieve changed to work for duplicate RDSRs, ref #114
  • #609 Interface: fixed the feature that toggles the selection when clicking anywhere on a display name table row
  • #608 Interface: fixed the broken sorting of display name table
  • #603 Interface: fixed JavaScript error if there are any None values in fluoro detail irradiation type table
  • #602 Skin dose maps: fixed error when there are multiple kVp values for a single irradiation event
  • #599 Installation: postgres instructions now include note about differing security choices
  • #597 Skin dose maps: documented that using a production webserver the default timeout value must be increased
  • #596 Documentation: added docs for using Gunicorn and NGINX on linux
  • #594 Display: corrected display of dual-plane DAP and RP dose in RF filtered view
  • #593 Imports: properly handles MultiValue filter material tags and permits aluminium spelling
  • #592 Documentation: added docs for using IIS on Windows
  • #589 Exports: now handles zero studies and studies deleted during exports sensibly
  • #587 Documentation: added instructions for Linux users to rotate logs
  • #586 Documentation: updated exports and detailed how pulse level data is exported
  • #585 Documentation: added information about multiple cumulative RDSRs
  • #584 Import, Interface, Export: RDSR with pulse level data now function
  • #583 Documentation: added information about dual mode modalities and deleting all from an X-ray unit
  • #582 Celery: updated results backend as amqp deprecated and slow
  • #581 Import scripts: interpreter line now always first, functions imported specifically
  • #580 Imports and Interface: one modality creating both DX and RF can now be handled appropriately
  • #579 Imports: dummy values for Toshiba CT import function now in settings.py, log file config in docs
  • #578 Exports: fixed NHSBSP export that was excluding RDSR imported Hologic studies
  • #575 Exports: export page now updates using AJAX and has a select all button
  • #573 Exports: corrected and clarified exposure time and duration units, added number of pulses
  • #572 Interface: homepage now populates as AJAX to increase responsiveness
  • #570 Charts: simplified chart function code
  • #569 Charts: fixed frequency issue with mean averages selected
  • #568 Imports: missing DICOM date-time no longer causes an error
  • #567 Celery: fixed dual-namespace imports of tasks
  • #566 Interface: correctly show “assumed patient mass” in case of set value of zero
  • #565 Interface: correctly handle dose area product with zero value
  • #564 Skin dose maps: text information on skin dose maps now embedded when saving the 2d or 3d map as a graphic
  • #562 Skin dose maps: error message on calculation failure now more explicit
  • #561 Imports: patient orientation modifier now correctly extracted from RDSR
  • #560 Exports: added study level comments
  • #559 Interface: date pickers inconsistent start day fixed
  • #558 Skin dose maps: set defaults instead of crashing if kV, dose, table or tube/detector position are missing
  • #557 Skin dose maps: improved construction of patient orientation code
  • #556 Exports: DX exports where TotalNumberOfRadiographicFrames is not populated now export
  • #552 Documentation: documented extractor for older Toshiba CT scanners
  • #551 Documentation: added procedure for opening csv files in Excel with non-ASCII characters
  • #550 Documentation: added a note to describe exposure time and duration for fluoroscopy studies
  • #549 Documentation: added procedure for fixing laterality on Hologic studies, ref #411
  • #547 Interface: improved handling of available time information for fluoro studies
  • #546 Query Retrieve: added flag and functionality to query for Toshiba images
  • #544 Interface: added procedure, requested procedure to summary listings and details and filtering
  • #543 Interface: added drop-down box to choose how many studies are displayed on filtered pages
  • #542 Interface: added display name to all detailed html pages
  • #541 Documentation: updated for celery on Windows
  • #540 Documentation: updated for current skinDose functionality
  • #539 Documentation: updated chart document to include series toggle buttons
  • #537 Charts: hide series function added
  • #536 Code quality: reduced javascript duplication and collected file groups into subfolders
  • #535 Interface: fixed problem where category names that included a plus symbol caused filtering and chart issues
  • #534 Interface: chart drilldown reported as not working - was actually due to a user’s database migrations
  • #533 Query Retrieve: Reduced number of simultaneous associations to one, reused for everything
  • #532 DICOM: documented how to work-around missing encoding charsets due to old pydicom
  • #529 Charts: added CT charts of number of irradiation events per study description and requested procedure
  • #528 Query Retrieve: reduced number of simultaneous associations to one, reused for everything
  • #526 Code quality: addressed some of the code quality/style issues raised by Codacy
  • #525 Importing: improved mammo import by checking compression force before converting to float
  • #524 Importing: improved mammo import by checking anode exists before converting to DICOM terms
  • #523 Importing: changed mammo import to use del_no_match instead of del_mg_im if not mammo
  • #522 Documentation: made it clearer on offline-install docs that version numbers will change
  • #521 Testing: added tests for dual source CT imports
  • #520 Imports: removed XML styling from Philips legacy CT comment creation
  • #519 Skin dose maps: fixed black on black text issue
  • #518 Importing: fixed imports where CT Target Region isn’t specified
  • #517 Interface: operator name is now displayed on the detail page for each modality, along with physician for CT and fluoro
  • #516 Imports: MultiValue person names are now stored as a decoded string, not a list
  • #511 Testing: develop and other branches can now be deployed to dev.openrem.org and testing.openrem.org automatically
  • #510 Imports: ‘not-patient-indicators’ can now be configured in the interface
  • #509 Skin dose maps: now recalculated on view if recorded height or weight has changed since last calculation
  • #508 Testing: DX sample files are now tested
  • #507 Interface: Mammo now filterable by study description, procedure, requested procedure and acquisition protocol
  • #506 Documentation: updated query-retrieve docs
  • #505 Charts: n is now displayed on charts
  • #504 Charts: Fixed issue with null values
  • #503 Internationalisation: more robust decoding and use of unicode throughout
  • #502 Testing: tests now work with SQLite3 and PostgreSQL databases
  • #501 Imports: Changed field type for CodeValue from 16 chars to text, allows for illegal long values
  • #500 Imports: Philips SC Dose Info with missing time stamps now import
  • #499 Imports: Now aborts gracefully with error log if no template in RDSR
  • #498 Exports: Missing units added to header fields
  • #497 Interface: Detailed fluoro study view: added irradiation type, pulse rate, dose to ref. point, secondary angle, total DAP and ref. point dose from each irradition type
  • #495 Charts: Reduced time taken to render scatter plots with multiple series
  • #494 Charts: Charts now ignore blank and zero-value data when calculating mean, median and number of events
  • #493 Charts: Added user option to made chart categories all lower case
  • #492 Exports: Each view is now unique for NHSBSP mammo exports as required by the NCCPM database
  • #491 Imports, Interface and Exports: CT Dose Check alerts and notifications are now extracted, displayed and exported
  • #490 Exports: Response object included for messages - removed as now asynchronous
  • #489 Exports: NHSBSP mammo exports deals with all views, excludes biopsies and specimens
  • #488 Exports: All exports now include study time
  • #487 Imports: CT RDSR now imports ‘procedure context’ correctly
  • #486 Imports: CT RDSR now imports ‘NameOfPhysiciansReadingStudy’ correctly
  • #485 Imports: CT RDSR now imports ‘target region’ correctly
  • #484 Exports and Interface: Exports and interface page views are now more efficient and (much) faster
  • #482 Imports: DX extractor now extracts acquisition protocol, requested procedure name and study name for Fuji Go mobile; extracts acquisition protocol for Toshiba Radrex equipment; extracts requested procedure name from Carestream DRX-Revolution mobiles
  • #480 Imports: Code and instructions to create and import an RDSR from Toshiba CT dose summary images and studies
  • #476 Imports: Mixed latin-1 and UTF8 characters now imported, but need to be handled better if possible
  • #475 Query Retrieve: Made -sr a stand-alone option - it has a very niche use-case!
  • #474 Logging: Changing to DEBUG logging level in local_settings.py will now be respected
  • #473 Query Retrieve: Added tests
  • #472 Query Retrieve: Overhauled the query retrieve routines
  • #471 Internationalisation: added configuration and docs to set the timezone
  • #470 Query Retrieve: Optimised CT filtering
  • #468 Query Retrieve: Station names can now be used for filtering if returned
  • #467 Testing: Added tests for mammography RDSR imports
  • #466 Query Retrieve: RDSR now retrieved in preference to images for MG and DX/CR
  • #465 Added newer SSDE and water equivalent diameter fields to database
  • #464 Imports: DX RDSR now imported properly
  • #463 Imports: Properly checks that Enhanced SR are GE dose reports before importing
  • #460 Interface: Display names table now sortable
  • #458 Exports: Filter thicknesses are rounded to max 4 significant figures on export
  • #454 Exports: Mean filter thickness now reported in exports
  • #453 Imports: DX with min filter thickness greater than max have values switched on import
  • #452 Exports: Added CTDIw phantom size to CT exports
  • #451 Skin dose maps: fixed issue with filters being referenced before being defined
  • #450 Imports: DX imports with filter thickness of 0.00 are now recorded as such
  • #449 Exports: Fixed a bug that prevented fluoro exports if protocol names had non-ASCII characters
  • #448 Documentation: Added a diagram showing the relationship between the OpenREM system components
  • #447 Imports: Modified rdsr and ctdetail template to import and display data from Pixelmed generated Toshiba RDSR
  • #446 Import: Extract additional Philips private information for Allura Xper systems, create workaround for missing end angles for rotational acquisitions
  • #445 Interface: Added function for user to determine between DX and fluoro for ambiguous modalities
  • #444 Imports: DX systems that submit RDSRs that look like fluoro can now be reclassified using #445
  • #443 Exports: Accession number and ID are now exported to XLSX as text. Thanks to @LuukO
  • #442 Exports: Fixed RF exports with multiple filters, added tests. Thanks to @LuukO
  • #441 Charts: Fixed a bug that broke chart links containing non-ASCII characters
  • #440 Charts: Fixed a bug in sorting.js so that undefined strings are handled correctly
  • #439 Charts: Added controls for plotting a series per system and calculation histogram data to each filtered view
  • #438 Skin dose maps: skin dose maps successfully calculated from existing studies; indication of assumed or extracted data shown
  • #434 Internationalisation: added passing char_set throughout the extractor functions (since largely made redundant again!)
  • #432 Imports: RDSR import function now looks in comment field for patient_table_relationship data
  • #431 Imports: fixed DX imports with MultiValue filter values (Cu+Al) again!
  • #430 Exports: fixed DX exports with multiple filters again, added tests
  • #429 Charts: added new mammo scatter plots. Thanks to @rijkhorst
  • #427 Testing: added a large number of tests that are automatically run on commit to bitbucket
  • #414 Reduced use of JavaScript global variables and improved JavaScript objects
  • #411 Imports: fixed laterality and accumulated AGD failure for Hologic DBT proprietary projection images
  • #323 Documentation: code autodocumentation largely now working again
  • #318 Database management: Display names view can be used to review and delete all studies from one source
  • #114 Imports: Subsequent RDSRs of the same study will now replace existing study in database
  • #61 Skin dose maps: These have been re-enabled, and currently work for Siemens systems
0.7.4 (2016-10-17)
  • #436 Install: temporary fix blocking django-filter latest version that breaks OpenREM
  • #431 Imports: fixed DX imports with MultiValue filter values (Cu+Al)
  • #430 Exports: fixed DX exports with multiple filters (Cu + Al)
0.7.3 (2016-08-30)
  • #426 Charts: added css so that wide chart data tables are displayed above the filter form div
  • #425 Exports: fixed error with non-ASCII characters being exported to csv
  • #424 Charts: fixed error where png or svg export of chart would show incorrect x-axis labels
  • #423 Charts: fixed error where some chart plotting options were not updated after being changed by the user
  • #422 Charts: added a button below each chart to toggle the display of the data table
  • #421 Charts: fixed error where only some scatter plot data was being exported to csv or xls files
  • #420 Charts: fixed error where frequency pie charts were only showing data from the first system
  • #419 Interface: fixed error where “Cancel” was ignored when deleting study in Firefox browser
  • #418 Exports: fixed error when exporting fluoroscopy study with missing xray_filter_material
  • #416 Charts: improved efficiency of JavaScript
  • #415 Database: migration for 0.6 upgraded installs to fix acquisition_device_type failures
  • #413 Documentation: removed erroneous reference to store queue in stop celery command
  • #410 Charts: fixed display of bar charts containing only one data point
  • #408 Charts: Increased number of items that can be shown on some Highcharts plots
  • #407 Fixed issue where skin dose map data was not being calculated on import
  • #406 Replaced Math.log10 JavaScript function with alternative function to fix IE11 skin dose map error
  • #405 Altered multi-line cell links in filtered pages so they work with IE8
0.7.1 (2016-06-10)
  • #403 Now deals with PersonName fields with latin-1 extended characters correctly
  • #402 Skin dose map data pickle files saved using gzip compression to save space
  • #401 Updated skin dose map documentation to say it won’t be in this release
  • #400 Strings are encoded as UTF-8 before being hashed to prevent errors with non-ASCII characters
  • #399 Migration file brought up to date for 0.6 to 0.7 upgrades
  • #398 Skin exposure maps are now stored in folders (feature postponed for future release)
  • #397 Skin exposure maps no longer available until orientation errors are fixed
  • #396 Charts: zooming on bar charts of average value vs. category now works
  • #395 Docs: offline Windows install instructions created, plus offline upgrade instructions
  • #394 Charts: made charts resize to fit containing div when browser is resized
  • #392 Charts: normalised histogram tooltip now correctly reports frequency
  • #391 Basic troubleshooting is now documented
  • #390 Charts: mammography and fluoroscopy charts added
  • #389 Charts: series without a name are now plotted under the name of Blank rather than not being plotted at all
  • #387 Added laterality to mammography exports
  • #385 Fixed issue with non-ASCII letters in RDSR sequence TextValue fields
  • #384 Fluoro exports for OpenSkin only consider copper filters now
  • #383 Refreshed settings.py to django 1.8 including updating template settings and TEMPLATE_CONTEXT_PROCESSORS
  • #380 Tube current now extracted from Siemens Intevo RDSR despite non-conformance
  • #379 Exposure time now populated for fluoro if not supplied by RDSR
  • #378 The display name of multiple systems can now be updated together using a single new name
  • #376 Corrected an ill-advised model change
  • #374 CTDIw phantom size now displayed in CT detail view
  • #373 Charts in some releases used GT rather than greater than or equal to for start date, now fixed
  • #372 Mammography studies now record an accumulated AGD per breast. Existing joint accumulated AGD values won’t be changed. Ordering by Accumulated AGD now creates an entry per accumulated AGD, one per breast
  • #371 Mammo RDSR generates average mA where not recorded, mammo image populates mA
  • #370 Added study description to mammography export
  • #369 Bi-plane fluoroscopy studies now export correctly
  • #368 Mammo RDSR now imports correctly
  • #365 Tube filtration is now displayed in the RF detail view
  • #364 Philips Allura fluorscopy RDSRs now import correctly
  • #362 Display of RF where bi-plane RDSRs have been imported no longer crash the interface
  • #360 Charts: saving data from average data charts as csv or xls now includes frequency values
  • #359 Added missing ‘y’ to query retrieve command line help
  • #358 Charts: chart sorting links and instructions now hidden when viewing histograms
  • #357 Charts: button to return from histogram now displays the name of the main chart
  • #356 Charts: histogram normalise button appears for all appropriate charts
  • #355 Charts: sorting now works as expected for plots with a series per system
  • #352 Fixed CT xlsx exports that had complete study data in each series protocol sheet (from earlier beta)
  • #351 Charts: simplified chart JavaScript and Python code
  • #350 DICOM networking documented for use with 3rd party store and advanced use with native
  • #348 Study delete confirmation page now displays total DAP for DX or CR radiographic studies
  • #346 Charts: exporting a chart as an image no longer requires an internet connection
  • #345 CSV size imports in cm are now stored as m in the database. Interface display of size corrected.
  • #343 Charts: user can now specify number of histogram bins in the range of 2 to 40
  • #342 Charts: improved the colours used for plotting chart data
  • #340 Fixed store failure to save due to illegal values in Philips private tags, improved exception code
  • #339 Improved extraction of requested procedure information for radiographic studies
  • #338 Fix Kodak illegally using comma in filter thickness values
  • #335 DICOM Store keep_alive and echo_scu functions now log correctly
  • #334 Fixed issue with tasks needing to be explicitly named
  • #333 Fixed StoreSCP not starting in beta 11 error
  • #332 Charts: some charts can now be plotted with a series per x-ray system
  • #331 Keep_alive tasks are now discarded if not executed, so don’t pile up
  • #329 All existing logging is now done via the same log files
  • #328 Store SCP no longer uses Celery tasks
  • #327 Celery workers now only take one task at a time
  • #325 Charts: switching charts off now leaves the user on the same page, rather than going to the home page
  • #324 Charts: forced chart tooltip background to be opaque to make reading the text easier
  • #320 The week now begins on Monday rather than Sunday on date form fields
  • #316 Query retrieve function can now exclude and include based on strings entered
  • #315 Charts: made size of exported chart graphics follow the browser window size
  • #314 One version number declaration now used for distribute, docs and interface
  • #313 Replaced non-working function with code to extract SeriesDescription etc in query response message
  • #312 Display names are now grouped by modality
  • #311 Queries are deleted from database after a successful C-Move
  • #310 Series level QR feedback now presented. Any further would require improvements in pynetdicom
  • #309 StoreSCP now deals safely with incoming files with additional transfer syntax tag
  • #308 Secondary capture images that don’t have the manufacturer field no longer crash the StoreSCP function
  • #306 Charts: added a button to each chart to toggle full-screen display
  • #305 Added links to documentation throughout the web interface
  • #304 Date of birth is now included in all exports that have either patient name or ID included
  • #303 Fixed a typo in 0.6.0 documents relating to the storescp command
  • #302 Improved handling of Philips Dose Info objects when series information sequence has UN value representation
  • #301 Charts: fixed bug that could stop average kVp and mAs radiographic plots from working
  • #300 Calling AE Title for Query Retrieve SCU is now configured not hardcoded
  • #299 Hash of MultiValued DICOM elements now works
  • #298 Added ordering by accumulated AGD for mammographic studies
  • #297 Fixed ordering by Total DAP for radiographic studies
  • #296 StoreSCP now logs an error message and continues if incoming file has problems
  • #295 Charts: fixed bug that arose on non-PostgreSQL databases
  • #294 Harmonised time display between filter list and detail view, both to HH:mm
  • #292 Added keep-alive and auto-start to DICOM stores
  • #291 Charts: fixed issue with CTDI and DLP not showing correct drilldown data
  • #290 Added new tables and fields to migration file, uses #288 and median code from #241
  • #289 Crispy forms added into the requires file
  • #288 Added device name hashes to migration file
  • #286 Increased granularity of permission groups
  • #285 Tidied up Options and Admin menus
  • #284 Fixed DICOM Query that looped if SCP respected ModalitiesInStudy
  • #282 Missing javascript file required for IE8 and below added
  • #281 Added check to import function to prevent extract failure
  • #280 Fixed typo in mammography export
  • #279 Charts: Fixed issue with median CTDI series from appearing
  • #278 Charts: Fixed javascript namespace pollution that caused links to fail
  • #277 Overhaul of acquisition level filters to get tooltip generated filters to follow through to export
  • #276 Unique fields cannot have unlimited length in MySQL - replaced with hash
  • #274 Charts: Fixed legend display issue
  • #273 Charts: Added plots of average kVp and mAs over time for DX
  • #272 Tweak to display of exam description for DX
  • #271 Fixed DX import failure where AcquisitionDate or AcquisitionTime are None
  • #270 Django 1.8 Admin site has a ‘view site’ link. Pointed it back to OpenREM
  • #268 Improved population of procedure_code_meaning for DX imports
  • #266 DICOM C-Store script added back in - largely redundant with web interface
  • #265 DICOM Store and Query Retrieve services documented
  • #263 Settings for keeping or deleting files once processed moved to database and web interface
  • #262 Dealt with issue where two exposures from the same study would race on import
  • #260 Fixed issue where import and export jobs would get stuck behind StoreSCP task in queue
  • #259 Link to manage users added to Admin menu
  • #258 Fixed DX import error where manufacturer or model name was not provided
  • #257 Documentation update
  • #256 Fixed errors with non-ASCII characters in imports and query-retrieve
  • #255 Charts: Small y-axis values on histograms are more visible when viewing full-screen
  • #254 Charts: Simplified chart data processing in the templates
  • #253 Charts: AJAX used to make pages responsive with large datasets when charts enabled
  • #252 Fixed duplicate entries in DX filtered data for studies with multiple exposures
  • #248 Charts: can now be ordered by frequency or alphabetically
  • #247 Fixed incorrect reference to manufacturer_model_name
  • #246 Charts: Added median data for PostgreSQL users
  • #245 Fixed error in csv DX export
  • #244 Fixed issue where scripts wouldn’t function after upgrade to Django 1.8
  • #243 Added distance related data to DX exports
  • #242 Distance source to patient now extracted from DX images
  • #241 Charts: Median values can be plotted for PostgreSQL users
  • #240 Charts: Improved DAP over time calculations
  • #239 Configurable equipment names to fix multiple sources with the same station name
  • #237 Charts: Tidied up plot data calculations in views.py
  • #235 Added patient sex to each of the exports
  • #234 Charts: Fixed error with datetime combine
  • #232 Charts: on or off displayed on the home page
  • #231 Charts: made links from requested procedure frequency plot respect the other filters
  • #230 Fixed error in OperatorsName field in DICOM extraction
  • #229 Charts: Added chart of DLP per requested procedure
  • #223 Charts: speed improvement for weekday charts
  • #217 Charts: Further code optimisation to speed up calculation time
  • #207 DICOM QR SCU now available from web interface
  • #206 DICOM Store SCP configuration now available from web interface
  • #183 Added options to store patient name and ID, and options to hash name, ID and accession number
  • #171 Root URL now resolves so /openrem is not necessary
  • #151 Suspected non-patient studies can now be filtered out
  • #135 GE Senographe DS now correctly records compression force in Newtons for new imports
  • #120 Improved testing of data existing for exports
  • #118 Upgraded to Django 1.8
  • #70 User is returned to the filtered view after deleting a study
  • #61 Skin dose maps for fluoroscopy systems can now be calculated and displayed
0.6.2 (2016-01-27)
  • #347 Django-filter v0.12 has minimum Django version of 1.8, fixed OpenREM 0.6.2 to max django-filter 0.11
  • #341 Changed references to the OpenSkin repository for 0.6 series.
0.6.1 (2015-10-30)
  • #303 Corrected name of Store SCP command in docs
0.6.0 (2015-05-14)
  • #227 Fixed import of RDSRs from Toshiba Cath Labs
  • #226 Charts: Updated Highcharts code and partially fixed issues with CTDIvol and DLP combined chart
  • #225 Charts: Added link from mAs and kVp histograms to associated data
  • #224 Charts: Added link from CTDIvol histograms to associated data
  • #221 Charts: Fixed issue where filters at acquisition event level were not adequately restricting the chart data
  • #219 Charts: Fixed issue where some charts showed data beyond the current filter
  • #217 Charts: Code optimised to speed up calculation time
  • #216 Fixed typo that prevented import of RSDR when DICOM store settings not present
  • #215 Charts: Fixed x-axis labels for mean dose over time charts
  • #214 Charts: Improved consistency of axis labels
  • #213 Fixed admin menu not working
  • #212 Charts: Created off-switch for charts
  • #210 OpenSkin exports documented
  • #209 Charts: Fixed server error when CT plots switched off and filter form submitted
  • #208 Charts: Fixed blank chart plotting options when clicking on histogram tooltip link
  • #205 Charts: Fixed issue of histogram tooltip links to data not working
  • #204 Charts: Fixed issue of not being able to export with the charts features added
  • #203 Charts: Fixed display of HTML in plots issue
  • #202 Charts: Added mean CTDIvol to charts
  • #200 Charts: Now exclude Philips Ingenuity SPRs from plots
  • #196 Added comments and entrance exposure data to DX export
  • #195 Fixed error with no users on fresh install
  • #194 Added more robust extraction of series description from DX
  • #193 Charts: Fixed reset of filters when moving between pages
  • #192 Created RF export for OpenSkin
  • #191 Charts: Factored out the javascript from the filtered.html files
  • #190 Charts: Added time period configuration to dose over time plots
  • #189 Charts: Fixed plotting of mean doses over time when frequency not plotted
  • #187 Charts: Merged the charts work into the main develop branch
  • #186 Fixed duplicate data in DX exports
  • #179 Charts: Added kVp and mAs plots for DX
  • #177 Charts: Fixed issue with date ranges for DX mean dose over time charts
  • #176 Charts: Added link to filtered dataset from mean dose over time charts
  • #175 Charts: Allowed configuration of the time period for mean dose trend charts to improve performance
  • #174 Charts: Fixed number of decimal places for mean DLP values
  • #173 Charts: Fixed plot of mean DLP over time y-axis issue
  • #170 Charts: Added plot of mean dose over time
  • #169 Charts: Improved chart colours
  • #157 Charts: Added chart showing number of studies per day of the week, then hour in the day
  • #156 Charts: Fixed issue with some protocols not being displayed
  • #155 Charts: Added chart showing relative frequency of protocols and study types
  • #140 Charts: Added configuration options
  • #139 Charts: Link to filtered dataset from histogram chart
  • #138 Charts: Number of datapoints displayed on tooltip
  • #135 Mammography compression force now only divides by 10 if model contains senograph ds Change in behaviour
  • #133 Documented installation of NumPy, initially for charts
  • #41 Preview of DICOM Store SCP now available
  • #20 Modality sections are now suppressed until populated
0.5.1 (2015-03-12)
  • #184 Documentation for 0.5.1
  • #180 Rename all reverse lookups as a result of #62
  • #178 Added documentation regarding backing up and restoring PostgreSQL OpenREM databases
  • #172 Revert all changes made to database so #62 could take place first
  • #165 Extract height and weight from DX, height from RDSR, all if available
  • #161 Views and exports now look for accumulated data in the right table after changes in #159 and #160
  • #160 Created the data migration to move all the DX accumulated data from TID 10004 to TID 10007
  • #159 Modified the DX import to populate TID 10007 rather than TID 10004. RDSR RF already populates both
  • #158 Demo website created by DJ Platten: http://demo.openrem.org/openrem
  • #154 Various decimal fields are defined with too few decimal places - all have now been extended.
  • #153 Changed home page and modality pages to have whole row clickable and highlighted
  • #150 DJ Platten has added Conquest configuration information
  • #137 Carestream DX multiple filter thickness values in a DS VR now extracted correctly
  • #113 Fixed and improved recording of grid information for mammo and DX and RDSR import routines
  • #62 Refactored all model names to be less than 39 characters and be in CamelCase to allow database migrations and to come into line with PEP 8 naming conventions for classes.
0.5.0 (2014-11-19)
  • Pull request from DJ Platten: Improved display of DX data and improved export of DX data
  • #132 Fixed mammo export error that slipped in before the first beta
  • #130 Only creates ExposureInuAs from Exposure if Exposure exists now
  • #128 Updated some non-core documentation that didn’t have the new local_settings.py reference or the new openremproject folder name
  • #127 DX IOD studies with image view populated failed to export due to lack of conversion to string
  • #126 Documentation created for the radiographic functionality
  • #125 Fixes issue where Hologic tomo projection objects were dropped as they have the same event time as the 2D element
  • #123 Fixed issue where filters came through on export as lists rather than strings on some installs
  • #122 Exports of RF data should now be more useful when exporting to xlsx. Will need refinement in the future
  • #26 Extractors created for radiographic DICOM images. Contributed by DJ Platten
  • #25 Views and templates added for radiographic exposures - either from RDSRs or from images - see #26. Contributed by DJ Platten
  • #9 Import of *.dcm should now be available from Windows and Linux alike
0.4.3 (2014-10-01)
  • #119 Fixed issue where Celery didn’t work on Windows. Django project folder is now called openremproject instead of openrem
  • #117 Added Windows line endings to patient size import logs
  • #113 Fixed units spelling error in patient size import logs
  • #112 File system errors during imports and exports are now handled properly with tasks listed in error states on the summary pages
  • #111 Added abort function to patient size imports and study exports
  • #110 Converted exports to use the FileField handling for storage and access, plus modified folder structure.
  • #109 Added example MEDIA_ROOT path for Windows to the install docs
  • #108 Documented ownership issues between the webserver and Celery
  • #107 Documented process for upgrading to 0.4.2 before 0.4.3 for versions 0.3.9 or earlier
  • #106 Added the duration of export time to the exports table. Also added template formatting tag to convert seconds to natural time
  • #105 Fixed bug in Philips CT import where decimal.Decimal was not imported before being used in the age calculation
  • #104 Added documentation for the additional study export functions as a result of using Celery tasks in task #19 as well as documentation for the code
  • #103 Added documentation for using the web import of patient size information as well as the new code
  • #102 Improved handling of attempts to process patient size files that have been deleted for when users go back in the browser after the process is finished
  • #101 Set the security of the new patient size imports to prevent users below admin level from using it
  • #100 Logging information for patient size imports was being written to the database - changed to write to file
  • #99 Method for importing remapp from scripts and for setting the DJANGO_SETTINGS_MODULE made more robust so that it should work out of the box on Windows, debian derivatives and virtualenvs
  • #98 Versions 0.4.0 to 0.4.2 had a settings.py.new file to avoid overwriting settings files on upgrades; renaming this file was missing from the installation documentation for new installs
  • #97 Changed the name of the export views file from ajaxviews as ajax wasn’t used in the end
  • #96 Changed mammo and fluoro filters to use named fields to avoid needing to use the full database path
  • #93 Set the security of the new exports to prevent users below export level from creating or downloading exports
  • #92 Add NHSBSP specific mammography csv export from Jonathan Cole - with Celery
  • #91 Added documentation for Celery and RabbitMQ
  • #90 Added delete function for exports
  • #89 Added the Exports navigation item to all templates, limited to export or admin users
  • #88 Converted fluoroscopy objects to using the Celery task manager after starting with CT for #19
  • #87 Converted mammography objects to using the Celery task manager after starting with CT for #19
  • #86 Digital Breast Tomosynthesis systems have a projections object that for Hologic contains required dosimetry information
  • #85 Fix for bug introduced in #75 where adaption of ptsize import for procedure import broke ptsize imports
  • #74 ‘Time since last study’ is now correct when daylight saving time kicks in
  • #39 Debug mode now defaults to False
  • #21 Height and weight data can now be imported through forms in the web interface
  • #19 Exports are now sent to a task manager instead of locking up the web interface
Reopened issue
  • #9 Issue tracking import using *.dcm style wildcards reopened as Windows cmd.exe shell doesn’t do wildcard expansion, so this will need to be handled by OpenREM in a future version
0.4.2 (2014-04-15)
  • #83 Fix for bug introduced in #73 that prevents the import scripts from working.
0.4.1 (2014-04-15)
  • #82 Added instructions for adding users to the release notes
0.4.0 (2014-04-15)

Note

  • #80 Added docs for installing Apache with auto-start on Windows Server 2012. Contributed by JA Cole
  • #79 Updated README.rst instructions
  • #78 Moved upgrade documentation into the release notes page
  • #77 Removed docs builds from repository
  • #76 Fixed crash if exporting from development environment
  • #75 Fixed bug where requested procedure wasn’t being captured on one modality
  • #73 Made launch scripts and ptsizecsv2db more robust
  • #72 Moved the secret key into the local documentation and added instructions to change it to release notes and install instructions
  • #71 Added information about configuring users to the install documentation
  • #69 Added documentation about the new delete study function
  • #68 Now checks sequence code meaning and value exists before assigning them. Thanks to JA Cole
  • #67 Added ‘Contributing authors’ section of documentation
  • #66 Added ‘Release notes’ section of documentation, incuding this file
  • #65 Added new local_settings.py file for database settings and other local settings
  • #64 Fixed imports failing due to non-conforming strings that were too long
  • #63 The mammography import code stored the date of birth unnecessarily. Also now gets decimal_age from age field if necessary
  • #60 Removed extraneous colon from interface data field
  • #18 Studies can now be deleted from the web interface with the correct login
  • #16 Added user authentication with different levels of access
  • #9 Enable import of *.dcm
0.3.9 (2014-03-08)

Note

#51 includes changes to the database schema – make sure South is in use before upgrading. See http://docs.openrem.org/page/upgrade.html

  • #59 CSS stylesheet referenced particular fonts that are not in the distribution – references removed
  • #58 Export to xlsx more robust - limitation of 31 characters for sheet names now enforced
  • #57 Modified the docs slightly to include notice to convert to South before upgrading
  • #56 Corrected the mammography target and filter options added for issue #44
  • #53 Dates can now be selected from a date picker widget for filtering studies
  • #52 Split the date field into two so either, both or neither can be specified
  • #51 Remove import modifications from issue #28 and #43 now that exports are filtered in a better way after #48 and #49 changes.
  • #50 No longer necessary to apply a filter before exporting – docs changed to reflect this
  • #49 CSV exports changed to use the same filtering routine introduced for #48 to better handle missing attributes
  • #48 New feature – can now filter by patient age. Improved export to xlsx to better handle missing attributes
  • #47 Install was failing on pydicom – fixed upstream
0.3.8 (2014-03-05)
  • – File layout modified to conform to norms
  • #46 Updated documentation to reflect limited testing of mammo import on additional modalities
  • #45 mam.py was missing the licence header - fixed
  • #44 Added Tungsten, Silver and Aluminum to mammo target/filter strings to match – thanks to DJ Platten for strings
  • #43 Mammography and Philips CT import and export now more robust for images with missing information such as accession number and collimated field size
  • #42 Documentation updated to reflect #37
  • #37 Studies now sort by time and date
0.3.7 (2014-02-25)
  • #40 Restyled the filter section in the web interface and added a title to that section
  • #38 Column titles tidied up in Excel exports
  • #36 openrem_ptsizecsv output of log now depends on verbose flag
  • #35 Numbers no longer stored as text in Excel exports
0.3.6 (2014-02-24)
  • #34 Localised scripts that were on remote web servers in default Bootstrap code
  • #33 Documentation now exists for adding data via csv file
  • #24 Web interface has been upgraded to Bootstrap v3
  • #5 Web interface and export function now have some documentation with screenshots
0.3.5-rc2 (2014-02-17)
  • #32 Missing sys import bug prevented new patient size import from working
0.3.5 (2014-02-17)
  • – Prettified this document!
  • #31 Promoted patient size import from csv function to the scripts folder so it will install and can be called from the path
  • #30 Improved patient size import from csv to allow for arbitary column titles and study instance UID in addition to accession number.
  • #29 Corrected the docs URL in the readme
0.3.4-rc2 (2014-02-14)
  • #28 XLSX export crashed if any of the filter fields were missing. Now fills on import with ‘None’
  • #27 Use requested procedure description if requested procedure code description is missing
0.3.4 (2014-02-14)
  • – General improvements and addition of logo to docs
  • #23 Added Windows XP MySQL backup guide to docs
  • #22 Added running Conquest as a Windows XP service to docs
  • #15 Added version number and copyright information to xlsx exports
  • #14 Added version number to the web interface
  • #13 Improve the docs with respect to South database migrations
0.3.3-r2 (2014-02-04)
0.3.3 (2014-02-01)

Note

Installs of OpenREM earlier than 0.3.3 will break on upgrade if the scripts are called from other programs. For example openrem_rdsr is now called openrem_rdsr.py

  • – Added warning of upgrade breaking existing installs to docs
  • #10 Added .py suffix to the scripts to allow them to be executed on Windows (thanks to DJ Platten)
  • #8 Removed superfluous ‘/’ in base html file, harmless on linux, prevented Windows loading stylesheets (thanks to DJ Platten)
  • #7 Added windows and linux path examples for test SQLite database creation
  • #6 Corrected renaming of example files installation instruction (thanks to DJ Platten)
  • #4 Added some text to the documentation relating to importing files to OpenREM
  • #3 Corrected copyright notice in documentation
0.3.2 (2014-01-29)
  • Initial version uploaded to bitbucket.org

Release notes and upgrade instructions

Each release comes with specific upgrade instructions, so please follow the links below for the appropriate version.

Version specific information

OpenREM Release Notes version 0.7.4
Headline changes
  • Imports: DX images now import with multiple filters that are MultiValue as well as comma separated
  • Exports: DX data now correctly exports to csv and xlsx if studies include multiple filters (eg Cu+Al)
  • Install: New release of dependency django-filter breaks OpenREM. Pegged at previous version for now
Upgrading an OpenREM server with no internet access

Follow the instructions to OpenREM Release Notes version 0.7.1 and to OpenREM Release Notes version 0.7.3 first.

Then use the instructions found at Upgrade an offline OpenREM installation again, but this time change the pip commands from openrem==0.7.1 to openrem==0.7.4.

Upgrading from 0.6 series

Follow the instructions to OpenREM Release Notes version 0.7.1 and to OpenREM Release Notes version 0.7.3 first.

Finally, return to these instructions to upgrade to 0.7.4.

Upgrading from version 0.7.1

Follow the instructions to OpenREM Release Notes version 0.7.3 first, then return to these instructions to upgrade to 0.7.4.

Upgrading from version 0.7.3
  • Back up your database

    • For PostgreSQL you can refer to Backup the database
    • For a non-production SQLite3 database, simply make a copy of the database file
  • Stop any Celery workers

  • If you are using a virtualenv, activate it

  • Install the new version of OpenREM:

pip install openrem==0.7.4

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\
Check for any migrations
python manage.py makemigrations remapp

The expected response should is: No changes detected in app 'remapp'

Restart all the services

Follow the guide at Start all the services.

OpenREM Release Notes version 0.7.3
Headline changes
  • Database: New migration file for upgrades from 0.6 series databases
  • Charts: Fixed display and export errors, improved layout and increased the number of data points that can be plotted
  • Interface: Fixed multi-line cells in tables so that the links work in IE8
  • Interface: Fixed delete cancel button in firefox
  • Exports: Fixed export of non-ASCII characters to csv file
Upgrading an OpenREM server with no internet access

Upgrade using the instructions found at Upgrade an offline OpenREM installation, but change the pip commands from openrem==0.7.1 to openrem==0.7.3. If you are still on a 0.6 series install, upgrade to 0.7.1 first.

Upgrading from version 0.7.1
  • Back up your database

    • For PostgreSQL you can refer to Backup the database
    • For a non-production SQLite3 database, simply make a copy of the database file
  • Stop any Celery workers

  • If you are using a virtualenv, activate it

  • Install the new version of OpenREM:

pip install openrem==0.7.3

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\
Check the current status of your migrations
python manage.py showmigrations

If you are using the PostgreSQL database and installed 0.7.1 as a fresh install, the remapp section should look like this:

remapp
 [X] 0001_initial
 [X] 0002_0_7_fresh_install_add_median

If you installed 0.7.1 as a fresh install and are using a different database – such as MySQL or the built-in test database SQLite3 – the remapp section should look like this:

remapp
 [X] 0001_initial

For both of these scenarios your upgrade is complete and you can Start all the services.

If you have an installation that has been upgraded from the 0.6 series, it should have a remapp section that looks like this:

remapp
 [X] 0001_initial
 [X] 0002_upgrade_0_7_from_0_6

For this scenario, please continue and apply the new migration using the instructions below.

If your migrations list is different from these, particularly if there are any migrations listed with an empty [ ] check box and you don’t know why, please ask a question on the Google group before continuing. Don’t forget to tell us what is in the remapp section of your showmigrations listing and what upgrades you have done so far.

Apply the new migration

Rename the file

remapp/migrations/000x_delete_060_acq_field.py.inactive

to:

remapp/migrations/000x_delete_060_acq_field.py

Check that the rename was successful by running python manage.py showmigrations again. The new migration should be listed with an empty pair of square brackets.

Now run

python manage.py migrate remapp

This should result in an error similar to this:

CommandError: Conflicting migrations detected (0002_upgrade_0_7_from_0_6, 000x_delete_060_acq_field in remapp).
To fix them run 'python manage.py makemigrations --merge'

Now run

python manage.py makemigrations --merge

This will then list the merge actions, finishing with the following text:

Merging will only work if the operations printed above do not conflict
with each other (working on different fields or models)
Do you want to merge these migration branches? [y/N]

Respond with a y, then run python manage.py showmigrations again. This should result in the following listing:

remapp
 [X] 0001_initial
 [ ] 000x_delete_060_acq_field
 [X] 0002_upgrade_0_7_from_0_6
 [ ] 0003_merge

Now run the migration:

python manage.py migrate remapp

A final python manage.py showmigrations should show:

remapp
 [X] 0001_initial
 [X] 000x_delete_060_acq_field
 [X] 0002_upgrade_0_7_from_0_6
 [X] 0003_merge
Restart all the services

Follow the guide at Start all the services.

Import all the failed studies since 0.6 series upgrade

Re-import any fluoroscopy, radiography or mammography data that has not imported since the upgrade from the 0.6 series. This relates to issue #415 on the Bitbucket issue tracker.

If you have any studies complaining

remapp.models.DoesNotExist: ProjectionXRayRadiationDose matching query does not exist.

You should check to see if the study you are importing has been partially imported before the database was fixed. If it has, you might need to delete it using the delete function in the web interface. You will only see the delete function if you have admin privileges - see Configure the settings for details.

Upgrading from 0.6 series

Follow the instructions to OpenREM Release Notes version 0.7.1 first, then return to these instructions to upgrade to 0.7.3.

OpenREM Release Notes version 0.7.1
Headline changes
  • System

    • Django upgraded to version 1.8
    • Median function added to the database if using PostgreSQL
    • New user-defined display name for each unique system so that rooms with the same DICOM station name are displayed separately
    • Patient name and ID can optionally be stored in system, available for searching and export, but not displayed
    • Patient name, ID and accession number can be stored as a one-way hash, and remain searchable
    • Permission system has become more granular
    • System can now accept non-ASCII characters in protocol names etc
    • Menus have been tidied up
    • Settings file has been updated
  • Charts and interface

    • Bar chart data points sorted by frequency, value or name in ascending or descending order
    • CT chart of DLP per requested procedure type
    • CT chart of requested procedure frequency
    • CT chart of CTDIvol per study description
    • Chart data returned using AJAX to make pages more responsive
    • Chart plotting options available via Config menu
    • Charts can now be made full-screen
    • CTDIw phantom size is displayed with the CTDIvol measurement on the CT study detail page
    • Charts show a series called “Blank” when the series name is None
    • Queries for chart data now faster in most situations
    • Histograms can be disabled or enabled for bar charts
    • User-specified number of histogram bins from 2 to 40
    • Mammography chart of average glandular dose vs. compressed thickness
    • Mammography chart showing the number of studies carried out per weekday
    • Fluoroscopy chart of average DAP for each study description
    • Fluoroscopy chart of the frequency of each study description
    • Fluoroscopy chart showing the number of studies carried out per weekday
    • Context specific documentation has been added to the Docs menu
  • DICOM Networking

    • Query retrieve function is now built in to query PACS systems or modalities via the Import menu
    • Configuring and running DICOM Store SCP is available and managed in the web interface, but not recommended
    • Documentation improved
  • Imports

    • Mammography RDSRs import correctly
    • Mammography imports from images now create an accumulated AGD value per breast
    • GE Senographe DS compression now recorded correctly in Newtons for new imports
    • Philips Allura fluoroscopy RDSRs import correctly, including calculating the exposure time
    • Bi-plane fluoroscopy imports can now be displayed in the web interface
    • Patient height imports from csv now convert from cm to m - previously height was assumed to be cm and inserted into database without change. Existing height data will remain as cm value for csv imports, and m value for RDSR imports
    • Better handling of non-ASCII characters
    • Tube current is now extracted from Siemens Intevo RDSRs
  • Exports

    • Patient sex is included in all exports
    • Filters generated by navigating through charts can now be used to filter export data
    • Study description and laterality are now included in mammography exports
    • Bi-fluoroscopy studies can be exported
  • Skin dose maps

    • Skin dose maps have been withdrawn from OpenREM version 0.7.0 due to incorrect orientation calculations that need to be fixed before openSkin can be reimplemented into OpenREM
Changes since 0.7.0

Extremely minor change to the documenation links

Upgrading an OpenREM server with no internet access

Follow the instructions found at Upgrade an offline OpenREM installation, before returning here to update the database and configuration.

Upgrading from version 0.6.0
  • Back up your database

    • For PostgreSQL you can refer to Backup the database
    • For a non-production SQLite3 database, simply make a copy of the database file
  • Stop any Celery workers

  • The 0.7.0 upgrade must be made from a 0.6.0 (or later) database, and a schema migration is required:

pip install openrem==0.7.1

In a shell/command window, move into the openrem folder:

  • Ubuntu linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Other linux: /usr/lib/python2.7/site-packages/openrem/
  • Linux virtualenv: vitualenvfolder/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
  • Windows virtualenv: virtualenvfolder\Lib\site-packages\openrem\

Delete all numbered migration files in openrem’s remapp/migrations folder, leaving the 0002 files ending in .inactive

  • If there is no file named __init__.py in the remapp/migrations folder, please create it.
  • If you have accidentally deleted the 0002 files ending in .inactive, you can get a new copy from the bitbucket repository.
python manage.py migrate --fake-initial
python manage.py makemigrations remapp
python manage.py migrate remapp --fake

Now rename the file

remapp/migrations/0002_upgrade_0_7_from_0_6.py.inactive

to:

remapp/migrations/0002_upgrade_0_7_from_0_6.py

and then run

python manage.py migrate remapp

Note

With a large database, this may take some time!

  • Review the new openremproject/local_settings.py.example file and copy accross the logging section. Then see Log file settings in the install docs.

    If you are using PuTTY on Windows to interact with a linux server, you can select the logging configuration section of the example file with your mouse, and it will be automatically copied to the clipboard. Then open the existing local_settings.py file with nano, move the curser down to the bottom and click the right mouse button to paste.

Restart all the services!

Some of the commands and services have changed - follow the guide at Start all the services.

Upgrading from version 0.7.0 beta 7 or later
  • Stop any Celery workers
  • You will need to do a database migration.
pip install openrem==0.7.1

From the openrem folder (see above):

python manage.py makemigrations remapp
python manage.py migrate remapp
  • Review the new local_settings.py.example file and copy accross the logging section. Then see Log file settings in the install docs.
Restart all the services!

Some of the commands and services have changed - follow the guide at Start all the services.

OpenREM Release Notes version 0.6.0
Headline changes
  • Charts
  • Preview of DICOM Store SCP functionality
  • Exports available to import into openSkin
  • Modalities with no data are hidden in the user interface
  • Mammography import compression force behaviour changed
  • Import of Toshiba planar RDSRs fixed
Changes for 0.6.2

Minor update due prevent new installs from installing a non-compatible version of django-filter. The link to openSkin has also been updated in the fluoroscopy detail page.

There is no advantage to updating to this version over 0.6.0

Release 0.6.1 was just a documentation only change to update the link to openSkin.

Preparing for the upgrade
Convert to South

Make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py convert_to_south remapp

# Windows, assuming no virtualenv
python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp
Additional installs

OpenREM requires two additional programs to be installed to enable the new features: Numpy for charts, and pynetdicom for the DICOM Store Service Class Provider. Note that the version of pynetdicom must be later than the current pypi release!

Install NumPy

For linux:

sudo apt-get install python-numpy
# If using a virtualenv, you might need to also do:
pip install numpy

For Windows, there are various options:

  1. Download executable install file from SourceForge:

    • Download a pre-compiled Win32 .exe NumPy file from http://sourceforge.net/projects/numpy/files/NumPy/. You need to download the file that matches the Python version, which should be 2.7. At the time of writing the latest version was 1.9.2, and the filename to download was numpy-1.9.2-win32-superpack-python2.7.exe. The filename is truncated on SourceForge, so you may need to click on the i icon to see which is which. It’s usually the third superpack.
    • Run the downloaded binary file to install NumPy.
  2. Or download a pip installable wheel file:

    • Download NumPy from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy - numpy‑1.9.2+mkl‑cp27‑none‑win32.whl is likely to be the right version, unless you have 64bit Python installed, in which case use the numpy‑1.9.2+mkl‑cp27‑none‑win_amd64.whl version instead.

    • Install using pip:

      pip install numpy‑1.9.2+mkl‑cp27‑none‑win32.whl
      
Install pynetdicom
pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2
Upgrading from versions prior to 0.5.1

You must upgrade to 0.5.1 first. Instructions for doing this can be found in the OpenREM Release Notes version 0.5.1.

Upgrading from version 0.5.1
  • Back up your database

  • The 0.6.0 upgrade must be made from a 0.5.1 (or later) database, and a schema migration is required:

pip install openrem==0.6.0

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
# Windows:
python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  • Restart the services

    • Restart the webserver
    • Restart Celery
Summary of new features
Charts

Release 0.6.0 has a range of charting options available for CT and radiographic data. These charts allow visualisation of trends and frequencies to inform surveys and monitor performance. For more information, please see Charts.

DICOM Store Service Class Provider

OpenREM can now act as the DICOM Store service, allowing direct sending of DICOM objects from modalities to OpenREM without needing to use Conquest or any other DICOM Store SCP. This feature is a preview as it hasn’t been extensively tested, but it is expected to work. For more information, please see Direct from modalities.

Exports for openSkin

Fluoroscopy studies can now be exported in a format suitable for importing into Jonathan Cole’s openSkin software. The export link is on the fluoroscopy study detail page. The software for creating the exposure incidence map can be downloaded from https://bitbucket.org/openskin/openskin/downloads (choose the zip file), and information about the project can be found on the openSkin wiki. The software allows the user to choose between a 2D phantom that would represent the dose to a film laying on the couch surface, or a simple 3D phantom made up of a cuboid and two semi-cylinders (these can be seen on the Phantom design section of the wiki). For both options the output is an image of the dose distribution in 2D, along with calculated peak skin dose information.

Automatic hiding of unused modality types

A fresh install of OpenREM will no longer show any of the four modality types in the tables or in the navigation bar at the top. As DICOM objects are ingested, the appropriate tables and navigation links are created.

Therefore a site that has no mammography for example will no longer have that table or navigation link in their interface.

Mammography import compression force change

Prior to version 0.6, the compression force extracted from the mammography image header was divided by ten before being stored in the database. This was because the primary author only had access to GE Senograph DS units, which store the compression force in dN, despite claiming using Newtons in the DICOM conformance statement.

The code now checks for the term senograph ds contained in the model name. If it matches, then the value is divided by ten. Otherwise, the value is stored without any further change. We know that later GE units, the GE Senograph Essential for example, and other manufacturer’s units store this value in N. If you have a case that acts like the Senograph DS, please let us know and we’ll try and cater for that.

If you have existing non-GE Senograph mammography data in your database, the compression force field for those studies is likely to be incorrect by a factor of ten (it will be too small). Studies imported after the upgrade will be correct. If this is a problem for you, please let us know and we’ll see about writing a script to correct the existing data.

Import of Toshiba Planar RDSRs fixed

Toshiba include Patient Orientation and Patient Orientation Modifier information in their cath lab RDSRs. The extractor code was deficient for this as the RDSRs previously used didn’t have this information. This has now been fixed. There might however be an issue with Station Name not being provided - it is not yet clear if this is a configuration issue.

OpenREM Release Notes version 0.5.1
Headline changes
  • Major database modification to remove table name length errors
  • Extended the field value lengths to better incorporate all possible values and decimal places
  • Improved import of grid and filter information from DX images
  • Improved DX summary and detail web pages
  • Any item in a row can now be clicked to move between the home and filtered pages
Upgrades: Convert to South

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py convert_to_south remapp

# Windows, assuming no virtualenv
python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp
Upgrading from before 0.5.0
Upgrading from version 0.3.9 or earlier
  • Back up your database

  • pip install openrem==0.4.2

  • Migrate the schema

    # Linux: Debian/Ubuntu and derivatives
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
    # Windows:
    python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
    

    When South has considered the changes to the schema, you will see the following message:

    ? The field 'Observer_context.device_observer_name' does not have a default specified, yet is NOT NULL.
    ? Since you are making this field nullable, you MUST specify a default
    ? value to use for existing rows. Would you like to:
    ?  1. Quit now.
    ?  2. Specify a one-off value to use for existing columns now
    ?  3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
    ? Please select a choice: 3
    
    • As per the final line above, please select option 3, and then execute the migration:
    # Linux: Debian/Ubuntu and derivatives
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
    
    # Windows, assuming no virtualenv
    python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
    
  • Create and populate the database settings in the new local_settings.py file

    The openrem/openrem folder can be found at:

    # Linux: Debian/Ubuntu and derivatives
    /usr/lib/python2.7/dist-packages/openrem/openrem
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    /usr/lib/python2.7/site-packages/openrem/openrem
    # Windows:
    C:\Python27\Lib\site-packages\openrem\openrem
    

    In the openrem/openrem folder, create a new file called local_settings.py and copy the contents of this link into a the file and save it. Alternatively, rename local_settings.py.example to local_settings.py - this is an older version of the file.

    Copy the database details from settings.py into local_settings.py

  • Change the secret key - you can use http://www.miniwebtool.com/django-secret-key-generator/ to generate a new one

  • Move the existing settings.py out of the python directories (delete or move somewhere as a backup)

  • Rename the settings.py.new to settings.py

  • Restart your webserver to check everything looks ok

  • Add some users

    • Go to the admin interface (eg http://localhost:8000/admin) and log in with the user created when you originally created the database (the manage.py syncdb command - Do you want to create a superuser)

    • Create some users and add them to the appropriate groups (if there are no groups, go to the OpenREM homepage and they should be there when you go back to admin).

      • viewgroup can browse the data only
      • exportgroup can do as view group plus export data to a spreadsheet, and will be able to import height and weight data in due course (See Issue #21)
      • admingroup can delete studies in addition to anything the export group can do
Upgrading from versions 0.4.0 - 0.4.2
  • Back up your database

  • Install version 0.5.0

    • pip install openrem==0.5.0
  • Install RabbitMQ

  • Move local_settings.py details from openrem to openremproject

    The inner openrem Django project folder has now been renamed openremproject The customised local_settings.py file and the wsgi.py file have remain in the old openrem folder. The openrem/openrem folder can be found as detailed in the upgrade from ‘0.3.9 or earlier’ instructions above, and the new openrem/openremproject folder is in the same place.

    • Move local_settings.py to openremproject. If you have kept the older local_settings file, you may like to instead rename the local_settings.py.example file instead, then transfer the database settings and change the secret key.
    • Set the path for MEDIA_ROOT. The webserver needs to be able to write to this location - it is where OpenREM will store export files etc so that they can be downloaded. For suggestions, see the main _install instructions.
    • Set ALLOWED_HOSTS. For details see the Django docs A '*' allows any host - see the Django docs for the risk of this.
  • Move wsgi.py from openrem to openremproject or rename wsgi.py.example in openremproject

    If you haven’t edited it, simply rename the new version in openremproject. Otherwise, move the old version and edit the following line as follows:

    # Old:
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openrem.settings")
    # New:
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openremproject.settings")
    
  • Tidying up - you should delete the old openrem folder - you might like to take a backup first!

  • Update web server configuration

    The configuration of the webserver will need to be updated to reflect the new location for the settings.py file and the wsgi.py file.

    If you are using the built-in test webserver, static files will no-longer be served unless you use the insecure option:

    python manage.py runserver x.x.x.x:8000 --insecure
    
  • Migrate the schema

    # Linux: Debian/Ubuntu and derivatives
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
    # Windows:
    python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
    python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
    

After restarting the webserver, you should now have OpenREM 0.5.0 up and running. If you wish to test export functionality at this stage, start the Celery task queue - instructions in the Installing OpenREM docs or at the end of this guide.

Now move to Upgrading from version 0.5.0.

Upgrading from version 0.4.3
  • Back up your database

  • The 0.5.1 upgrade must be made from a 0.5.0 database, so a schema migration is required:

    pip install openrem==0.5.0
    
        # Linux: Debian/Ubuntu and derivatives
        python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
        python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
        # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
        python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
        python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
        # Windows:
        python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
        python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
    
Upgrading from version 0.5.0
  • Back up your database

  • Install 0.5.1:

    pip install openrem==0.5.1
    
  • Find out how many migration files you have

    Method 1:

    Use a file browser or terminal to list the contents of the migrations folder, eg:

    # Linux: Debian/Ubuntu and derivatives
    ls /usr/local/lib/python2.7/dist-packages/openrem/remapp/migrations/
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    ls /usr/local/lib/python2.7/site-packages/openrem/remapp/migrations/
    # Windows (alternatively use the file browser):
    dir C:\Python27\Lib\site-packages\openrem\remapp\migrations\
    

    Method 2:

    Use the Django manage.py program to list the existing migrations:

    # Linux: Debian/Ubuntu and derivatives
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate --list remapp
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate --list remapp
    # Windows
    python C:\Python27\Lib\site-packages\openrem\manage.py migrate --list remapp
    

    The output should look something like this - there can be any number of existing migrations (though 0001_initial will always be present):

    remapp
    (*) 0001_initial
    (*) 0002_auto__chg_field_ct_accumulated_dose_data_ct_dose_length_product_total_
    (*) 0003_auto__chg_field_general_equipment_module_attributes_station_name
    (*) 0004_auto__chg_field_ct_radiation_dose_comment__chg_field_accumulated_proje
    (*) 0005_auto__add_exports__add_size_upload
    (*) 0006_auto__chg_field_exports_filename
    (*) 0007_auto__add_field_irradiation_event_xray_detector_data_relative_xray_exp
    ( ) 000x_051datamigration
    ( ) 000x_051schemamigration
    
  • Rename the two 051 migration files to follow on from the existing migrations, for example 0008_051schemamigration.py and 0009_051datamigration.py for the existing migrations above, or 0002_051schemamigration.py and 0003_051datamigration.py if the only migration is the initial migration. The 051schemamigration must come before the 051datamigration.

    If you are using linux, you might like to do it like this (from within the openrem folder):

    mv remapp/migrations/000{x,8}_051schemamigration.py
    mv remapp/migrations/000{x,9}_051datamigration.py
    
  • If you now re-run migrate --list remapp you should get a listing with the 051schemamigration and the 051datamigration listed at the end:

    remapp
     (*) 0001_initial
     (*) 0002_auto__chg_field_ct_accumulated_dose_data_ct_dose_length_product_total_
     (*) 0003_auto__chg_field_general_equipment_module_attributes_station_name
     (*) 0004_auto__chg_field_ct_radiation_dose_comment__chg_field_accumulated_proje
     (*) 0005_auto__add_exports__add_size_upload
     (*) 0006_auto__chg_field_exports_filename
     (*) 0007_auto__add_field_irradiation_event_xray_detector_data_relative_xray_exp
     ( ) 0008_051schemamigration
     ( ) 0009_051datamigration
    

    The star indicates that a migration has already been completed. If you have any that are not completed apart from the 051schemamigration and the 051datamigration then please resolve these first.

  • Now execute the migrations:

    # Linux: Debian/Ubuntu and derivatives
    python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
    # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
    python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
    # Windows
    python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
    
Restart the web server

If you are using the built-in test web server (not for production use):

python manage.py runserver x.x.x.x:8000 --insecure

Otherwise restart using the command for your web server

Restart the Celery task queue

For testing, in a new shell:

# Linux: Debian/Ubuntu and derivatives
cd /usr/local/lib/python2.7/dist-packages/openrem/
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
cd /usr/local/lib/python2.7/site-packages/openrem/
# Windows
cd C:\Python27\Lib\site-packages\openrem\

# All
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.5.0
Headline changes
  • Import, display and export of CR/DX data from image headers
  • Export of study data from fluoroscopy to xlsx files
  • Importing data from Windows using *.dcm style wildcards
  • Hologic tomography projection images are no longer excluded if part of a Combo exposure
Specific upgrade instructions

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already:

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp
Upgrading from versions before 0.4.3

If you are upgrading from 0.3.9 or earlier, you will need to upgrade to version 0.4.2 first. See the OpenREM Release Notes version 0.4.3.

If you are upgrading from 0.4.0 or later, the instructions in OpenREM Release Notes version 0.4.3 still need to be followed to install/setup RabbitMQ and Celery and to update the configuration files, but you can go straight to 0.5.0 rather than installing 0.4.3.

Upgrading from version 0.4.3
pip install openrem==0.5.0

(Will need sudo or equivalent if using linux without a virtualenv)

Database migration

Assuming no virtualenv

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp

Windows:

C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
Restart the web server

If you are using the built-in test web server (not for production use):

python manage.py runserver x.x.x.x:8000 --insecure

Otherwise restart using the command for your web server

Restart the Celery task queue

For testing, in a new shell: (assuming no virtualenv)

Linux:

cd /usr/local/lib/python2.7/dist-packages/openrem/
celery -A openremproject worker -l info

Windows:

cd C:\Python27\Lib\site-packages\openrem\
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.4.3
Headline changes
  • Export of study information is now handled by a task queue - no more export time-outs.
  • Patient size information in csv files can now be uploaded and imported via a web interface.
  • Proprietary projection image object created by Hologic tomography units can now be interrogated for details of the tomosynthesis exam.
  • Settings.py now ships with its proper name, this will overwrite important local settings if upgrade is from 0.3.9 or earlier.
  • Time since last study is no longer wrong just because of daylight saving time!
  • Django release set to 1.6; OpenREM isn’t ready for Django 1.7 yet
  • The inner openrem Django project folder is now called openremproject to avoid import conflicts with Celery on Windows
  • DEBUG mode now defaults to False
Specific upgrade instructions

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already:

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp
Upgrading from 0.3.9 or earlier

It is essential that you upgrade to at least 0.4.0 first, then upgrade to 0.4.3. Otherwise the settings file will be overwritten and you will lose your database settings. There is also a trickier than usual database migration and instructions for setting up users. Fresh installs should start with the latest version.

Upgrade to version 0.4.2

pip install openrem==0.4.2

(Will need sudo or equivalent if using linux without a virtualenv)

Then follow the instructions in OpenREM Release Notes version 0.4.0 from migrating the database onwards, before coming back to these instructions.

Upgrading from 0.4.0 or above
Install OpenREM version 0.4.3
pip install openrem==0.4.3

(Will need sudo or equivalent if using linux without a virtualenv)

RabbitMQ

The message broker RabbitMQ needs to be installed to enable the export and upload features

Move and edit local_settings.py file and wsgi.py files

The inner openrem Django project folder has now been renamed openremproject to avoid import confusion that prevented Celery working on Windows.

When you upgrade, the local_settings.py file and the wsgi.py file will remain in the old openrem folder. Both need to be moved across to the openremproject folder, and edited as below.

The new and old folders will be found in:

  • Linux: /usr/local/lib/python2.7/dist-packages/openrem/
  • Linux with virtualenv: /home/myname/openrem/lib/python2.7/site-packages/openrem/
  • Windows: C:\Python27\Lib\site-packages\openrem\
Edit the local_settings.py file

The MEDIA_ROOT path needs to be defined. This is the place where the study exports will be stored for download and where the patient size information csv files will be stored temporarily whilst they are bing processed.

The path set for MEDIA_ROOT is up to you, but the user that runs the webserver must have read/write access to the location specified because it is the webserver than reads and writes the files. In a debian linux, this is likely to be www-data for a production install. Remember to use forward slashes in the config file, even for Windows.

Linux example:

MEDIA_ROOT = "/var/openrem/media/"

Windows example:

MEDIA_ROOT = "C:/Users/myusername/Documents/OpenREM/media/"

The ALLOWED_HOSTS needs to be defined, as the DEBUG mode is now set to False. This needs to contain the server name or IP address that will be used in the URL in the web browser. For example:

ALLOWED_HOSTS = [
    '192.168.56.102',
    '.doseserver.',
    'localhost',
]

A dot before a hostname allows for subdomains (eg www.doseserver), a dot after a hostname allows for FQDNs (eg doseserver.ad.trust.nhs.uk). Alternatively, a single '*' allows any host, but removes the security the feature gives you.

Edit the wsgi.py file with the new project folder name

If you aren’t using the wsgi.py file as part of your webserver setup, you might like to simply rename the wsgi.py.example file in the openremproject folder.

If you are using it, edit the line:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openrem.settings")

to read:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openremproject.settings")
Tidying up

Finally, you should delete the old openrem folder - you might like to take a backup first!

Database migration

Assuming no virtualenv

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp

Windows:

C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
Web server

If you are using a production webserver, you will probably need to edit some of the configuration to reflect the change in location of settings.py, for example DJANGO_SETTINGS_MODULE = openremproject.settings, and then restart the web server. You may need to do something similar for the location of wsgi.py.

If you are using the built-in test web server (not for production use), then the static files will not be served unless you add --insecure to the command. This is one of the consequences of setting DEBUG to False:

python manage.py runserver x.x.x.x:8000 --insecure
Start the Celery task queue

Note

The webserver and Celery both need to be able to read and write to the MEDIA_ROOT location. Therefore you might wish to consider starting Celery using the same user or group as the webserver, and setting the file permissions accordingly.

For testing, in a new shell: (assuming no virtualenv)

Linux:

cd /usr/local/lib/python2.7/dist-packages/openrem/
celery -A openremproject worker -l info

Windows:

cd C:\Python27\Lib\site-packages\openrem\
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.4.2
Headline changes
  • This release fixes a major bug introduced in 0.4.0 regarding the import scripts.
Specific upgrade instructions
Upgrading from 0.3.9 or earlier

Follow the instructions in OpenREM Release Notes version 0.4.0

Upgrading from 0.4.0 or above

Move straight to version 0.4.3 and follow the instructions in OpenREM Release Notes version 0.4.3

OpenREM Release Notes version 0.4.1
Headline changes
  • This release is exacly the same as 0.4.1 bar some documentation corrections
Specific upgrade instructions

Please use the 0.4.0 release notes for upgrades from 0.3.9

OpenREM Release Notes version 0.4.0

OpenREM Release Notes version 0.4.0
Headline changes
  • User authentication has been added
  • Studies can be deleted from the web interface
  • Import scripts can now be passed a list of files, eg python openrem_rdsr.py *.dcm
  • Date of birth no longer retained for mammography (bug fix - correct behaviour already existed for other imports)
  • General bug fixes to enable import from wider range of sources
  • Improved user documentation
Specific upgrade instructions
  • pip install openrem==0.4.2 Go straight to 0.4.2

  • Migrate the database

    Warning

    A database migration is required that will need a choice to be made

    • Linux: python /usr/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
    • Windows: C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp

    When South has considered the changes to the schema, you will see the following message:

    ? The field 'Observer_context.device_observer_name' does not have a default specified, yet is NOT NULL.
    ? Since you are making this field nullable, you MUST specify a default
    ? value to use for existing rows. Would you like to:
    ?  1. Quit now.
    ?  2. Specify a one-off value to use for existing columns now
    ?  3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
    ? Please select a choice: 3
    

    As per the final line above, the correct choice is 3. The fields that are now nullable previously weren’t. Existing data in those fields will have a value, or those tables don’t exist in the current database. The problem scenario is if after the migration these tables are used and one of the new nullable fields is left as null, you will not be able to migrate back to the old database schema without error. This is not something that you will want to do, so this is ok.

    Do the migration:

    • Linux: python /usr/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
    • Windows: C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  • Update the settings files

    Warning

    The settings file has changed and will need to be manually edited.

    Changes need to be made to the settings.py file where the database details are stored. Normally upgrades don’t touch this file and the copy in the upgrade has a .example suffix. This upgrade and potentially future ones will need to change this file, so the format has been changed. The settings.py file will now be replaced each time the code is upgraded. In addition, there is a new local_settings.py file that contains things that are specific to your installation, such as the database settings.

    This upgrade will include a file called settings.py.new and the local_settings.py.example file. You will need to do the following:

    • Copy the database settings from your current settings.py file to the local_settings.py.example file and rename it to remove the .example. Both of these files are in the openrem/openrem directory, which will be somewhere like
      • Linux: /usr/lib/python2.7/dist-packages/openrem/openrem/
      • Windows: C:\Python27\Lib\site-packages\openrem\openrem\
    • Move the existing settings.py out of the python directories
    • Rename the settings.py.new to settings.py
  • Create a new secret key

    All versions of openrem ship with the same secret key. This key is used for web security checks, and should be unique (and secret) for each installation.

  • Restart your webserver

  • Add some users

    • Go to the admin interface (eg http://localhost:8000/admin) and log in with the user created when you originally created the database (manage.py syncdb)

    • Create some users and add them to the appropriate groups (if there are no groups, go to the OpenREM homepage and they should be created).

      • viewgroup can browse the data only
      • exportgroup can do as view group plus export data to a spreadsheet, and will be able to import height and weight data in due course (See Issue #21)
      • admingroup can delete studies in addition to anything the export group can do

Contributing authors

Many people have contributed to OpenREM - either with code, documentation, bugs, examples or ideas, including:

Diagram of system components

digraph {
   node [fixedsize=true width=2.0 height=0.75 fontsize=10 margin="0.0,0.0"];

   // Define the things on the server
   subgraph cluster_server {

      label="OpenREM server";
      tooltip="The server";
      fontname="Helvetica-bold";
      fontsize=16;
      style=filled;
      color=lightgrey;
      node [style=filled color=white];

      // Define the nodes for the data storage, display and retrieval
      webserver [label="Web server&sup1;" fontname="Helvetica" tooltip="Serve web pages to the user" shape="box"];
      python_django [label="OpenREM\nDjango app" fontname="Helvetica" tooltip="Python web framework" shape="box"];
      database [label="Database&sup3;" fontname="Helvetica" tooltip="Relational database management system" shape="parallelogram"];
      rabbitmq [label="RabbitMQ\nmessage broker" fontname="Helvetica" tooltip="Message broker" shape="box"];
      celery [label="Celery\ntask queue" fontname="Helvetica" tooltip="Asynchronous task queue" shape="hexagon"];
      skin_dose_map_data [label="Skin dose map\ndata calculation,\nstorage, retrieval" fontname="Helvetica" tooltip="Calculate, store and retrieve skin dose map data" shape="parallelogram"];
      server_media_folder [label="Server file storage\n(Media Home folder)" fontname="Helvetica" tooltip="File storage on the server" shape="parallelogram"];
      data_export [label="Data export to\nlocal file system" fontname="Helvetica" tooltip="Files are made available to the user via a web page URL" shape="box"];

      // Define the links between the data storage, display and retrieval
      webserver -> python_django [dir=both];
      python_django -> database [dir=both label="via psycopg2\nPython adapter" fontsize=8 fontname="Courier"];
      python_django -> rabbitmq;
      rabbitmq -> celery;
      celery -> skin_dose_map_data;
      celery -> data_export;
      skin_dose_map_data -> server_media_folder [dir=both];
      skin_dose_map_data -> python_django [style=dotted dir=both];
      data_export -> server_media_folder;
      data_export -> python_django [style=dotted dir=both];

      // Define the nodes for the DICOM store, database population and skin dose map calculation
      conquest [label="DICOM StoreSCP\n(Conquest&sup2;)" fontname="Helvetica" tooltip="Conquest, acting as a DICOM storage SCP" shape="box"];
      conquest_script [shape=diamond label="Does the\nobject contain\nuseful data?" fontname="Helvetica" tooltip="Process the rules in dicom.ini"];
      populate_database [label="Extract information from\nthe DICOM object to the\nOpenREM database" fontname="Helvetica", tooltip="Extract data using OpenREM's python scripts" shape="box"];
      delete_object [label="Delete the DICOM object\nfrom the Conquest store" fontname="Helvetica" tooltip="Delete the DICOM object from the local store SCP" shape="box"];
      calc_skin_dose_map [shape=diamond label="Calculate\nskin dose\nmap?" fontname="Helvetica" tooltip="Calculate the skin dose map?"];
      blank_node_1 [shape=none style=invisible];

      // Define the links between the DICOM store, database population and skin dose map calculation
      conquest_script -> populate_database [label="Yes" fontcolor=darkgreen fontsize=8 fontname="Courier"];
      populate_database -> delete_object;
      conquest_script -> delete_object [label="No" fontcolor=red fontsize=8 fontname="Courier"];
      conquest -> conquest_script;
      populate_database -> calc_skin_dose_map;

      // Define the links between the two groups
      python_django -> populate_database [dir=back]
      calc_skin_dose_map -> celery [style=dotted label="Yes" fontcolor=darkgreen fontsize=8 fontname="Courier"]

      // Force certain nodes to be on the same level so that the diagram looks good (hopefully)
      {rank=same; webserver conquest};
      {rank=same; celery calc_skin_dose_map };
      {rank=same; python_django->blank_node_1->delete_object->populate_database [style=invis]; rankdir=LR;}
   }

   // Define the web browser, modality and pacs nodes
   web_browser [label="Client\nweb browser" fontname="Helvetica" tooltip="The user's web browser" shape="box" style=rounded];
   modality [label="X-ray imaging\nmodality" fontname="Helvetica" tooltip="Data send from an x-ray imaging modality" shape="parallelogram"];
   pacs [label="PACS" fontname="Helvetica" tooltip="A Picture Archiving and Communication System" shape="parallelogram"];
   blank_node_2 [label="" shape=none];

   // Define the links that the browser, modality and pacs have
   web_browser -> webserver [dir=both label="via user-requests\land Ajax\l" fontsize=8 fontname="Courier" tooltip="Ajax used to retrieve chart data"];
   modality -> conquest [label="via modality\lconfiguration\l" fontsize=8 fontname="Courier"];
   pacs -> conquest [label="via OpenREM\lquery-retrieve\l" fontsize=8 fontname="Courier"];

   // Force the web browser, blank node, modality and pacs to be on the same level in a specific order
   {rank=same; web_browser->blank_node_2->modality->pacs [style=invis]; rankdir=LR;};
}

Alternatives

1: Web servers

The recommended web server for Windows is Microsoft IIS - see Running OpenREM on Windows with IIS for details. This has replaced the recommendation to use Apache due to difficulties in obtaining the required binary files, as described in the Web servers section of the installation document.

The recommended web server for Linux is Gunicorn with NGINX - see Running OpenREM on Linux with Gunicorn and NGINX for details.

Alternatively, a built-in web server is included that will suffice for testing purposes and getting started.

2: DICOM Store node

Any DICOM Store can be used, as long as it can be used to call the OpenREM import script. A built-in store is available, but not recommended for production use. See DICOM Network Configuration for more details. Conquest is the recommended DICOM Store service to use.

3: Database

PostgreSQL is the recommended database to use with OpenREM. It is the only database that OpenREM will calculate median values for charts with. Other databases can be used with varying capabilities; see the Django documentation for more details. For testing only, the built-in SQLite3 database can be used, but this is not suitable for later migration to a production database.

Indices and tables