edeposit.amqp.antivirus¶

This module provides wrappers over ClamAV antivirus for edeposit project.

Installation¶

Module is hosted at PIP, so you can install it easily with following command:

sudo pip install edeposit.amqp.antivirus

This will install the module and necessary requirements with one exception - the ClamAV itself. That can be installed manually or using package manager from your distribution.

Ubuntu/Debian:

sudo apt-get install clamav clamav-daemon

OpenSuse:

sudo zypper install clamav

Initialization¶

After installation of the ClamAV and edeposit.amqp.antivirus, run the edeposit_clamd_init.py script (should be in your path), which will configure ClamAV and create all necessary files and directories.

You may also want to check settings module, to change some of the paths using JSON configuration files.

Database update¶

You should update the signature database from time to time.

You can do it by running freshclam command, or by sending UpdateDatabase structure over AMQP.

I think, that the best way is to put the freshclam command to cron.

Usage¶

To check some file, encode it to base64, put it into ScanFile structure and send it over AMQP to reactToAMQPMessage().

Here is example without AMQP communication, but at the AMQP level of abstraction:

$ python
>>> import base64
>>> import antivirus as av
>>> fn = "test_file.exe"
>>> data = open(fn).read()
>>> av.reactToAMQPMessage(
...    av.structures.ScanFile(fn, base64.b64encode(data)),
...    "UUID"
... )
ScanResult(filename='test_file.exe', result={})  # result is blank -> file is ok

Or positive detection:

$ python
>>> import base64
>>> import antivirus as av
>>> fn = "eicar.com"
>>> data = open(fn).read()
>>> av.reactToAMQPMessage(
...    av.structures.ScanFile(fn, base64.b64encode(data)),
...    "UUID"
... )
ScanResult(filename='test_file.exe', result={u'/tmp/tmpuCA2fe_eicar.com': ('FOUND', 'Eicar-Test-Signature')})

Content¶

Parts of the module can be divided into two subcategories - script and API.

Standalone script¶

Script can be found in bin/ folder and it should be automatically put into your path, so you can just simply run edeposit_clamd_init.py from shell.

Initializer script¶

Initialization script used to set necessary settings in ClamAV configuration file and correct permissions.

edeposit_clamd_init.REQUIRED_SETTINGS = {'MaxThreads': '2', 'LocalSocket': '/var/run/clamav/clamd.ctl', 'AllowSupplementaryGroups': 'true', 'ScanPE': 'true', 'FixStaleSocket': 'true', 'ReadTimeout': '180', 'LocalSocketMode': '666', 'LocalSocketGroup': '$groupname', 'ScanArchive': 'true', 'LogRotate': 'true', 'LogSyslog': 'false', 'User': '$username', 'LogTime': 'true', 'MaxConnectionQueueLength': '3', 'ScanELF': 'true', 'LogFile': '/var/log/clamav/clamav.log', 'PidFile': '/var/run/clamav/clamd.pid'}¶: All required settings is there, rest is not important.

edeposit_clamd_init.get_username()[source]¶: Return username depending on type of system (deb/suse).

edeposit_clamd_init.update_configuration(configuration)[source]¶

Set all configuration specified in REQUIRED_SETTINGS.

Parameters:	configuration (str) – Configuration file content.
Returns:	Updated configuration.
Return type:	str

edeposit_clamd_init.create_config(cnf_file, uid, overwrite)[source]¶

Creates configuration file and the directory where it should be stored and set correct permissions.

Parameters:	cnf_file (str) – Path to the configuration file. uid (int) – User ID - will be used for chown. overwrite (bool) – Overwrite the configuration with `CLEAN_CONFIG`.

edeposit_clamd_init.create_log(log_file, uid)[source]¶

Create log file and set necessary permissions.

Parameters:	log_file (str) – Path to the log file. uid (int) – User ID - will be used for chown.

edeposit_clamd_init.get_service_name()[source]¶: Return name of the daemon depending on the system type.

edeposit_clamd_init.main(*args, **kwargs)[source]¶

Create configuration and log file. Restart the daemon when configuration is done.

Parameters:	conf_file (str) – Path to the configuration file. overwrite (bool) – Overwrite the configuration file with clean config?

Usage¶

$ ./edeposit_clamd_init.py -h
usage: edeposit_clamd_init.py [-h] [-v] [-o] [-c CONFIG]

edeposit.amqp.antivirus ClamAV initializer.

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Print logging messages.
  -o, --overwrite       Overwrite default configuration file. Don't worry,
                        your original file will be stored in backup_.
  -c CONFIG, --config CONFIG
                        Path to the configuration file. Default
                        /etc/clamav/clamd.conf.

API¶

antivirus package¶

There are two levels of abstraction - AMPQ API and python API.

AQMP API is higlevel API, where you send some structure, something happens in magick box and you get back another structure.

Python API is just collection of “lowlevel” python wrappers over ClamAV.

AMQP API¶

AMQP interface used by edeposit.amqp package.

antivirus.reactToAMQPMessage(message, UUID)[source]¶

React to given (AMQP) message. message is expected to be collections.namedtuple() structure from structures filled with all necessary data.

Parameters:	message (object) – One of the request objects defined in `structures`. UUID (str) – Unique ID of received message.
Returns:	Response class from `structures`.
Return type:	object
Raises:	`ValueError` – if bad type of message structure is given.

All AMQP communication structures can be found in structures submodule.

Python API¶

Antivirus wrapper¶

ClamAV wrapper to scan files for malware.

antivirus.antivirus.scan_file(path)[source]¶

Scan path for viruses using clamd or clamscan (depends on settings.USE_CLAMD.

Parameters:	path (str) – Relative or absolute path of file/directory you need to scan.
Returns:	`{filename: ("FOUND", "virus type")}` or blank dict.
Return type:	dict
Raises:	`ValueError` – When the server is not running. `AssertionError` – When the internal file doesn’t exists.

antivirus.antivirus.save_and_scan(filename, b64_data)[source]¶

Save b64_data to temporary file and scan it for viruses.

Parameters:	filename (str) – Name of the file - used as basename for tmp file. b64_data (str) – Content of the file encoded in base64.
Returns:	`{filename: ("FOUND", "virus type")}` or blank dict.
Return type:	dict

Configuration writer¶

This module is used to write and update configuration for ClamAV daemon.

antivirus.conf_writer.add_or_update(data, item, value)[source]¶

Add or update value in configuration file format used by proftpd.

Parameters:	data (str) – Configuration file as string. item (str) – What option will be added/updated. value (str) – Value of option.
Returns:	updated configuration
Return type:	str

antivirus.conf_writer.comment(data, what)[source]¶

Comments line containing what in string data.

Parameters:	data (str) – Configuration file in string. what (str) – Line which will be commented out.
Returns:	Configuration file with commented what.
Return type:	str

Wrappers¶

wrappers package¶

This package contains lowlevel wrappers over clamd, clamscan and freshclam.

Submodules¶

clamd wrapper¶

API for scanning files using clamd daemon.

antivirus.wrappers.clamd.scan_file(path)[source]¶

Scan path for viruses using clamd antivirus daemon.

Parameters:	path (str) – Relative or absolute path of file/directory you need to scan.
Returns:	`{filename: ("FOUND", "virus type")}` or blank dict.
Return type:	dict
Raises:	`ValueError` – When the server is not running. `AssertionError` – When the internal file doesn’t exists.

clamscan wrapper¶

API for scanning files using clamscan standalone program.

antivirus.wrappers.clamscan.scan_file(path)[source]¶

Scan path for viruses using clamscan program.

Parameters:	path (str) – Relative or absolute path of file/directory you need to scan.
Returns:	`{filename: ("FOUND", "virus type")}` or blank dict.
Return type:	dict
Raises:	`AssertionError` – When the internal file doesn’t exists.

freshclam wrapper¶

Wrapper over freshclam program to update database over amqp.

antivirus.wrappers.freshclam.require_root(fn)[source]¶: Decorator to make sure, that user is root.

antivirus.wrappers.freshclam.update_database(*args, **kwargs)[source]¶: Run freshclam. Make sure, that user is root.

Package configuration¶

If you wish to change behavior or paths to some of the files, you can use do it in settings submodule.

AMQP communication structures¶

Definitions of the communication structures used in edeposit.amqp.antivirus project.

class antivirus.structures.ScanFile[source]¶

Bases: antivirus.structures.ScanFile

Request to scan file.

Parameters:	filename (str) – Path of the file at your system. It will be used in result structure. b64_data (str) – Base64 encoded content of the file.
Returns:	`ScanResult`
Return type:	object

Create new instance of ScanFile(filename, b64_data)

class antivirus.structures.UpdateDatabase[source]¶

Bases: antivirus.structures.UpdateDatabase

Request to update clamav database (= to run freshclam program).

Returns:	`DatabaseUpdated`
Return type:	object

Create new instance of UpdateDatabase()

class antivirus.structures.ScanResult[source]¶

Bases: antivirus.structures.ScanResult

Result of the file scan.

Parameters:	filename (str) – Name of the file as was specified in `ScanFile` request. result (dict) – Dictionary in following format:

{
    "local_path": ("RESULT", "TYPE")
}

Where RESULT is “FOUND” or string like that and TYPE is name of the malware.

Note

When no malware is found, result is blank dict.

Create new instance of ScanResult(filename, result)

class antivirus.structures.DatabaseUpdated[source]¶

Bases: antivirus.structures.DatabaseUpdated

Response to UpdateDatabase.

Attr:: log (str): Log of the freshclam run.

Create new instance of DatabaseUpdated(log,)

Settings and configuration¶

Module is containing all necessary global variables for the package.

Module also has the ability to read user-defined data from two paths:

$HOME/_SETTINGS_PATH
/etc/_SETTINGS_PATH

See _SETTINGS_PATH for details.

Note

If the first path is found, other is ignored.

Example of the configuration file ($HOME/edeposit/antivirus.json):

{
    "USE_CLAMD": false
}

Attributes¶

antivirus.settings.is_deb_system()[source]¶: Badly written test whether the system is deb/apt based or not.

antivirus.settings.USE_CLAMD = True¶: True - clamd daemon will be used, False - clamscan will be used. clamscan takes much less memory, but takes a LOT more time to scan. clamd takes huge amouts of memory (500MB min), but scans in fractions of seconds.

antivirus.settings.DEB_CONF_PATH = '/etc/clamav/'¶: Configuration file directory at debian systems.

antivirus.settings.SUSE_CONF_PATH = '/etc/'¶: Configuration file directory at suse systems.

antivirus.settings.CONF_FILE = 'clamd.conf'¶: Name of the configuration file.

antivirus.settings.CONF_PATH = '/etc/clamav/clamd.conf'¶: Path to the configuration file.

antivirus.settings.LOCALSOCKET = '/var/run/clamav/clamd.ctl'¶: Path to the local unix socket - don’t change this if you are not sure (it will break things).

antivirus.settings.PIDFILE = '/var/run/clamav/clamd.pid'¶: Path to the pid file - don’t change this if you are not sure (it will break things).

antivirus.settings.LOGFILE = '/var/log/clamav/clamav.log'¶: Path to the log file.

antivirus.settings.get_all_constants()[source]¶

Get list of all uppercase, non-private globals (doesn’t start with _).

Returns:	Uppercase names defined in globals() (variables from this module).
Return type:	list

antivirus.settings.substitute_globals(config_dict)[source]¶

Set global variables to values defined in config_dict.

Parameters:	config_dict (dict) – dictionary with data, which are used to set globals.

Note

config_dict have to be dictionary, or it is ignored. Also all variables, that are not already in globals, or are not types defined in _ALLOWED (str, int, float) or starts with _ are silently ignored.

Source code¶

The project is opensource (GPL) and source codes can be found at GitHub:

https://github.com/edeposit/edeposit.amqp.antivirus

Testing¶

Almost every feature of the project is tested in unit/integration tests. You can run this tests using provided run_tests.sh script, which can be found in the root of the project.

Requirements¶

Test script expects that pytest is installed. In case you don’t have it yet, it can be easily installed using following command:

pip install --user pytest

or for all users:

sudo pip install pytest

Options¶

Script provides three options - to run just unittests (-u switch), to run integration tests (-i switch) or to run both (-a switch).

Integration tests requires that ClamAV is installed, running and that the test script has root permissions.

Example of the success output from the test script:

$ sudo service clamav-daemon start
[sudo] password for bystrousak:
 * Starting ClamAV daemon clamd

$ ./run_tests.sh -a
============================= test session starts ==============================
platform linux2 -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2
collected 7 items

src/edeposit/amqp/antivirus/tests/integration/test_antivirus.py .....
src/edeposit/amqp/antivirus/tests/unittests/test_amqp.py ..

========================== 7 passed in 44.04 seconds ===========================

edeposit.amqp.antivirus¶

Installation¶

Initialization¶

Database update¶

Usage¶

Content¶

Standalone script¶

Initializer script¶

Usage¶

API¶

antivirus package¶

AMQP API¶

Python API¶

Antivirus wrapper¶

Configuration writer¶

Wrappers¶

Package configuration¶

AMQP communication structures¶

Settings and configuration¶

Attributes¶

Source code¶

Testing¶

Requirements¶

Options¶

Indices and tables¶