Cuckoo Sandbox Book¶

Using a Sandbox¶

Before starting to install, configure and use Cuckoo, you should take some time to think on what you want to achieve with it and how.

Some questions you should ask yourself:

• What kind of files do I want to analyze?
• What volume of analyses do I want to be able to handle?
• Which platform do I want to use to run my analysis on?
• What kind of information I want about the file?

The creation of the isolated environment (the virtual machine) is probably the most critical and important part of a sandbox deployment: it should be done carefully and with proper planning.

Before getting hands on the virtualization product of your choice, you should already have a design plan that defines:

• Which operating system, language and patching level to use.
• Which software to install and which versions (particularly important when analyzing exploits).

Consider that automated malware analysis is not deterministic and its success might depend on a trillion of factors: you are trying to make a malware run in a virtualized system as it would do on a native one, which could be tricky to achieve and may not always succeed. Your goal should be both to create a system able to handle all the requirements you need as well as try to make it as realistic as possible.

For example you could consider leaving some intentional traces of normal usage, such as browsing history, cookies, documents, images etc. If a malware is designed to operate, manipulate or steal such files you’ll be able to notice it.

Virtualized operating systems usually carry a lot of traces with them that makes them very easily detectable. Even if you shouldn’t overestimate this problem, you might want to take care of this and try to hide as many virtualization traces as possible. There is a lot of literature on Internet regarding virtualization detection techniques and countermeasures.

Once you finished designing and preparing the prototype of system you want, you can proceed creating it and deploying it. You will be always in time to change things or slightly fix them, but remember that good planning at the beginning always means less troubles in the long run.

Some History¶

Cuckoo Sandbox started as a Google Summer of Code project in 2010 within The Honeynet Project. It was originally designed and developed by Claudio “nex” Guarnieri, who is still the main developer and coordinates all efforts from joined developers and contributors.

After initial work during the summer 2010, the first beta release was published on Feb. 5th 2011, when Cuckoo was publicly announced and distributed for the first time.

In March 2011, Cuckoo has been selected again as a supported project during Google Summer of Code 2011 with The Honeynet Project, during which Dario Fernandes joined the project and extended its functionality.

On November 2nd 2011 Cuckoo the release of its 0.2 version to the public as the first real stable release. On late November 2011 Alessandro “jekil” Tanasi joined the team expanding Cuckoo’s processing and reporting functionality.

On December 2011 Cuckoo v0.3 gets released and quickly hits release 0.3.2 in early February.

In late January 2012 we opened Malwr.com, a free and public running Cuckoo Sandbox instance provided with a full fledged interface through which people can submit files to be analysed and get results back.

In March 2012 Cuckoo Sandbox wins the first round of the Magnificent7 program organized by Rapid7.

During the Summer of 2012 Jurriaan “skier” Bremer joined the development team, refactoring the Windows analysis component sensibly improving the analysis’ quality.

On 24th July 2012, Cuckoo Sandbox 0.4 is released.

On 20th December 2012, Cuckoo Sandbox 0.5 “To The End Of The World” is released.

On 15th April 2013 we released Cuckoo Sandbox 0.6, shortly after having launched the second version of Malwr.com.

On 1st August 2013 Claudio “nex” Guarnieri, Jurriaan “skier” Bremer and Mark “rep” Schloesser presented Mo’ Malware Mo’ Problems - Cuckoo Sandbox to the rescue at Black Hat Las Vegas.

On 9th January 2014, Cuckoo Sandbox 1.0 is released.

In March 2014 Cuckoo Foundation born as non-profit organization dedicated to growth of Cuckoo Sandbox and the surrounding projects and initiatives.

On 7th April 2014, Cuckoo Sandbox 1.1 is released.

Use Cases¶

Cuckoo is designed to be used both as a standalone application as well as to be integrated in larger frameworks, thanks to its extremely modular design.

It can be used to analyze:

• Generic Windows executables
• DLL files
• PDF documents
• Microsoft Office documents
• URLs and HTML files
• PHP scripts
• CPL files
• Visual Basic (VB) scripts
• ZIP files
• Java JAR
• Python files
• Almost anything else

Thanks to its modularity and powerful scripting capabilities, there’s not limit to what you can achieve with Cuckoo.

For more information on customizing Cuckoo, see the Customization chapter.

Architecture¶

Cuckoo Sandbox consists of a central management software which handles sample execution and analysis.

Each analysis is launched in a fresh and isolated virtual machine. Cuckoo’s infrastructure is composed by an Host machine (the management software) and a number of Guest machines (virtual machines for analysis).

The Host runs the core component of the sandbox that manages the whole analysis process, while the Guests are the isolated environments where the malware samples get actually safely executed and analyzed.

The following picture explains Cuckoo’s main architecture:

Although the recommended setup is GNU/Linux (Ubuntu preferably) as host and Windows XP Service Pack 3 as guest, Cuckoo has proved to work smoothly also on Mac OS X as host and Windows Vista and Windows 7 as guests.

Obtaining Cuckoo¶

Cuckoo can be downloaded from the official website, where the stable and packaged releases are distributed, or can be cloned from our official git repository.

Warning

While being more updated, including new features and bugfixes, the version available in the git repository should be considered an under development stage. Therefore its stability is not guaranteed and it most likely lacks updated documentation.

Upgrade starting from scratch¶

To start from scratch you have to perform a fresh setup as described in Installation. The following steps are suggested:

1. Backup your installation.
2. Read the documentation shipped with the new release.
3. Make sure to have installed all required dependencies, otherwise install them.
4. Do a Cuckoo fresh installation of the Host components.
5. Reconfigure Cuckoo as explained in this book (copying old configuration files is not safe because options can change between releases).
6. If you are using an external database instead of the default or you are using the MongoDb reporting module is suggested to start all databases from scratch, due to possible schema changes between Cuckoo releases.
7. Test it!

If something goes wrong you probably failed to do some steps during the fresh installation or reconfiguration. Check again the procedure explained in this book.

It’s not recommended to rewrite an old Cuckoo installation with the latest release files, as it might raise some problems because:

• You are overwriting Python source files (.py) but Python bytecode files (.pyc) are still in place.
• There are configuration files changes across the two versions, check our CHANGELOG file for added or removed configuration options.
• The part of Cuckoo which runs inside guests (agent.py) may change.
• If you are using an external database like the reporting module for MongoDb a change in the data schema may corrupt your database.

Migrate your Cuckoo¶

Data migration is shipped starting from Cuckoo 1.1 and supports migration starting from Cuckoo 0.6. If your Cuckoo release is older than 0.6 you can’t migrate your data.

The following steps are suggested as requirement to migrate your data:

1. Backup your installation.
2. Read the documentation shipped with the new release.
3. Make sure to have installed all required dependencies, otherwise install them.
4. Download and extract the latest Cuckoo.
5. Reconfigure Cuckoo as explained in this book (copying old configuration files is not safe because options can change between releases), and update agent in your virtual machines.
6. Copy from your backup “storage” and “db” folders. (Reports and analyses already present in “storage” folder will keep the old format.)

Now setup Alembic (the framework used for migrations) and dateutil with:

pip install alembic
pip install python-dateutil

Enter the alembic migration directory in “utils/db_migration” with:

cd utils/db_migration

Before starting the migration script you must set your database connection in “cuckoo.conf” if you are using a custom one. Alembic migration script will use the database connection parameters configured in cuckoo.conf.

Again, please remember to backup before launching migration tool! A wrong configuration may corrupt your data, backup should save kittens!

Run the database migrations with:

alembic upgrade head

Submission Utility¶

The easiest way to submit an analysis is to use the provided submit.py command-line utility. It currently has the following options available:

usage: submit.py [-h] [--remote REMOTE] [--url] [--package PACKAGE]
                 [--custom CUSTOM] [--timeout TIMEOUT] [--options OPTIONS]
                 [--priority PRIORITY] [--machine MACHINE]
                 [--platform PLATFORM] [--memory] [--enforce-timeout]
                 [--clock CLOCK] [--tags TAGS] [--max MAX] [--pattern PATTERN]
                 [--shuffle] [--unique] [--quiet]
                 target

positional arguments:
  target               URL, path to the file or folder to analyze

optional arguments:
  -h, --help           show this help message and exit
  --remote REMOTE      Specify IP:port to a Cuckoo API server to submit
                       remotely
  --url                Specify whether the target is an URL
  --package PACKAGE    Specify an analysis package
  --custom CUSTOM      Specify any custom value
  --timeout TIMEOUT    Specify an analysis timeout
  --options OPTIONS    Specify options for the analysis package (e.g.
                       "name=value,name2=value2")
  --priority PRIORITY  Specify a priority for the analysis represented by an
                       integer
  --machine MACHINE    Specify the identifier of a machine you want to use
  --platform PLATFORM  Specify the operating system platform you want to use
                       (windows/darwin/linux)
  --memory             Enable to take a memory dump of the analysis machine
  --enforce-timeout    Enable to force the analysis to run for the full
                       timeout period
  --clock CLOCK        Set virtual machine clock
  --tags TAGS          Specify tags identifier of a machine you want to use
  --max MAX            Maximum samples to add in a row
  --pattern PATTERN    Pattern of files to submit
  --shuffle            Shuffle samples before submitting them
  --unique             Only submit new samples, ignore duplicates
  --quiet              Only print text on failure

If you specify a directory as path, all the files contained in it will be submitted for analysis.

The concept of analysis packages will be dealt later in this documentation (at Analysis Packages). Following are some usage examples:

Example: submit a local binary:

$ ./utils/submit.py /path/to/binary

Example: submit an URL:

$ ./utils/submit.py --url http://www.example.com

Example: submit a local binary and specify an higher priority:

$ ./utils/submit.py --priority 5 /path/to/binary

Example: submit a local binary and specify a custom analysis timeout of 60 seconds:

$ ./utils/submit.py --timeout 60 /path/to/binary

Example: submit a local binary and specify a custom analysis package:

$ ./utils/submit.py --package <name of package> /path/to/binary

Example: submit a local binary and specify a custom analysis package and some options (in this case a command line argument for the malware):

$ ./utils/submit.py --package exe --options arguments=--dosomething /path/to/binary.exe

Example: submit a local binary to be run on virtual machine cuckoo1:

$ ./utils/submit.py --machine cuckoo1 /path/to/binary

Example: submit a local binary to be run on a Windows machine:

$ ./utils/submit.py --platform windows /path/to/binary

Example: submit a local binary and take a full memory dump of the analysis machine:

$ ./utils/submit.py --memory /path/to/binary

Example: submit a local binary and force the analysis to be executed for the full timeout (disregarding the internal mechanism that Cuckoo uses to decide when to terminate the analysis):

$ ./utils/submit.py --enforce-timeout /path/to/binary

Example: submit a local binary and set virtual machine clock. Format is %m-%d-%Y %H:%M:%S. If not specified, the current time is used. For example if we want run a sample the 24 january 2001 at 14:41:20:

$ ./utils/submit.py --clock "01-24-2001 14:41:20" /path/to/binary

Example: submit a sample for Volatility analysis (to reduce side effects of the cuckoo hooking, switch it off with options free=True):

$ ./utils/submit.py --memory --options free=True /path/to/binary

web.py¶

Cuckoo provides a very small utility under utils/web.py, which will bind a simple webserver on localhost port 8080, through which you will be able to browse through existing reports as well as submit new files.

Beware that this is not a full-fledged web interface, which is instead provided under the folder web/ as a Django-powered application. You can find more details about that under Web interface.

API¶

Detailed usage of the REST API interface is described in REST API.

Distributed Cuckoo¶

Detailed usage of the Distributed Cuckoo API interface is described in Distributed Cuckoo.

Python Functions¶

In order to keep track of submissions, samples and overall execution, Cuckoo uses a popular Python ORM called SQLAlchemy that allows you to make the sandbox use SQLite, MySQL, PostgreSQL and several other SQL database systems.

Cuckoo is designed to be easily integrated in larger solutions and to be fully automated. In order to automate analysis submission we suggest to use the REST API interface described in REST API, but in case you want to write your own Python submission script, you can also use the add_path() and add_url() functions.

add_path(file_path[, timeout=0[, package=None[, options=None[, priority=1[, custom=None[, machine=None[, platform=None[, memory=False[, enforce_timeout=False], clock=None[]]]]]]]]])¶

Add a local file to the list of pending analysis tasks. Returns the ID of the newly generated task.

Parameters:

file_path (string) – path to the file to submit timeout (integer) – maximum amount of seconds to run the analysis for package (string or None) – analysis package you want to use for the specified file options (string or None) – list of options to be passed to the analysis package (in the format key=value,key=value) priority (integer) – numeric representation of the priority to assign to the specified file (1 being low, 2 medium, 3 high) custom (string or None) – custom value to be passed over and possibly reused at processing or reporting machine (string or None) – Cuckoo identifier of the virtual machine you want to use, if none is specified one will be selected automatically platform (string or None) – operating system platform you want to run the analysis one (currently only Windows) memory (True or False) – set to True to generate a full memory dump of the analysis machine enforce_timeout (True or False) – set to True to force the execution for the full timeout clock (string or None) – provide a custom clock time to set in the analysis machine

Return type:

integer

Example usage:

1 2 3 4 5

>>> from lib.cuckoo.core.database import Database >>> db = Database() >>> db.add_path("/tmp/malware.exe") 1 >>>

add_url(url[, timeout=0[, package=None[, options=None[, priority=1[, custom=None[, machine=None[, platform=None[, memory=False[, enforce_timeout=False], clock=None[]]]]]]]]])¶

Add a local file to the list of pending analysis tasks. Returns the ID of the newly generated task.

Parameters:

url (string) – URL to analyze timeout (integer) – maximum amount of seconds to run the analysis for package (string or None) – analysis package you want to use for the specified URL options (string or None) – list of options to be passed to the analysis package (in the format key=value,key=value) priority (integer) – numeric representation of the priority to assign to the specified URL (1 being low, 2 medium, 3 high) custom (string or None) – custom value to be passed over and possibly reused at processing or reporting machine (string or None) – Cuckoo identifier of the virtual machine you want to use, if none is specified one will be selected automatically platform (string or None) – operating system platform you want to run the analysis one (currently only Windows) memory (True or False) – set to True to generate a full memory dump of the analysis machine enforce_timeout (True or False) – set to True to force the execution for the full timeout clock (string or None) – provide a custom clock time to set in the analysis machine

Return type:

integer

Example Usage:

1
2
3
4
5

>>> from lib.cuckoo.core.database import Database
>>> db = Database()
>>> db.add_url("http://www.cuckoosandbox.org")
2
>>>

Configuration¶

The web interface pulls data from a Mongo database, so having the Mongo reporting module enabled in reporting.conf is mandatory for this interface. If that’s not the case, the application won’t start and it will raise an exception.

The interface can be configured by editing local_settings.py under web/web/:

# If you want to customize your cuckoo path set it here.
# CUCKOO_PATH = "/where/cuckoo/is/placed/"

# Maximum upload size.
MAX_UPLOAD_SIZE = 26214400

# Override default secret key stored in secret_key.py
# Make this unique, and don't share it with anybody.
# SECRET_KEY = "YOUR_RANDOM_KEY"

# Local time zone for this installation. Choices can be found here:
# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
# although not all choices may be available on all operating systems.
# On Unix systems, a value of None will cause Django to use the same
# timezone as the operating system.
# If running in a Windows environment this must be set to the same as your
# system time zone.
TIME_ZONE = "America/Chicago"

# Language code for this installation. All choices can be found here:
# http://www.i18nguy.com/unicode/language-identifiers.html
LANGUAGE_CODE = "en-us"

ADMINS = (
    # ("Your Name", "your_email@example.com"),
)

MANAGERS = ADMINS

# Allow verbose debug error message in case of application fault.
# It's strongly suggested to set it to False if you are serving the
# web application from a web server front-end (i.e. Apache).
DEBUG = True

# A list of strings representing the host/domain names that this Django site
# can serve.
# Values in this list can be fully qualified names (e.g. 'www.example.com').
# When DEBUG is True or when running tests, host validation is disabled; any
# host will be accepted. Thus it's usually only necessary to set it in production.
ALLOWED_HOSTS = ["*"]

Usage¶

In order to start the web interface, you can simply run the following command from the web/ directory:

$ python manage.py runserver

If you want to configure the web interface as listening for any IP on a specified port, you can start it with the following command (replace PORT with the desired port number):

$ python manage.py runserver 0.0.0.0:PORT

You can serve Cuckoo’s web interface using WSGI interface with common web servers: Apache, Nginx, Unicorn and so on. Please refer both to the documentation of the web server of your choice as well as Django documentation.

Starting the API server¶

In order to start the API server you can simply do:

$ ./utils/api.py

By default it will bind the service on localhost:8090. If you want to change those values, you can for example do this:

$ ./utils/api.py --host 0.0.0.0 --port 1337

Resources¶

Following is a list of currently available resources and a brief description of each one. For details click on the resource name.

Dependencies¶

The distributed script uses a few Python libraries which can be installed through the following command (on Debian/Ubuntu):

$ sudo pip install flask flask-restful flask-sqlalchemy requests

Starting the Distributed REST API¶

The Distributed REST API requires a few commandline options in order to run. Following is a listing of all available commandline options:

$ ./utils/dist.py -h

usage: dist.py [-h] [-d] [--db DB] --samples-directory SAMPLES_DIRECTORY
            [--uptime-logfile UPTIME_LOGFILE] --report-formats
            REPORT_FORMATS --reports-directory REPORTS_DIRECTORY
            [host] [port]

positional arguments:
    host                  Host to listen on
    port                  Port to listen on

optional arguments:
    -h, --help            show this help message and exit
    -d, --debug           Enable debug logging
    --db DB               Database connection string
    --samples-directory SAMPLES_DIRECTORY
                            Samples directory
    --uptime-logfile UPTIME_LOGFILE
                            Uptime logfile path
    --report-formats REPORT_FORMATS
                            Reporting formats to fetch
    --reports-directory REPORTS_DIRECTORY
                            Reports directory

In particular the --report-formats, --samples-directory, and --reports-directory are required.

RESTful resources¶

Following are all RESTful resources. Also make sure to check out the Quick usage section which documents the most commonly used commands.

$ curl http://localhost:9003/node
{
    "nodes": {
        "localhost": {
            "machines": [
                {
                    "name": "cuckoo1",
                    "platform": "windows",
                    "tags": [
                        ""
                    ]
                }
            ],
            "name": "localhost",
            "url": "http://0:8090/"
        }
    }
}

$ curl http://localhost:9003/node -F name=localhost \
    -F url=http://localhost:8090/
{
    "machines": [
        {
            "name": "cuckoo1",
            "platform": "windows",
            "tags": []
        }
    ],
    "name": "localhost"
}

$ curl http://localhost:9003/node/localhost
{
    "name": "localhost",
    "url": "http://localhost:8090/"
}

$ curl -XPUT http://localhost:9003/node/localhost -F name=newhost \
    -F url=http://1.2.3.4:8090/
null

$ curl -XDELETE http://localhost:9003/node/localhost
null

$ curl http://localhost:9003/task
{
    "tasks": {
        "1": {
            "clock": null,
            "custom": null,
            "enforce_timeout": null,
            "machine": null,
            "memory": null,
            "options": null,
            "package": null,
            "path": "/tmp/dist-samples/tmphal8mS",
            "platform": "windows",
            "priority": 1,
            "tags": null,
            "task_id": 1,
            "timeout": null
        }
    }
}

$ curl http://localhost:9003/task -F file=@sample.exe
{
    "task_id": 2
}

$ curl http://localhost:9003/task/2
{
    "tasks": {
        "2": {
            "clock": null,
            "custom": null,
            "enforce_timeout": null,
            "machine": null,
            "memory": null,
            "options": null,
            "package": null,
            "path": "/tmp/tmpPwUeXm",
            "platform": "windows",
            "priority": 1,
            "tags": null,
            "task_id": 2,
            "timeout": null
        }
    }
}

$ curl -XDELETE http://localhost:9003/task/2
null

# Defaults to the JSON report.
$ curl http://localhost:9003/report/2
...

# Get an XML report.
$ curl http://localhost:9003/report/2/maec -H "Accept: application/xml"

Quick usage¶

For practical usage the following few commands will be most interesting.

Register a Cuckoo node - a Cuckoo REST API running on the same machine in this case:

$ curl http://localhost:9003/node -F name=localhost -F url=http://localhost:8090/

Disable a Cuckoo node:

$ curl -XDELETE http://localhost:9003/node/<name>

Submit a new analysis task without any special requirements (e.g., using Cuckoo tags, a particular machine, etc):

$ curl http://localhost:9003/task -F file=@/path/to/sample.exe

Get the report of a task has been finished (if it hasn’t finished you’ll get a 404 page). Following example will default to the JSON report:

$ curl http://localhost:9003/report/1

In order to fetch an XML report such as a MAEC report, use the following instead:

$ curl http://localhost:9003/report/1/maec -H 'Accept: application/xml'

Proposed setup¶

The following description depicts a Distributed Cuckoo setup with two Cuckoo machines, cuckoo0 and cuckoo1. In this setup the first machine, cuckoo0, also hosts the Distributed Cuckoo REST API.

./cuckoo.py
./utils/api.py -H 1.2.3.4  # IP accessible by the Distributed script.
./utils/process.py auto

$ screen -S cuckoo  ./cuckoo.py
$ screen -S api     ./utils/api.py
$ screen -S process ./utils/process.py auto

$ screen -S distributed ./utils/dist.py --samples-directory /a/b/samples \
    --report-formats json --reports-directory /a/b/reports

$ curl http://localhost:9003/node -F name=cuckoo0 -F url=http://localhost:8090/
$ curl http://1.2.3.4:9003/node -F name=cuckoo1 -F url=http://1.2.3.4:8090/

analysis.conf¶

This is a configuration file automatically generated by Cuckoo to give its analyzer some details about the current analysis. It’s generally of no interest to the end-user, as it’s used internally by the sandbox.

analysis.log¶

This is a log file generated by the analyzer and that contains a trace of the analysis execution inside the guest environment. It will report the creation of processes, files and eventual errors occurred during the execution.

dump.pcap¶

This is the network dump generated by tcpdump or any other corresponding network sniffer.

memory.dmp¶

In case you enabled it, this file contains the full memory dump of the analysis machine.

files/¶

This directory contains all the files the malware operated on and that Cuckoo was able to dump.

logs/¶

This directory contains all the raw logs generated by Cuckoo’s process monitoring.

reports/¶

This directory contains all the reports generated by Cuckoo as explained in the Configuration chapter.

shots/¶

This directory contains all the screenshots of the guest’s desktop taken during the malware execution.

Cleanup utility¶

If you want to delete all history, analysis, data and begin again from the first task you need the clean.sh utility.

To clean your setup, run:

$ ./utils/clean.sh

This utility is designed to be used with Cuckoo (including API and web interface) not running.

If you are using a custom database (MySQL, PostgreSQL or SQLite in custom location) clean.sh doesn’t clean it, you have to take care of that.

If you are using the MongoDB reporting module clean.sh does not clean your database, you have to take care of that.

Submission Utility¶

Submits samples to analysis. This tool is already described in Submit an Analysis.

Web Utility¶

Cuckoo’s web interface. This tool is already described in Submit an Analysis.

Processing Utility¶

Run the results processing engine and optionally the reporting engine (run all reports) on an already available analysis folder, in order to not re-run the analysis if you want to re-generate the reports for it. This is used mainly in debugging and developing Cuckoo. For example if you want run again the report engine for analysis number 1:

$ ./utils/process.py 1

If you want to re-generate the reports:

$ ./utils/process.py --report 1

Following are the usage options:

$ ./utils/process.py -h

usage: process.py [-h] [-d] [-r] [-p PARALLEL] id

positional arguments:
  id                    ID of the analysis to process (auto for continuous
                        processing of unprocessed tasks).

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Display debug messages
  -r, --report          Re-generate report
  -p PARALLEL, --parallel PARALLEL
                        Number of parallel threads to use (auto mode only).

As best practice we suggest to adopt the following configuration if you are running Cuckoo with many virtual machines:

• Run a stand alone process.py in auto mode (you choose the number of parallel threads)
• Disable Cuckoo reporting in cuckoo.conf (set process_results to off)

This could increase the performance of your system because the reporting is not yet demanded to Cuckoo.

Community Download Utility¶

This utility downloads signatures from Cuckoo Community Repository and installs specific additional modules in your local setup and for example update id with all the latest available signatures. Following are the usage options:

$ ./utils/community.py -h

usage: community.py [-h] [-a] [-s] [-p] [-m] [-r] [-f] [-w] [-b BRANCH]

optional arguments:
  -h, --help            show this help message and exit
  -a, --all             Download everything
  -s, --signatures      Download Cuckoo signatures
  -p, --processing      Download processing modules
  -m, --machinemanagers
                        Download machine managers
  -r, --reporting       Download reporting modules
  -f, --force           Install files without confirmation
  -w, --rewrite         Rewrite existing files
  -b BRANCH, --branch BRANCH
                        Specify a different branch

Example: install all available signatures:

$ ./utils/community.py --signatures --force

Database migration utility¶

This utility is developed to migrate your data between Cuckoo’s release. It’s developed on top of the Alembic framework and it should provide data migration for both SQL database and Mongo database. This tool is already described in Upgrade from a previous release.

Stats utility¶

This is a really simple utility which prints some statistics about processed samples:

$ ./utils/stats.py

1 samples in db
1 tasks in db
pending 0 tasks
running 0 tasks
completed 0 tasks
recovered 0 tasks
reported 1 tasks
failed_analysis 0 tasks
failed_processing 0 tasks
roughly 32 tasks an hour
roughly 778 tasks a day

Machine utility¶

The machine.py utility is designed to help you automatize the configuration of virtual machines in Cuckoo. It takes a list of machine details as arguments and write them in the specified configuration file of the machinery module enabled in cuckoo.conf. Following are the available options:

$ ./utils/machine.py -h
usage: machine.py [-h] [--debug] [--add] [--ip IP] [--platform PLATFORM]
                [--tags TAGS] [--interface INTERFACE] [--snapshot SNAPSHOT]
                [--resultserver RESULTSERVER]
                vmname

positional arguments:
  vmname                Name of the Virtual Machine.

optional arguments:
  -h, --help            show this help message and exit
  --debug               Debug log in case of errors.
  --add                 Add a Virtual Machine.
  --ip IP               Static IP Address.
  --platform PLATFORM   Guest Operating System.
  --tags TAGS           Tags for this Virtual Machine.
  --interface INTERFACE
                        Sniffer interface for this machine.
  --snapshot SNAPSHOT   Specific Virtual Machine Snapshot to use.
  --resultserver RESULTSERVER
                        IP:Port of the Result Server.

Configuration¶

Every machinery module should come with a dedicated configuration file located in conf/<machinery module name>.conf. For example for modules/machinery/kvm.py we have a conf/kvm.conf.

The configuration file should follow the default structure:

[kvm]
# Specify a comma-separated list of available machines to be used. For each
# specified ID you have to define a dedicated section containing the details
# on the respective machine. (E.g. cuckoo1,cuckoo2,cuckoo3)
machines = cuckoo1

[cuckoo1]
# Specify the label name of the current machine as specified in your
# libvirt configuration.
label = cuckoo1

# Specify the operating system platform used by current machine
# [windows/darwin/linux].
platform = windows

# Specify the IP address of the current machine. Make sure that the IP address
# is valid and that the host machine is able to reach it. If not, the analysis
# will fail.
ip = 192.168.122.105

A main section called [<name of the module>] with a machines field containing a comma-separated list of machines IDs.

For each machine you should specify a label, a platform and its ip.

These fields are required by Cuckoo in order to use the already embedded initialize() function that generates the list of available machines.

If you plan to change the configuration structure you should override the initialize() function (inside your own module, no need to modify Cuckoo’s core code). You can find its original code in the Machinery abstract inside lib/cuckoo/common/abstracts.py.

LibVirt¶

Starting with Cuckoo 0.5 developing new machinery modules based on LibVirt is easy. Inside lib/cuckoo/common/abstracts.py you can find LibVirtMachinery that already provides all the functionality for a LibVirt module. Just inherit this base class and specify your connection string, as in the example below:

1 2 3 4 5

from lib.cuckoo.common.abstracts import LibVirtMachinery class MyMachinery(LibVirtMachinery): # Set connection string. dsn = "my:///connection"

This works for all the virtualization technologies supported by LibVirt. Just remember to check if your LibVirt package (if you are using one, for example from your Linux distribution) is compiled with the support for the technology you need.

You can check it with the following command:

$ virsh -V
Virsh command line tool of libvirt 0.9.13
See web site at http://libvirt.org/

Compiled with support for:
 Hypervisors: QEmu/KVM LXC UML Xen OpenVZ VMWare Test
 Networking: Remote Daemon Network Bridging Interface Nwfilter VirtualPort
 Storage: Dir Disk Filesystem SCSI Multipath iSCSI LVM
 Miscellaneous: Nodedev AppArmor Secrets Debug Readline Modular

If you don’t find your virtualization technology in the list of Hypervisors, you will need to recompile LibVirt with the specific support for the missing one.

Getting started¶

As an example we’ll take a look at the default package for analyzing generic Windows executables (located at analyzer/windows/packages/exe.py):

1 2 3 4 5 6 7 8

from lib.common.abstracts import Package class Exe(Package): """EXE analysis package.""" def start(self, path): args = self.options.get("arguments") return self.execute(path, args)

It seems really easy, thanks to all method inherited by Package object. Let’s have a look as some of the main methods an analysis package inherits from Package object:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

from lib.api.process import Process from lib.common.exceptions import CuckooPackageError class Package(object): def start(self): raise NotImplementedError def check(self): return True def execute(self, path, args): dll = self.options.get("dll") free = self.options.get("free") suspended = True if free: suspended = False p = Process() if not p.execute(path=path, args=args, suspended=suspended): raise CuckooPackageError("Unable to execute the initial process, " "analysis aborted.") if not free and suspended: p.inject(dll) p.resume() p.close() return p.pid def finish(self): if self.options.get("procmemdump"): for pid in self.pids: p = Process(pid=pid) p.dump_memory() return True

Let’s walk through the code:

• Line 1: import the Process API class, which is used to create and manipulate Windows processes.
• Line 2: import the CuckooPackageError exception, which is used to notify issues with the execution of the package to the analyzer.
• Line 4: define the main class, inheriting object.
• Line 5: define the start() function, which takes as argument the path to the file to execute. It should be implemented by each analysis package.
• Line 8: define the check() function.
• Line 13: acquire the free option, which is used to define whether the process should be monitored or not.
• Line 18: initialize a Process instance.
• Line 19: try to execute the malware, if it fails it aborts the execution and notify the analyzer.
• Line 23: check if the process should be monitored.
• Line 24: inject the process with our DLL.
• Line 25: resume the process from the suspended state.
• Line 27: return the PID of the newly created process to the analyzer.
• Line 29: define the finish() function.
• Line 30: check if the procmemdump option was enabled.
• Line 31: loop through the currently monitored processes.
• Line 32: open a Process instance.
• Line 33: take a dump of the process memory.

def check(self):
    if os.path.exists("C:\\config.bin"):
        return False
    else:
        return True

Options¶

Every package have automatically access to a dictionary containing all user-specified options (see Submit an Analysis).

Such options are made available in the attribute self.options. For example let’s assume that the user specified the following string at submission:

foo=1,bar=2

The analysis package selected will have access to these values:

from lib.common.abstracts import Package

class Example(Package):

    def start(self, path):
        foo = self.options["foo"]
        bar = self.options["bar"]

    def check():
        return True

    def finish():
        return True

These options can be used for anything you might need to configure inside your package.

Process API¶

The Process class provides access to different process-related features and functions. You can import it in your analysis packages with:

from lib.api.process import Process

You then initialize an instance with:

p = Process()

In case you want to open an existing process instead of creating a new one, you can specify multiple arguments:

• pid: PID of the process you want to operate on.
• h_process: handle of a process you want to operate on.
• thread_id: thread ID of a process you want to operate on.
• h_thread: handle of the thread of a process you want to operate on.

This class implements several methods that you can use in your own scripts.

Global Container¶

After an analysis is completed, Cuckoo will invoke all the processing modules available in the modules/processing/ directory. Any additional module you decide to create must be placed inside that directory.

Every module should also have a dedicated section in the file conf/processing.conf: for example if you create a module module/processing/foobar.py you will have to append the following section to conf/processing.conf:

[foobar]
enabled = on

Every module will then be initialized and executed and the data returned will be appended in a data structure that we’ll call global container.

This container is simply just a big Python dictionary that includes the abstracted results produced by all the modules classified by their identification key.

Cuckoo already provides a default set of modules which will generate a standard global container. It’s important for the existing reporting modules (HTML report etc.) that these default modules are not modified, otherwise the resulting global container structure would change and the reporting modules wouldn’t be able to recognize it and extract the information used to build the final reports.

The currently available default processing modules are:

• AnalysisInfo (modules/processing/analysisinfo.py) - generates some basic information on the current analysis, such as timestamps, version of Cuckoo and so on.
• BehaviorAnalysis (modules/processing/behavior.py) - parses the raw behavioral logs and perform some initial transformations and interpretations, including the complete processes tracing, a behavioral summary and a process tree.
• Debug (modules/processing/debug.py) - includes errors and the analysis.log generated by the analyzer.
• Dropped (modules/processing/dropped.py) - includes information on the files dropped by the malware and dumped by Cuckoo.
• Memory (modules/processing/memory.py) - executes Volatility on a full memory dump.
• NetworkAnalysis (modules/processing/network.py) - parses the PCAP file and extracts some network information, such as DNS traffic, domains, IPs, HTTP requests, IRC and SMTP traffic.
• ProcMemory (modules/processing/procmemory.py) - performs analysis of process memory dump. Note: the module is able to process user defined Yara rules from data/yara/memory/index_memory.yar. Just edit this file to add your Yara rules.
• StaticAnalysis (modules/processing/static.py) - performs some static analysis of PE32 files.
• Strings (modules/processing/strings.py) - extracts strings from the analyzed binary.
• TargetInfo (modules/processing/targetinfo.py) - includes information on the analyzed file, such as hashes.
• VirusTotal (modules/processing/virustotal.py) - searches on VirusTotal.com for antivirus signatures of the analyzed file. Note: the file is not uploaded on VirusTotal.com, if the file was not previously uploaded on the website no results will be retrieved.

Getting started¶

In order to make them available to Cuckoo, all processing modules must be placed inside the folder at modules/processing/.

A basic processing module could look like:

1 2 3 4 5 6 7 8

from lib.cuckoo.common.abstracts import Processing class MyModule(Processing): def run(self): self.key = "key" data = do_something() return data

Every processing module should contain:

• A class inheriting Processing.
• A run() function.
• A self.key attribute defining the name to be used as a sub container for the returned data.
• A set of data (list, dictionary, string, etc.) that will be appended to the global container.

You can also specify an order value, which allows you to run the available processing modules in an ordered sequence. By default all modules are set with an order value of 1 and are executed in alphabetical order.

If you want to change this value your module would look like:

1 2 3 4 5 6 7 8 9

from lib.cuckoo.common.abstracts import Processing class MyModule(Processing): order = 2 def run(self): self.key = "key" data = do_something() return data

You can also manually disable a processing module by setting the enabled attribute to False:

1 2 3 4 5 6 7 8 9

from lib.cuckoo.common.abstracts import Processing class MyModule(Processing): enabled = False def run(self): self.key = "key" data = do_something() return data

The processing modules are provided with some attributes that can be used to access the raw results for the given analysis:

• self.analysis_path: path to the folder containing the results (e.g. storage/analysis/1)
• self.log_path: path to the analysis.log file.
• self.conf_path: path to the analysis.conf file.
• self.file_path: path to the analyzed file.
• self.dropped_path: path to the folder containing the dropped files.
• self.logs_path: path to the folder containing the raw behavioral logs.
• self.shots_path: path to the folder containing the screenshots.
• self.pcap_path: path to the network pcap dump.
• self.memory_path: path to the full memory dump, if created.
• self.pmemory_path: path to the process memory dumps, if created.

With these attributes you should be able to easily access all the raw results stored by Cuckoo and perform your analytic operations on them.

As a last note, a good practice is to use the CuckooProcessingError exception whenever the module encounters an issue you want to report to Cuckoo. This can be done by importing the class like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14

from lib.cuckoo.common.exceptions import CuckooProcessingError from lib.cuckoo.common.abstracts import Processing class MyModule(Processing): def run(self): self.key = "key" try: data = do_something() except SomethingFailed: raise CuckooProcessingError("Failed") return data

Getting started¶

Creation of signatures is a very simple process and requires just a decent understanding of Python programming.

First things first, all signatures must be located inside modules/signatures/.

The following is a basic example signature:

1 2 3 4 5 6 7 8 9 10 11 12 13

from lib.cuckoo.common.abstracts import Signature class CreatesExe(Signature): name = "creates_exe" description = "Creates a Windows executable on the filesystem" severity = 2 categories = ["generic"] authors = ["Cuckoo Developers"] minimum = "0.5" def run(self): return self.check_file(pattern=".*\\.exe$", regex=True)

As you can see the structure is really simple and consistent with the other modules. We’re going to get into details later, but as you can see at line 12 from version 0.5 Cuckoo provides some helper functions that make the process of creating signatures much easier.

In this example we just walk through all the accessed files in the summary and check if there is anything ending with “.exe”: in that case it will return True, meaning that the signature matched, otherwise return False.

In case the signature gets matched, a new entry in the “signatures” section will be added to the global container as follows:

"signatures": [
    {
        "severity": 2,
        "description": "Creates a Windows executable on the filesystem",
        "alert": false,
        "references": [],
        "data": [
            {
                "file_name": "C:\\d.exe"
            }
        ],
        "name": "creates_exe"
    }
]

We could rewrite the exact same signature by accessing the global container directly:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

from lib.cuckoo.common.abstracts import Signature class CreatesExe(Signature): name = "creates_exe" description = "Creates a Windows executable on the filesystem" severity = 2 categories = ["generic"] authors = ["Cuckoo Developers"] minimum = "0.5" def run(self): for file_path in self.results["behavior"]["summary"]["files"]: if file_path.endswith(".exe"): return True return False

This obviously requires you to know the structure of the global container, which you can observe represented in the JSON report of your analyses.

Creating your new signature¶