Welcome to Offensive Web Testing Framework’s documentation!¶
Contents:
Installation¶
Prerequisites¶
There are few packages which are mandatory before you proceed
- Git client:
sudo apt-get install git
- Python 2.7, installed by default in most systems
Installation¶
There are two ways in which you can proceed:
Manual Installation¶
Manual installation of OWTF is nothing but cloning the repo and running the owtf setup.
git clone https://github.com/owtf/owtf.git cd owtf/ python setup.py install
Docker¶
Docker automates the task of setting up owtf doing all the bootstraping it needs. Just make sure that you have
docker
anddocker-compose
installed and run:docker-compose up
- If you wish to override the environment variables for docker setup, use the file named
owtf.env
Advanced Installation¶
If your distro is not officially supported in the install script, the following packages might not have been installed. So please make sure you atleast have the mandatory packages installed. Almost all the packages can be obtained using package manager of any major distro.
Mandatory¶
Optional Packages¶
- Tor (For Botnet mode)
- Proxychains (For Botnet mode)
owtf¶
owtf package¶
Subpackages¶
owtf.api package¶
Subpackages¶
-
class
owtf.api.handlers.base.
APIRequestHandler
(application, request, **kwargs)[source]¶ Bases:
tornado.web.RequestHandler
-
error
(message, data=None, code=None)[source]¶ An error occurred in processing the request, i.e. an exception was thrown.
Parameters: - data (A JSON-serializable object) – A generic container for any other information about the error, i.e. the conditions that caused the error, stack traces, etc.
- message (A JSON-serializable object) – A meaningful, end-user-readable (or at the least log-worthy) message, explaining what went wrong
- code (int) – A numeric code corresponding to the error, if applicable
-
fail
(data)[source]¶ There was a problem with the data submitted, or some pre-condition of the API call wasn’t satisfied.
Parameters: data (A JSON-serializable object) – Provides the wrapper for the details of why the request failed. If the reasons for failure correspond to POST values, the response object’s keys SHOULD correspond to those POST values.
-
success
(data)[source]¶ When an API call is successful, the JSend object is used as a simple envelope for the results, using the data key.
Parameters: data (A JSON-serializable object) – Acts as the wrapper for any data returned by the API call. If the call returns no data, data should be set to null.
-
-
class
owtf.api.handlers.config.
ConfigurationHandler
(application, request, **kwargs)[source]¶ Bases:
owtf.api.handlers.base.APIRequestHandler
Update framework settings and tool paths.
-
SUPPORTED_METHODS
= ['GET', 'PATCH']¶
-
get
()[source]¶ Return all configuration items.
Example request:
GET /api/v1/configuration HTTP/1.1 Accept: application/json
Example response:
HTTP/1.1 200 OK Vary: Accept-Encoding [ { "dirty":false, "section":"AUX_PLUGIN_DATA", "value":"report", "descrip":"Filename for the attachment to be sent", "key":"ATTACHMENT_NAME" }, { "dirty":false, "section":"DICTIONARIES", "value":"hydra", "descrip":"", "key":"BRUTEFORCER" } ]
-
patch
()[source]¶ Update configuration item
Example request:
PATCH /api/v1/configuration/ HTTP/1.1 Accept: */* Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Requested-With: XMLHttpRequest
Example response:
HTTP/1.1 200 OK Vary: Accept-Encoding Content-Length: 0 Content-Type: text/html; charset=UTF-8
-
-
class
owtf.api.handlers.health.
HealthCheckHandler
(application, request, **kwargs)[source]¶ Bases:
owtf.api.handlers.base.APIRequestHandler
API server health check
-
SUPPORTED_METHODS
= ['GET']¶
-
-
class
owtf.api.handlers.index.
IndexHandler
(application, request, **kwargs)[source]¶ Bases:
owtf.api.handlers.base.UIRequestHandler
Serves the main webapp
-
SUPPORTED_METHODS
= ['GET']¶
-
get
(path)[source]¶ Render the homepage with all JavaScript and context.
Example request:
GET / HTTP/1.1 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Example response:
HTTP/1.1 200 OK Content-Encoding: gzip Vary: Accept-Encoding Server: TornadoServer/5.0.1 Content-Type: text/html; charset=UTF-8
-
Submodules¶
owtf.api.main module¶
owtf.api.reporter module¶
owtf.api.routes module¶
owtf.api.utils module¶
owtf.db package¶
Submodules¶
owtf.db.database module¶
owtf.db.models module¶
The SQLAlchemy models for every table in the OWTF DB.
-
class
owtf.db.models.
Command
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
end_time
¶
-
modified_command
¶
-
original_command
¶
-
plugin_key
¶
-
run_time
¶
-
start_time
¶
-
success
¶
-
target_id
¶
-
-
class
owtf.db.models.
ConfigSetting
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
descrip
¶
-
dirty
¶
-
key
¶
-
section
¶
-
value
¶
-
-
class
owtf.db.models.
Error
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
github_issue_url
¶
-
id
¶
-
owtf_message
¶
-
reported
¶
-
traceback
¶
-
user_message
¶
-
-
class
owtf.db.models.
GrepOutput
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
id
¶
-
name
¶
-
output
¶
-
target_id
¶
-
-
class
owtf.db.models.
Mapping
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
category
¶
-
mappings
¶
-
owtf_code
¶
-
-
class
owtf.db.models.
Plugin
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
attr
¶
-
code
¶
-
descrip
¶
-
file
¶
-
group
¶
-
key
¶
-
max_time
¶ Consider last 5 runs only, better performance and accuracy
-
min_time
¶ Consider last 5 runs only, better performance and accuracy
-
name
¶
-
outputs
¶
-
title
¶
-
type
¶
-
works
¶
-
-
class
owtf.db.models.
PluginOutput
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
date_time
¶
-
end_time
¶
-
error
¶
-
id
¶
-
output
¶
-
output_path
¶
-
owtf_rank
¶
-
plugin_code
¶
-
plugin_group
¶
-
plugin_key
¶
-
plugin_type
¶
-
run_time
¶
-
start_time
¶
-
status
¶
-
target_id
¶
-
user_notes
¶
-
user_rank
¶
-
-
class
owtf.db.models.
Resource
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
dirty
¶
-
id
¶
-
resource
¶
-
resource_name
¶
-
resource_type
¶
-
-
class
owtf.db.models.
Session
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
active
¶
-
id
¶
-
name
¶
-
targets
¶
-
-
class
owtf.db.models.
Target
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
alternative_ips
¶
-
commands
¶
-
host_ip
¶
-
host_name
¶
-
host_path
¶
-
id
¶
-
ip_url
¶
-
max_owtf_rank
¶
-
max_user_rank
¶
-
port_number
¶
-
poutputs
¶
-
scope
¶
-
target_url
¶
-
top_domain
¶
-
top_url
¶
-
transactions
¶
-
url_scheme
¶
-
urls
¶
-
works
¶
-
-
class
owtf.db.models.
TestGroup
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
code
¶
-
descrip
¶
-
group
¶
-
hint
¶
-
plugins
¶
-
priority
¶
-
url
¶
-
-
class
owtf.db.models.
Transaction
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
binary_response
¶
-
data
¶
-
grep_outputs
¶
-
id
¶
-
local_timestamp
¶
-
login
¶
-
logout
¶
-
method
¶
-
raw_request
¶
-
response_body
¶
-
response_headers
¶
-
response_size
¶
-
response_status
¶
-
scope
¶
-
session_tokens
¶
-
target_id
¶
-
time
¶
-
time_human
¶
-
url
¶
-
Module contents¶
owtf.filesrv package¶
Submodules¶
owtf.filesrv.handlers module¶
owtf.filesrv.main module¶
Module contents¶
owtf.http package¶
Submodules¶
owtf.http.requester module¶
owtf.http.transaction module¶
HTTP_Transaction is a container of useful HTTP Transaction information to simplify code both in the framework and the plugins.
-
class
owtf.http.transaction.
HTTPTransaction
(timer)[source]¶ Bases:
object
-
get_html_link
(link_name='')[source]¶ Get the HTML link to the transaction ID
Parameters: link_name (str) – Name of the link Returns: Formatted HTML link Return type: str
-
get_html_link_time
(link_name='')[source]¶ Get the HTML link to the transaction ID
Parameters: link_name (str) – Name of the link Returns: Formatted HTML link Return type: str
-
get_raw
()[source]¶ Get raw transaction request and response
Returns: Raw string with response and request Return type: str
-
get_raw_response
(with_status=True)[source]¶ Get the complete raw response
Parameters: with_status (bool) – Want status? Returns: Raw reponse Return type: str
-
get_raw_response_body
()[source]¶ Return raw response content
Returns: Raw response body Return type: str
-
get_raw_response_headers
(with_status=True)[source]¶ Get raw response headers for the transaction
Parameters: with_status (bool) – Want status? Returns: Raw response headers as a string Return type: str
-
import_proxy_req_resp
(request, response)[source]¶ Import proxy request and response
Parameters: - request –
- response –
Returns: Return type:
-
in_scope
()[source]¶ Check if the transaction is in scope
Returns: True if in scope, else False Return type: bool
-
init_data
(data)[source]¶ Sets the data for the transaction
Parameters: data (str) – Data to set Returns: None Return type: None
-
set_error
(error_message)[source]¶ Set the error message for a transaction
Parameters: error_message (str) – Message to set Returns: None Return type: None
-
set_id
(id, html_link_to_id)[source]¶ Sets the transaction id and format an HTML link
Parameters: - id (int) – transaction id
- html_link_to_id (str) – HTML link for the id
Returns: None
Return type: None
-
set_transaction
(found, request, response)[source]¶ Response can be “Response” for 200 OK or “Error” for everything else, we don’t care here.
Parameters: - found –
- request –
- response –
Returns: Return type:
-
set_transaction_from_db
(id, url, method, status, time, time_human, local_timestamp, request_data, raw_request, response_headers, response_size, response_body)[source]¶ Set the transaction from the DB
Parameters: - id –
- url –
- method –
- status –
- time –
- time_human –
- local_timestamp –
- request_data –
- raw_request –
- response_headers –
- response_size –
- response_body –
Returns: Return type:
-
Module contents¶
owtf.lib package¶
Submodules¶
owtf.lib.cli_options module¶
Main CLI processing machine
-
owtf.lib.cli_options.
parse_options
(cli_options, valid_groups, valid_types)[source]¶ Main arguments processing for the CLI
Parameters: - cli_options (dict) – CLI args Supplied by user
- valid_groups (list) – Plugin groups to chose from
- valid_types (list) – Plugin types to chose from
Returns: Return type:
owtf.lib.exceptions module¶
Declares the framework exceptions and HTTP errors
-
exception
owtf.lib.exceptions.
APIError
(message, code=400)[source]¶ Bases:
tornado.web.HTTPError
Exception for API-based errors
-
exception
owtf.lib.exceptions.
PluginsAlreadyLoaded
[source]¶ Bases:
owtf.lib.exceptions.PluginException
load_plugins() called twice.
-
exception
owtf.lib.exceptions.
PluginsDirectoryDoesNotExist
[source]¶ Bases:
owtf.lib.exceptions.PluginException
The specified plugin directory does not exist.
owtf.lib.filelock module¶
Implementation of a simple cross-platform file locking mechanism. This is a modified version of code retrieved on 2013-01-01 from http://www.evanfosmark.com/2009/01/cross-platform-file-locking-support-in-python. The original code was released under the BSD License, as is this modified version. Modifications in this version:
- Tweak docstrings for sphinx.
- Accept an absolute path for the protected file (instead of a file name relative to cwd).
- Allow timeout to be None.
- Fixed a bug that caused the original code to be NON-threadsafe when the same FileLock instance was shared by multiple threads in one process. (The original was safe for multiple processes, but not multiple threads in a single process. This version is safe for both cases.)
- Added
purge()
function.- Added
available()
function.- Expanded API to mimic
threading.Lock interface
: -__enter__
always callsacquire()
, and therefore blocks ifacquire()
was called previously. -__exit__
always callsrelease()
. It is therefore a bug to callrelease()
from within a context manager. - Addedlocked()
function. - Added blocking parameter toacquire()
method
# taken from https://github.com/ilastik/lazyflow/blob/master/lazyflow/utility/fileLock.py # original version from http://www.evanfosmark.com/2009/01/cross-platform-file-locking-support-in-python/
-
class
owtf.lib.filelock.
FileLock
(protected_file_path, timeout=None, delay=1, lock_file_contents=None)[source]¶ Bases:
object
A file locking mechanism that has context-manager support so you can use it in a
with
statement. This should be relatively cross compatible as it doesn’t rely onmsvcrt
orfcntl
for the locking.-
acquire
(blocking=True)[source]¶ - Acquire the lock, if possible. If the lock is in use, and blocking is False, return False.
- Otherwise, check again every self.delay seconds until it either gets the lock or exceeds timeout number of seconds, in which case it raises an exception.
Parameters: blocking (bool) – File blocked or not Returns: True if lock is acquired, else False Return type: bool
-
available
()[source]¶ Returns True iff the file is currently available to be locked.
Returns: True if lockfile is available Return type: bool
-
owtf.lib.owtf_process module¶
Module contents¶
owtf.managers package¶
Submodules¶
owtf.managers.command_register module¶
owtf.managers.config module¶
-
owtf.managers.config.
config_gen_query
(session, criteria)[source]¶ Generate query
Parameters: criteria (dict) – Filter criteria Returns: Return type:
-
owtf.managers.config.
derive_config_dict
(config_obj)[source]¶ Get the config dict from the obj
Parameters: config_obj – The config object Returns: Return type:
-
owtf.managers.config.
derive_config_dicts
(config_obj_list)[source]¶ Derive multiple config dicts
Parameters: config_obj_list (list) – List of all config objects Returns: List of config dicts Return type: list
-
owtf.managers.config.
get_all_config_dicts
(session, criteria=None)[source]¶ Get all config dicts for a criteria
Parameters: criteria (dict) – Filter criteria Returns: Config dict Return type: dict
-
owtf.managers.config.
get_all_tools
(session)[source]¶ Get all tools from the config DB
Returns: Config dict for all tools Return type: dict
-
owtf.managers.config.
get_config_val
(session, key)[source]¶ Get the value of the key from DB
Parameters: key (str) – Key to lookup Returns: Value Return type: str
-
owtf.managers.config.
get_replacement_dict
(session)[source]¶ Get the config dict
Returns: Replaced dict Return type: dict
-
owtf.managers.config.
get_sections_config
(session)[source]¶ Get all sections in from the config db
Returns: List of sections Return type: list
-
owtf.managers.config.
get_tcp_ports
(start_port, end_port)[source]¶ Get TCP ports from the config file
Parameters: - start_port (str) – Start port in a range
- end_port (str) – End port
Returns: Comma-separate string of tcp ports
Return type: str
-
owtf.managers.config.
get_udp_ports
(start_port, end_port)[source]¶ Get UDP ports from the config file
Parameters: - start_ort – Start port in a range
- end_port (str) – End port
Returns: Comma-separate string of udp ports
Return type: str
-
owtf.managers.config.
load_config_file
(file_path, fallback_file_path)[source]¶ Load YAML format configuration file
Parameters: - file_path (str) – The path to config file
- fallback_file_path (str) – The fallback path to config file
Returns: config_map
Return type: dict
-
owtf.managers.config.
load_framework_config
(default, fallback, root_dir, owtf_pid)[source]¶ Load framework configuration into a global dictionary.
Parameters: - default (str) – The path to config file
- fallback (int) – The fallback path to config file
- fallback – OWTF root directory
- fallback – PID of running program
Returns: None
Return type: None
owtf.managers.error module¶
Component to handle data storage and search of all errors
-
owtf.managers.error.
add_error
(session, message, trace)[source]¶ Add an error to the DB
Parameters: - message (str) – Message to be added
- trace (str) – Traceback
Returns: None
Return type: None
-
owtf.managers.error.
delete_error
(session, error_id)[source]¶ Deletes an error from the DB
Parameters: error_id (int) – ID of the error to be deleted Returns: None Return type: None
-
owtf.managers.error.
derive_error_dict
(error_obj)[source]¶ Get the error dict from an object
Parameters: error_obj – Error object Returns: Error dict Return type: dict
-
owtf.managers.error.
derive_error_dicts
(error_obj_list)[source]¶ Get error dicts for a list of error objs
Parameters: error_obj_list (list) – List of error objects Returns: List of error dicts Return type: list
-
owtf.managers.error.
gen_query_error
(session, criteria)[source]¶ Generates the ORM query using the criteria
Parameters: criteria (dict) – Filter criteria Returns: Return type:
-
owtf.managers.error.
get_all_errors
(session, criteria=None)[source]¶ Get all error dicts based on criteria
Parameters: criteria (dict) – Filter criteria Returns: Error dicts Return type: list
owtf.managers.mapping module¶
Manages the mapping between different plugin groups and codes
-
owtf.managers.mapping.
derive_mapping_dict
(obj)[source]¶ Fetch the mapping dict from an object
Parameters: obj – The mapping object Returns: Mappings dict Return type: dict
-
owtf.managers.mapping.
derive_mapping_dicts
(obj_list)[source]¶ Fetches the mapping dicts based on the objects list
Parameters: obj_list (list) – The plugin object list Returns: Mapping dicts as a list Return type: list
-
owtf.managers.mapping.
get_all_mappings
(session)[source]¶ Create a mapping between OWTF plugins code and OWTF plugins description.
Returns: Mapping dictionary {code: [mapped_code, mapped_description], code2: [mapped_code, mapped_description], …} Return type: dict
-
owtf.managers.mapping.
get_mapping_category
(session, plugin_code)[source]¶ Get the categories for a plugin code
Parameters: plugin_code (int) – The code for the specific plugin Returns: category for the plugin code Return type: str
-
owtf.managers.mapping.
get_mapping_types
()[source]¶ In memory data saved when loading db :return: None :rtype: None
-
owtf.managers.mapping.
get_mappings
(session, mapping_type)[source]¶ Fetches mappings from DB based on mapping type
Parameters: mapping_type (str) – Mapping type like OWTF, OWASP (v3, v4, Top 10), NIST, CWE Returns: Mappings Return type: dict
-
owtf.managers.mapping.
load_mappings
(session, default, fallback)[source]¶ Loads the mappings from the config file
Note
This needs to be a list instead of a dictionary to preserve order in python < 2.7
Parameters: - session (object) – SQLAlchemy database session
- default (str) – The fallback path to config file
- fallback (str) – The path to config file
Returns: None
Return type: None
owtf.managers.plugin module¶
This module manages the plugins and their dependencies
-
owtf.managers.plugin.
derive_plugin_dict
(obj)[source]¶ Fetch the plugin dict from an object
Parameters: obj – Plugin object Returns: Plugin dict Return type: dict
-
owtf.managers.plugin.
derive_plugin_dicts
(obj_list)[source]¶ Fetch plugin dicts from a obj list
Parameters: obj_list (list) – List of plugin objects Returns: List of plugin dicts Return type: list
-
owtf.managers.plugin.
derive_test_group_dict
(obj)[source]¶ Fetch the test group dict from the obj
Parameters: obj – The test group object Returns: Test group dict Return type: dict
-
owtf.managers.plugin.
derive_test_group_dicts
(obj_list)[source]¶ Fetch the test group dicts from the obj list
Parameters: obj_list (list) – The test group object list Returns: Test group dicts in a list Return type: list
-
owtf.managers.plugin.
get_all_plugin_dicts
(session, criteria=None)[source]¶ Get plugin dicts based on filter criteria
Parameters: criteria (dict) – Filter criteria Returns: List of plugin dicts Return type: list
-
owtf.managers.plugin.
get_all_plugin_groups
(session)[source]¶ Get all plugin groups from the DB
Returns: List of available plugin groups Return type: list
-
owtf.managers.plugin.
get_all_plugin_types
(session)[source]¶ Get all plugin types from the DB
Returns: All available plugin types Return type: list
-
owtf.managers.plugin.
get_all_test_groups
(session)[source]¶ Get all test groups from th DB
Returns: Return type:
-
owtf.managers.plugin.
get_groups_for_plugins
(session, plugins)[source]¶ Gets available groups for selected plugins
Parameters: plugins (list) – Plugins selected Returns: List of available plugin groups Return type: list
-
owtf.managers.plugin.
get_plugins_by_group
(session, plugin_group)[source]¶ Get plugins by plugin group
Parameters: plugin_group (str) – Plugin group Returns: List of plugin dicts Return type: list
-
owtf.managers.plugin.
get_plugins_by_group_type
(session, plugin_group, plugin_type)[source]¶ Get plugins by group and plugin type
Parameters: - plugin_group (str) – Plugin group
- plugin_type (str) – plugin type
Returns: List of plugin dicts
Return type: list
-
owtf.managers.plugin.
get_plugins_by_type
(session, plugin_type)[source]¶ Get plugins based on type argument
Parameters: plugin_type (str) – Plugin type Returns: List of plugin dicts Return type: list
-
owtf.managers.plugin.
get_test_group
(session, code)[source]¶ Get the test group based on plugin code
Parameters: code (str) – Plugin code Returns: Test group dict Return type: dict
-
owtf.managers.plugin.
get_test_groups_config
(file_path)[source]¶ Reads the test groups from a config file
Note
This needs to be a list instead of a dictionary to preserve order in python < 2.7
Parameters: file_path (str) – The path to the config file Returns: List of test groups Return type: list
-
owtf.managers.plugin.
get_types_for_plugin_group
(session, plugin_group)[source]¶ Get available plugin types for a plugin group
Parameters: plugin_group (str) – Plugin group Returns: List of available plugin types Return type: list
-
owtf.managers.plugin.
load_plugins
(session)[source]¶ Loads the plugins from the filesystem and updates their info.
Note
Walks through each sub-directory of PLUGINS_DIR. For each file, loads it thanks to the imp module. Updates the database with the information for each plugin:
- ‘title’: the title of the plugin
- ‘name’: the name of the plugin
- ‘code’: the internal code of the plugin
- ‘group’: the group of the plugin (ex: web)
- ‘type’: the type of the plugin (ex: active, passive, …)
- ‘descrip’: the description of the plugin
- ‘file’: the filename of the plugin
- ‘internet_res’: does the plugin use internet resources?
Returns: None Return type: None
-
owtf.managers.plugin.
load_test_groups
(session, file_default, file_fallback, plugin_group)[source]¶ Load test groups into the DB.
Parameters: - test_groups_file (str) – The path to the test groups config
- plugin_group (str) – Plugin group to load
Returns: None
Return type: None
owtf.managers.poutput module¶
owtf.managers.resource module¶
owtf.managers.session module¶
owtf.managers.target module¶
owtf.managers.transaction module¶
owtf.managers.url module¶
owtf.managers.worker module¶
owtf.managers.worklist module¶
Module contents¶
owtf.plugin package¶
Submodules¶
owtf.plugin.plugin_handler module¶
owtf.plugin.plugin_helper module¶
owtf.plugin.plugin_params module¶
owtf.plugin.scanner module¶
owtf.protocols package¶
Submodules¶
owtf.protocols.smb module¶
owtf.protocols.smtp module¶
Description: This is the OWTF SMTP handler, to simplify sending emails.
Module contents¶
owtf.proxy package¶
Submodules¶
owtf.proxy.cache_handler module¶
Inbound Proxy Module developed by Bharadwaj Machiraju (blog.tunnelshade.in) as a part of Google Summer of Code 2013
-
class
owtf.proxy.cache_handler.
CacheHandler
(cache_dir, request, cookie_regex, blacklist)[source]¶ Bases:
object
This class will be used by the request handler to either load or dump to cache. Main things that are done here :- * The request_hash is generated here * The file locks are managed here * .rd files are created here
-
calculate_hash
(callback=None)[source]¶ Based on blacklist boolean the cookie regex is used for filtering of cookies in request_hash generation. However the original request is not tampered.
Parameters: callback – Callback function Returns: Return type:
-
create_response_object
()[source]¶ Create a proxy response object from cache file
Returns: Return type:
-
-
class
owtf.proxy.cache_handler.
DummyObject
[source]¶ Bases:
object
This class is just used to create a fake response object
owtf.proxy.gen_cert module¶
Inbound Proxy Module developed by Bharadwaj Machiraju (blog.tunnelshade.in) as a part of Google Summer of Code 2013
-
owtf.proxy.gen_cert.
gen_signed_cert
(domain, ca_crt, ca_key, ca_pass, certs_folder)[source]¶ This function takes a domain name as a parameter and then creates a certificate and key with the domain name(replacing dots by underscores), finally signing the certificate using specified CA and returns the path of key and cert files. If you are yet to generate a CA then check the top comments
Parameters: - domain (str) – domain for the cert
- ca_crt (str) – ca.crt file path
- ca_key (str) – ca.key file path
- ca_pass (str) – Password for the certificate
- certs_folder (str) –
Returns: Key and cert path
Return type: str
owtf.proxy.main module¶
owtf.proxy.proxy module¶
owtf.proxy.socket_wrapper module¶
Inbound Proxy Module developed by Bharadwaj Machiraju (blog.tunnelshade.in) as a part of Google Summer of Code 2013
-
owtf.proxy.socket_wrapper.
wrap_socket
(socket, domain, ca_crt, ca_key, ca_pass, certs_folder, success=None, failure=None, io=None, **options)[source]¶ Wrap an active socket in an SSL socket.
Parameters: - socket –
- domain –
- ca_crt –
- ca_key –
- ca_pass –
- certs_folder –
- success –
- failure –
- io –
- options –
Returns: Return type:
owtf.proxy.tor_manager module¶
TOR manager module developed by Marios Kourtesis <name.surname@gmail.com>
-
class
owtf.proxy.tor_manager.
TOR_manager
(args)[source]¶ Bases:
object
-
authenticate
()[source]¶ This function is handling the authentication process to TOR control connection.
Returns: Return type:
-
static
is_tor_running
()[source]¶ Check if tor is running
Returns: True if running, else False Return type: bool
-
renew_ip
()[source]¶ Sends an NEWNYM message to TOR control in order to renew the IP address
Returns: True if IP is renewed, else False Return type: bool
-
owtf.proxy.transaction_logger module¶
Module contents¶
owtf.shell package¶
Submodules¶
owtf.shell.async_subprocess module¶
# Inspired from: # http://code.activestate.com/recipes/440554-module-to-allow-asynchronous-subprocess-use-on-win/
-
class
owtf.shell.async_subprocess.
AsyncPopen
(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)[source]¶ Bases:
subprocess.Popen
owtf.shell.blocking_shell module¶
owtf.shell.interactive_shell module¶
owtf.shell.pexpect_shell module¶
Module contents¶
owtf.utils package¶
Submodules¶
owtf.utils.app module¶
owtf.utils.commands module¶
owtf.utils.error module¶
The error handler provides a centralised control for aborting the application and logging errors for debugging later.
-
owtf.utils.error.
abort_framework
(message)[source]¶ Abort the OWTF framework.
Warning: If it happens really early and framework.core.Core
has note been instantiated yet, sys.exit() is called with error code -1Parameters: message (str) – Descriptive message about the abort. Returns: full message explaining the abort. Return type: str
-
owtf.utils.error.
user_abort
(level, partial_output='')[source]¶ This function handles the next steps when a user presses Ctrl-C
Parameters: - level (str) – The level which was aborted
- partial_output (str) – Partial output generated by the command or plugin
Returns: Message to present to the user
Return type: str
-
owtf.utils.error.
get_option_from_user
(options)[source]¶ Give the user options to select
Parameters: options (str) – Set of available options for the user Returns: The different options for the user to choose from Return type: str
owtf.utils.file module¶
-
class
owtf.utils.file.
FileOperations
[source]¶ Bases:
object
-
static
codecs_open
(*args, **kwargs)[source]¶ Call the original function while checking for errors. If owtf_clean parameter is not explicitely passed or if it is set to True, it force OWTF to properly exit.
-
static
create_missing_dirs
(*args, **kwargs)[source]¶ Call the original function while checking for errors. If owtf_clean parameter is not explicitely passed or if it is set to True, it force OWTF to properly exit.
-
static
dump_file
(*args, **kwargs)[source]¶ Call the original function while checking for errors. If owtf_clean parameter is not explicitely passed or if it is set to True, it force OWTF to properly exit.
-
static
make_dirs
(*args, **kwargs)[source]¶ Call the original function while checking for errors. If owtf_clean parameter is not explicitely passed or if it is set to True, it force OWTF to properly exit.
-
static
mkdir
(*args, **kwargs)[source]¶ Call the original function while checking for errors. If owtf_clean parameter is not explicitely passed or if it is set to True, it force OWTF to properly exit.
-
static
-
owtf.utils.file.
catch_io_errors
(func)[source]¶ Decorator on I/O functions. If an error is detected, force OWTF to quit properly.
-
owtf.utils.file.
clean_temp_storage_dirs
(owtf_pid)[source]¶ Rename older temporary directory to avoid any further confusions.
Returns: Return type: None
-
owtf.utils.file.
cleanup_target_dirs
(target_url)[source]¶ Cleanup the directories for the specific target
Returns: None Return type: None
-
owtf.utils.file.
create_output_dir_target
(target_url)[source]¶ Creates output directories for the target URL
Parameters: target_url (str) – The target URL Returns: None Return type: None
-
owtf.utils.file.
create_temp_storage_dirs
(owtf_pid)[source]¶ Create a temporary directory in /tmp with pid suffix.
Returns: Return type: None
-
owtf.utils.file.
directory_access
(path, mode)[source]¶ Check if a directory can be accessed in the specified mode by the current user.
Parameters: - path (str) – Directory path.
- mode (str) – Access type.
Returns: Valid access rights
Return type: str
-
owtf.utils.file.
get_dir_worker_logs
()[source]¶ Returns the output directory for the worker logs
Returns: Path to output directory for the worker logs Return type: str
-
owtf.utils.file.
get_file_as_list
(filename)[source]¶ Get file contents as a list
Parameters: filename (str) – File path Returns: Output list of the content Return type: list
-
owtf.utils.file.
get_log_path
(process_name)[source]¶ Get the log file path based on the process name :param process_name: Process name :type process_name: str :return: Path to the specific log file :rtype: str
-
owtf.utils.file.
get_logs_dir
()[source]¶ Get log directory by checking if abs or relative path is provided in config file
-
owtf.utils.file.
get_output_dir
()[source]¶ Gets the output directory for the session
Returns: The path to the output directory Return type: str
owtf.utils.formatters module¶
CLI string formatting
-
class
owtf.utils.formatters.
ConsoleFormatter
(fmt=None, datefmt=None)[source]¶ Bases:
logging.Formatter
Custom formatter to show logging messages differently on Console
-
debug_fmt
= '\x1b[92m[*] {}\x1b[0m'¶
-
error_fmt
= '\x1b[91m[-] {}\x1b[0m'¶
-
format
(record)[source]¶ Choose format according to record level
Parameters: record (str) – Record to format Returns: Formatted string Return type: str
-
info_fmt
= '\x1b[94m[+] {}\x1b[0m'¶
-
warn_fmt
= '\x1b[93m[!] {}\x1b[0m'¶
-
owtf.utils.http module¶
-
owtf.utils.http.
container
(dec)[source]¶ Meta-decorator (for decorating decorators)
Keeps around original decorated function as a property
orig_func
Parameters: dec (function) – Decorator to decorate Returns: Decorated decorator
-
owtf.utils.http.
deep_update
(source, overrides)[source]¶ Update a nested dictionary or similar mapping.
Modify
source
in place.Return type: collections.Mapping
-
owtf.utils.http.
derive_http_method
(method, data)[source]¶ Derives the HTTP method from Data, etc
Parameters: - method (str) – Method to check
- data (str) – Data to check
Returns: Method found
Return type: str
-
owtf.utils.http.
extract_method
(wrapped_method)[source]¶ Gets original method if wrapped_method was decorated
Return type: any([types.FunctionType, types.MethodType])
owtf.utils.ip module¶
-
owtf.utils.ip.
get_ip_from_hostname
(hostname)[source]¶ Get IP from the hostname
Parameters: hostname (str) – Target hostname Returns: IP address of the target hostname Return type: str
-
owtf.utils.ip.
get_ips_from_hostname
(hostname)[source]¶ Get IPs from the hostname
Parameters: hostname (str) – Target hostname Returns: IP addresses of the target hostname as a list Return type: list
owtf.utils.logger module¶
-
class
owtf.utils.logger.
OWTFLogger
[source]¶ Bases:
object
-
disable_console_logging
(**kwargs)[source]¶ Disables console logging
Note
Must be called from inside the process because we should remove handler for that root logger. Since we add console handler in the last, we can remove the last handler to disable console logging
Parameters: kwargs (dict) – Additional arguments to the logger Returns: Return type: None
-
owtf.utils.process module¶
owtf.utils.pycompat module¶
Helpers for compatibility between Python 2.x and 3.x.
owtf.utils.strings module¶
-
owtf.utils.strings.
add_to_dict
(from_dict, to_dict)[source]¶ Add the items from dict a with copy attribute to dict b
Parameters: - from_dict (dict) – Dict to copy from
- to_dict (dict) – Dict to copy to
Returns: None
Return type: None
-
owtf.utils.strings.
get_as_list
(key_list)[source]¶ Get values for keys in a list
Parameters: key_list (list) – List of keys Returns: List of corresponding values Return type: list
-
owtf.utils.strings.
get_header_list
(key)[source]¶ Get list from a string of values for a key
Parameters: key (str) – Key Returns: List of values Return type: list
-
owtf.utils.strings.
get_random_str
(len)[source]¶ Function returns random strings of length len
Parameters: len (int) – Length Returns: Random generated string Return type: str
-
owtf.utils.strings.
is_convertable
(value, conv)[source]¶ Convert a value
Parameters: - value –
- conv –
Returns: Return type:
-
owtf.utils.strings.
list_to_dict_keys
(list)[source]¶ Convert a list to dict with keys from list items
Parameters: list (list) – list to convert Returns: The newly formed dictionary Return type: dict
-
owtf.utils.strings.
merge_dicts
(a, b)[source]¶ Returns a by-value copy contained the merged content of the 2 passed dictionaries
Parameters: - a (dict) – Dict a
- b (dict) – Dict b
Returns: New merge dict
Return type: dict
-
owtf.utils.strings.
multi_replace
(text, replace_dict)[source]¶ Recursive multiple replacement function :param text: Text to replace :type text: str :param replace_dict: The parameter dict to be replaced with :type replace_dict: dict :return: The modified text after replacement :rtype: str
-
owtf.utils.strings.
multi_replace_dict
(text, replace_dict)[source]¶ Perform multiple replacements in one go using the replace dictionary in format: { ‘search’ : ‘replace’ }
Parameters: - text (str) – Text to replace
- replace_dict (dict) – The replacement strings in a dict
Returns: str
Return type:
-
owtf.utils.strings.
pad_key
(key)[source]¶ Add delimiters.
Parameters: key (str) – Key to pad Returns: Padded key string Return type: str
-
owtf.utils.strings.
paths_exist
(path_list)[source]¶ Check if paths in the list exist
Parameters: path_list (list) – The list of paths to check Returns: True if valid paths, else False Return type: bool
-
owtf.utils.strings.
remove_blanks_list
(src)[source]¶ Removes empty elements from the list
Parameters: src (list) – List Returns: New list without blanks Return type: list
-
owtf.utils.strings.
scrub_output
(output)[source]¶ Remove all ANSI control sequences from the output
Parameters: output (str) – Output to scrub Returns: Scrubbed output Return type: str
-
owtf.utils.strings.
str2bool
(string)[source]¶ Converts a string to a boolean
Parameters: string (str) – String to convert Returns: Boolean equivalent Return type: bool
-
owtf.utils.strings.
str_to_dict
(string)[source]¶ Convert a string to a dict
Parameters: string (str) – String to convert Returns: Resultant dict Return type: dict
-
owtf.utils.strings.
strip_key
(key)[source]¶ Replaces key with empty space
Parameters: key (str) – Key to clear Returns: Empty key Return type: str
owtf.utils.timer module¶
The time module allows the rest of the framework to time how long it takes for certain actions to execute and present this information in both seconds and human-readable form.
-
class
owtf.utils.timer.
Timer
(datetime_format='%d/%m/%Y-%H:%M')[source]¶ Bases:
object
-
end_timer
(offset='0')[source]¶ Sets the end of the timer
Parameters: offset (str) – Timer index Returns: Return type: None
-
static
get_current_date_time
()[source]¶ Current timestamp
Returns: The current time as a timestamp Return type: datetime
-
get_current_date_time_as_str
()[source]¶ Returns a datetime object as a string in a particular format
Returns: Datetime object in string form Return type: str
-
get_elapsed_time
(offset='0')[source]¶ Gets the time elapsed between now and start of the timer in Unix epoch
Parameters: offset (str) – Timer index Returns: Time difference Return type: datetime
-
get_elapsed_time_as_str
(offset='0')[source]¶ Returns the time elapsed a nice readable string
Parameters: offset (str) – Timer index Returns: Time elapsed as a string Return type: str
-
get_end_date_time
(offset='0')[source]¶ Get the end time for the timer
Parameters: offset (str) – Timer index Returns: End time for the timer as a timestamp Return type: datetime
-
get_end_date_time_as_str
(offset='0')[source]¶ Get the end time for the timer as a string
Parameters: offset (str) – Timer index Returns: End time for the timer as a string Return type: str
-
get_start_date_time
(offset='0')[source]¶ Get the start time for the timer
Parameters: offset (str) – Timer index Returns: Start time for the timer as a timestamp Return type: datetime
-
get_start_date_time_as_str
(offset='0')[source]¶ Get the start time for the timer as a string
Parameters: offset (str) – Timer index Returns: Start time for the timer as a string Return type: str
-
get_time_as_str
(timedelta)[source]¶ Get the time difference as a human readable string
Parameters: timedelta (datetime.timedelta) – Time difference Returns: Human readable form for the timedelta Return type: str
-
get_time_human
(seconds_str)[source]¶ Generates the human readable string for the timestamp
Parameters: seconds_str (str) – Unix style timestamp Returns: Timestamp in a human readable string Return type: str
-
start_timer
(offset='0')[source]¶ Adds a start time to the timer
Parameters: offset (str) – Timer index Returns: The start time for the timer Return type: datetime
-
timers
= {}¶
-
Module contents¶
Submodules¶
owtf.config module¶
owtf.config¶
The Configuration object parses all configuration files, loads them into memory, derives some settings and provides framework modules with a central repository to get info.
owtf.core module¶
Configuration¶
Database Configuration¶
Basic Setup¶
The connection settings for postgres database are present in
~/.owtf/db.cfg
.DATABASE_IP: 127.0.0.1 DATABASE_PORT: 5432 DATABASE_NAME: owtfdb DATABASE_USER: owtf_db_user DATABASE_PASS: random_passwordNote
Before starting OWTF, make sure you have the postgres database server running. This can be easily ensured by using scripts/db_run.sh
Database & User Creation¶
Make use of
scripts/db_setup.sh
to create the postgres db and user if needed.sh scripts/db_setup.sh initDatabase & User Deletion¶
Make use of
scripts/db_setup.sh
to delete the postgres db and user when needed.sh scripts/db_setup.sh cleanFramework Configuration (Optional)¶
Some basic settings like, where should the interface server listen etc.. can be controlled from a config file present at
framework/config/framework_config.cfg
. All the default values are ready by default.
The address on which the interface server listens can be changed which will allow you to access the interface over any network.
# ------------------------- Interface Server ------------------------- # SERVER_ADDR: 0.0.0.0 UI_SERVER_PORT: 8009 FILE_SERVER_PORT: 8010
Usage¶
Starting OWTF¶
Warning
Before starting OWTF, make sure you have the postgres database server running. This can be easily ensured by using scripts/db_run.sh
OWTF is controlled and used from a web interface, so you have to launch OWTF from command line and then move on to your favourite browser. OWTF can be launched by
./owtf.pyThe interface url is printed onto the console, so that you can directly click on it
Using Sessions¶
In order to keep things simple and separate, OWTF provides support for sessions. A session is your classification of targets. You can have the same target in multiple sessions.
Managing Targets¶
The targets page also known as the target manager presents a ton of information. It has three important features
- A textarea to add new targets
- A targets table to go search through targets
- A session manager to manage sessions
- A button to launch plugins against targets
- A button to export targets to a text file - helpful when you have a large number of targets in scope
Remove Targets¶
To present the information in an orderly fashion, all targets are shown in the form of a table. The labels beside the target name shows the severity of any vulnerability discovered either by OWTF or by user (yes, user can have his own rankings)
Understanding Plugins¶
Types of Plugins¶
There are loads of plugins available in OWTF, but what is interesting is their categorization. All the plugins are categorized into multiple groups and types
- WEB
- active
- external
- grep
- passive
- semi-passive
- NET
- active
- bruteforce
- AUX
- se
- exploit etc…
Launching Plugins¶
Plugins can be launched from the targets table or from the individual target report. In order to launch plugins against multiple targets, select the targets from the target manager and launch plugins
Analyzing results¶
After the execution of plugins, you can navigate to the individual target report to go through the results of the plugins executed for that target. The report looks like this
Individual aspects for going through the report
Understanding plugin report¶
For better organization, all plugins of the same test code are grouped together. When you open a plugin report and click on a test code, you get to see the related plugins that are run for that target
Each test group has an expandable report. The text of the link consists of there parts
- Code of the test group as per the mapping (Eg: OWTF-CM-008)
- Name of the test group as per the mapping (Eg: HTTP Methods and XST)
- Pentester translations for the code (Eg: PUT,TRACE, WebDAV etc..)
Now if you proceed to select a plugin type, you can see the corresponding report
The details presented in a plugin report are:
- Run time of the plugin
- Time interval during which it was running
- Status of the plugin (i.e if it was aborted by user etc..)
- A button to rerun the plugin
- A button to delete the plugin output
- A button to add notes
- Actual plugin output
If you click on the Browse button, then any file saved by the plugin can be seen
Saving your analysis¶
Once you start analyzing the plugin results, there is a need for ranking those findings along with saving some necessary information if needed. OWTF has both these features
Notes¶
Ranking is not the only thing, you can also write and save notes as well. Click on the NOTES button to open an editor and once you are done, click on the same button to save and close the editor
Advanced Filter¶
Advanced filter is used to filter the plugin results. Click on the FILTER button in the target report and you are good to go
As it can be seen from above image, you can filter the plugin outputs based on multiple criteria. You can even change the mapping of the results. Let us try the latest OWASP v4
Transaction Log¶
All the transactions that ever happened through the OWTF proxy can be searched through transaction log. You can search in multiple fields. A sample look of the transaction log is in the image below.
There are two ways in which individual transactions can be viewed
- Each transaction in new tab
- Transaction in a modal window
Clicking on the info button will open a model window which allows you to navigate back & forth between the filtered transactions. The search words are highlighted as well.
Managing Workers¶
Workers are the actual processes that run the plugins. Control over these worker processes is provided from the worker manager page.
There are three main controls in the worker manager:
Pausing/Resuming Workers¶
You can pause/resume all the workers at the same time or pause them individually through the workers page. We care a lot about your time. If your Internet connection down or if any target is not responding and your web vulnerability scanner plugin is halfway through? Don’t worry, we got your back. All you have to do is pause the worker and resume it when the target is back up. Isn’t this l33t?
Abort Workers¶
You can abort any worker. If you wish to abort any plugin during execution, just click on the red cross. Do the same if you wish to remove an extra idle worker.
Add Workers¶
You can add new workers on the fly if you have many targets and are running many plugins simultaneously.
Warning
Maximum of one plugin per target will be running at any moment in time
Controlling Worklist¶
- work
- When any plugin is launched against a target, it adds a (plugin, target) combination to the worklist. This combination is known as work.
- worklist
- The list consisting of all work which are yet to be assigned to a worker.
Worklist can be managed from the worklist manager which looks like this
Worklist table provides interesting information like:
- Estimated time for which the plugin will run
- All details about the plugin and the target against which it is launched
Pausing Work¶
Individual works or whole worklist can be paused. This will stop the work from getting assigned to any worker. The interesting part is
worklist is persistent
,i.e. if you pause the whole worklist and exit OWTF, the works will still be there in paused state when you start OWTF again.Deleting Work¶
Any work can be deleted from the worklist. The search boxes will help in filtering of the works when there are many entries.
Troubleshooting¶
Unable to install pycurl library, getting main.ConfigurationError: Could not run curl-config?
Luckily, we have faced this issue. If you ran the install script and still got this error, you can let us know. If not, check this issue on how to fix it.
Unable to run OWTF because of ImportError: No module named cryptography.hazmat.bindings.openssl.binding?
This actually means you do not have cryptography python module installed. It is recommended to rerun the install script (or) to just install the missing python libraries using the following command.
pip2 install --upgrade -r install/owtf.pip
Unable to run OWTF because of TypeError: parse_requirements() missing 1 required keyword argument: ‘session’
This is because of an older version of pip installed in your System. To resolve this run the following commands
pip install --upgrade pip (run as root if required) python install/install.py