Common OVF Tool (COT)

Introduction

Latest Version Python Versions Supported License Build Status Codecov Documentation Status

COT (the Common OVF Tool) is a tool for editing Open Virtualization Format (.ovf, .ova) virtual appliances, with a focus on virtualized network appliances such as the Cisco CSR 1000V and Cisco IOS XRv platforms.

COT's capabilities include:

  • Add a disk or other file to an OVF/OVA
  • Edit OVF hardware information (CPUs, RAM, NICs, configuration profiles, etc.)
  • Edit product description information in an OVF/OVA
  • Edit OVF environment properties
  • Display a descriptive summary of the contents of an OVA or OVF package
  • Embed a bootstrap configuration text file into an OVF/OVA.
  • Remove files and disks from an OVF or OVA package
  • Deploy an OVF/OVA to an ESXi (VMware vSphere or vCenter) server to provision a new virtual machine (VM).

Examples

Displaying a summary of OVA contents:

> cot info --brief csr1000v-universalk9.03.17.01.S.156-1.S1-std.ova
-------------------------------------------------------------------------
csr1000v-universalk9.03.17.01.S.156-1.S1-std.ova
COT detected platform type: Cisco CSR1000V
-------------------------------------------------------------------------
Product:  Cisco CSR 1000V Cloud Services Router
Vendor:   Cisco Systems, Inc.
Version:  03.17.01.S.156-1.S1-std

Files and Disks:                 File Size  Capacity Device
                                 --------- --------- --------------------
  csr1000v_harddisk.vmdk          71.5 KiB     8 GiB harddisk @ SCSI 0:0
  bdeo.sh                        52.42 KiB
  README-OVF.txt                 8.534 KiB
  README-BDEO.txt                6.748 KiB
  cot.tgz                        116.8 KiB
  csr1000v-universalk9.03.17....   425 MiB           cdrom @ IDE 1:0

Hardware Variants:
  System types:             vmx-08 vmx-09 vmx-10 vmx-11
                            Cisco:Internal:VMCloud-01
  SCSI device types:        virtio lsilogic
  Ethernet device types:    VMXNET3 virtio

Configuration Profiles:  CPUs    Memory NICs Serials Disks/Capacity
                         ---- --------- ---- ------- --------------
  1CPU-4GB (default)        1     4 GiB    3       2  1 /     8 GiB
  2CPU-4GB                  2     4 GiB    3       2  1 /     8 GiB
  4CPU-4GB                  4     4 GiB    3       2  1 /     8 GiB
  4CPU-8GB                  4     8 GiB    3       2  1 /     8 GiB

Networks:
  GigabitEthernet1  "Data network 1"
  GigabitEthernet2  "Data network 2"
  GigabitEthernet3  "Data network 3"

Environment:
  Transport types: iso

Properties:
  <config-version>                                        "1.0"
  Router Name                                             ""
  Login Username                                          ""
  Login Password                                          ""
  Management Interface                                    "GigabitEthernet1"
  Management VLAN                                         ""
  Management Interface IPv4 Address/Mask                  ""
  Management IPv4 Gateway                                 ""
  Management IPv4 Network                                 ""
  PNSC IPv4 Address                                       ""
  PNSC Agent Local Port                                   ""
  PNSC Shared Secret Key                                  ""
  Remote Management IPv4 Address (optional, deprecated)   ""
  Enable SCP Server                                       "false"
  Enable SSH Login and Disable Telnet Login               "false"
  Enable Password                                         ""
  Domain Name                                             ""
  License boot level                                      "ax"
  Console                                                 ""
  Resource template                                       "default"
  Intercloud Mode                                         ""
  Intercloud Mode Management Key                          ""
  Intercloud Control Port                                 ""
  Intercloud Tunnel Port                                  ""
  Intercloud Tunnel Header Size                           "148"
  Intercloud Tunnel Interface IPv4 Address                ""
  Intercloud Tunnel Interface Gateway IPv4 Address        ""

Adding a custom hardware configuration profile to an OVA:

> cot edit-hardware csr1000v.ova --output csr1000v_custom.ova \
      --profile 1CPU-4GB --cpus 1 --memory 4GB

Customizing OVF environment properties:

> cot edit-properties csr1000v.ova --output csr1000v_custom.ova \
      --properties mgmt-ipv4-addr=10.1.1.100/24 \
                   mgmt-ipv4-gateway=10.1.1.1

Installing COT

System requirements

  • COT requires either Python 2.7 or Python 3.3 or later.
  • COT is tested to work under Mac OS X and Ubuntu Linux and similar distros.
  • COT now has limited support for CentOS and related distros as well.

Installing COT using pip

Since COT is written in Python, it can be installed like any other Python package using the pip tool. For most users this is the recommended installation method.

sudo pip install cot

or, to install for the current user only (typically installing to ~/.local/):

pip install --user cot

If you have already installed COT and wish to update to the latest available version:

sudo pip install --upgrade cot

or

pip install --user --upgrade cot

Installing optional features

COT has a number of optional Python package dependencies that enable optional features. If you want to use these features, you can instruct pip to install them as part of the COT installation process, or you can install them separately after the fact.

  • Tab-completion of COT CLI parameters in bash, enabled with the argcomplete package.

    sudo pip install cot[tab-completion]

    or

    sudo pip install argcomplete

    Note

    After installing argcomplete by either method, you must configure your bash environment to enable it. Refer to the argcomplete documentation for the required steps.

Installing COT from source

If you wish to install bleeding-edge unreleased code or make code contributions of your own, you can install COT from source as described below.

Downloading COT

You can download COT via Git or using HTTP.

git clone git://github.com/glennmatthews/cot
cd cot

or

wget -O cot.tgz https://github.com/glennmatthews/cot/archive/master.tar.gz
tar zxf cot.tgz
cd cot-master

or

curl -o cot.tgz https://github.com/glennmatthews/cot/archive/master.tar.gz
tar zxf cot.tgz
cd cot-master

Install the COT libraries and script

sudo python setup.py install

Troubleshooting

"ValueError: Expected version spec"

If you get an error like

ValueError: ('Expected version spec in', 'enum34; python_version < "3.4"', 'at', '; python_version < "3.4"')

then you may need to update your version of pip and/or setuptools:

sudo pip install --upgrade pip setuptools

Installing helper programs

Certain COT features require helper programs - you can install these as part of the COT installation process, or they can be installed as-needed by COT:

  • COT uses qemu-img as a helper program for various operations involving the creation, inspection, and modification of hard disk image files packaged in an OVF.
  • The cot add-disk command requires either qemu-img (version 2.1 or later) or vmdktool as a helper program when adding hard disks to an OVF.
  • The cot inject-config command requires mkisofs (or its fork genisoimage) and/or xorriso to create ISO (CD-ROM) images for platforms that use ISOs to package the configuration.
  • Similarly, for platforms using hard disks for bootstrap configuration, cot inject-config requires fatdisk to format hard disk images.
  • The cot deploy ... esxi command requires ovftool to communicate with an ESXi server. If ovftool is installed, COT's automated unit tests will also make use of ovftool to perform additional verification that OVFs and OVAs created by COT align with VMware's expectations for these file types.

COT can attempt to install these tools using the appropriate package manager for your platform (i.e., MacPorts for Mac OS X, and either apt-get or yum for Linux).

Warning

Unfortunately, VMware requires a site login to download ovftool, so if you need this tool, you will have to install it yourself. COT cannot install it for you at present.

To let COT attempt to pre-install all of the above helpers, you can optionally run:

cot install-helpers

See here for more details.

If you skip this step, then when you are running COT, and it encounters the need for a helper that has not been installed, COT will prompt you to allow it to install the helper in question.

Using COT

Getting CLI help

Synopsis

cot --help
cot --version
cot help <command>
cot <command> --help
cot <options> <command> <command-options>

Description

Common OVF Tool (COT), version 2.1.0+2.g01b1446.dirty

A tool for editing Open Virtualization Format (.ovf, .ova) virtual appliances, with a focus on virtualized network appliances such as the Cisco CSR 1000V and Cisco IOS XRv platforms.

You can always get detailed help for COT by running cot --help, cot <command> --help, or cot help <command>.

Options

-h, --help show this help message and exit
-V, --version show program's version number and exit
-f, --force Perform requested actions without prompting for confirmation
-q, --quiet Decrease verbosity of the program (repeatable)
-v, --verbose Increase verbosity of the program (repeatable)
Commands
add-disk
Add a disk image to an OVF package and map it as a disk in the guest environment
add-file
Add a file to an OVF package
deploy
Create a new VM on the target hypervisor from the given OVF or OVA
edit-hardware
Edit virtual machine hardware properties of an OVF
edit-product
Edit product info in an OVF
edit-properties
Edit or create environment properties of an OVF
help
Print help for a command
info
Generate a description of an OVF package
inject-config
Inject a configuration file into an OVF package
install-helpers
Install/verify COT manual pages and any third- party helper programs that COT may require
remove-file
Remove a file from an OVF package

Verifying and installing helper programs with cot install-helpers

Synopsis

cot install-helpers --help
cot <opts> install-helpers --verify-only
cot <opts> install-helpers [--ignore-errors]

Description

Install or verify the installation of COT manual pages and various required third-party helper programs for COT.

Options

-h, --help show this help message and exit
--verify-only Only verify helpers -- do not attempt to install any missing helpers.
-i, --ignore-errors
 Do not fail even if helper installation fails.

Examples

Verify whether COT can find all expected helper programs

> cot install-helpers --verify-only
Results:
-------------
COT manpages: present in /usr/share/man/man1/
fatdisk:      present at /opt/local/bin/fatdisk
mkisofs:      present at /opt/local/bin/mkisofs
ovftool:      present at /usr/local/bin/ovftool
qemu-img:     present at /opt/local/bin/qemu-img
vmdktool:     NOT FOUND

Have COT attempt to install missing helpers for you. Note that most helpers require administrator / sudo privileges to install. If any installation fails, COT will exit with an error, unless you pass --ignore-errors.

> cot install-helpers
(...)
Results:
-------------
COT manpages: successfully installed to /usr/share/man
fatdisk:      successfully installed to /usr/local/bin/fatdisk
mkisofs:      present at /usr/bin/mkisofs
ovftool:      INSTALLATION FAILED: No support for automated
              installation of ovftool, as VMware requires a site
              login to download it. See
              https://www.vmware.com/support/developer/ovf/
qemu-img:     present at /usr/bin/qemu-img
vmdktool:     successfully installed to /usr/local/bin/vmdktool

[Errno 1] Unable to install some helpers

Warning

Unfortunately, VMware requires a site login to download ovftool, so if you need this tool, you will have to install it yourself. COT cannot install it for you at present.

Inspecting OVF contents with cot info

Synopsis

cot info --help
cot info [-b | -v] PACKAGE [PACKAGE ...]

Description

Show a summary of the contents of the given OVF(s) and/or OVA(s).

Options

PACKAGE [PACKAGE ...]
OVF descriptor(s) and/or OVA file(s) to describe
-h, --help show this help message and exit
-b, --brief Brief output (shorter)
-v, --verbose Verbose output (longer)

Updating product information with cot edit-product

Synopsis

cot edit-product --help
cot <opts> edit-product PACKAGE [-o OUTPUT] [-c PRODUCT_CLASS]
                        [-p PRODUCT] [-n VENDOR] [-v SHORT_VERSION]
                        [-V FULL_VERSION] [-u PRODUCT_URL ]
                        [-r VENDOR_URL] [-l APPLICATION_URL]

Description

Edit product information attributes of the given OVF or OVA

Options

PACKAGE
OVF descriptor or OVA file to edit
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new OVF/OVA package to create instead of updating the existing OVF
-c PRODUCT_CLASS, --product-class PRODUCT_CLASS
 Product class, such as "com.cisco.csr1000v"
-p PRODUCT, --product PRODUCT
 Product name string, such as "Cisco IOS-XE"
-n VENDOR, --vendor VENDOR
 Vendor string, such as "Cisco Systems, Inc."
-v SHORT_VERSION, --version SHORT_VERSION
 Software short version string, such as "15.3(4)S" or "5.2.0.01I"
-V FULL_VERSION, --full-version FULL_VERSION
 Software long version string, such as "Cisco IOS-XE Software, Version 15.3(4)S"
-u PRODUCT_URL, --product-url PRODUCT_URL
 Product URL, such as "http://www.cisco.com/go/iosxrv"
-r VENDOR_URL, --vendor-url VENDOR_URL
 Vendor URL, such as "http://www.cisco.com"
-l APPLICATION_URL, --application-url APPLICATION_URL
 Application URL, such as "https://router1:530/"

Adding disks to an OVF with cot add-disk

Synopsis

cot add-disk --help
cot <opts> add-disk DISK_IMAGE PACKAGE [-o OUTPUT] [-f FILE_ID]
                    [-t {harddisk,cdrom}] [-c {ide,sata,scsi}]
                    [-s SUBTYPE] [-a ADDRESS] [-d DESCRIPTION]
                    [-n DISKNAME]

Description

Add or replace a disk image in the specified OVF or OVA. If the specified disk image, controller/address, file-id, and/or instance match an existing entry in the OVF, will replace the existing disk with the provided file (prompting for confirmation if --force was not set); otherwise, will create a new disk entry.

Options

DISK_IMAGE
Disk image file to add to the package
PACKAGE
OVF descriptor or OVA file to edit
General options
-h, --help Show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new OVF/OVA package to create instead of updating the existing OVF
Descriptive options
-d DESCRIPTION, --description DESCRIPTION
 Description of this disk (optional)
-n DISKNAME, --name DISKNAME
 Name of this disk (default: "Hard disk #" or "CD-ROM #" as appropriate)

Packaging additional files into an OVF with cot add-file

Synopsis

cot add-file --help
cot <opts> add-file FILE PACKAGE [-o OUTPUT] [-f FILE_ID]

Description

Add or replace a file in the given OVF. If the specified file and/or file-id match existing package contents, will replace it (prompting for confirmation if --force was not set); otherwise, will create a new file entry.

Options

FILE
File to add to the package
PACKAGE
Package, OVF descriptor or OVA file to edit
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new VM package to create instead of updating the existing package
-f FILE_ID, --file-id FILE_ID
 File ID string within the package (default: same as filename)

Removing files from an OVF with cot remove-file

Synopsis

cot remove-file --help
cot <opts> remove-file [-f FILE_PATH] [-i FILE_ID] PACKAGE
                       [-o OUTPUT]

Description

Remove a file from the given OVF. Will prompt for confirmation unless --force is set.

Options

PACKAGE
Package, OVF descriptor or OVA file to edit
General options
-h, --help Show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new OVF/OVA package to create instead of updating the existing OVF
File selection options
-f FILE_PATH, --file-path FILE_PATH
 File name or path within the package
-i FILE_ID, --file-id FILE_ID
 File ID string within the package

Customizing hardware profiles with cot edit-hardware

Synopsis

cot edit-hardware --help
cot <opts> edit-hardware PACKAGE [-o OUTPUT] -v TYPE [TYPE2 ...]
cot <opts> edit-hardware PACKAGE [-o OUTPUT]
                         [-p PROFILE [PROFILE2 ...]
                         [--delete-all-other-profiles]] [-c CPUS]
                         [-m MEMORY] [-n NICS]
                         [--nic-types TYPE [TYPE2 ...]]
                         [-N NETWORK [NETWORK2 ...]]
                         [-M MAC1 [MAC2 ...]]
                         [--nic-names NAME1 [NAME2 ...]]
                         [-s SERIAL_PORTS] [-S URI1 [URI2 ...]]
                         [--scsi-subtypes TYPE [TYPE2 ...]]
                         [--ide-subtypes TYPE [TYPE2 ...]]

Description

Edit hardware properties of the specified OVF or OVA

Options

PACKAGE
OVF descriptor or OVA file to edit
General options
-h, --help Show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new OVF/OVA package to create instead of updating the existing OVF
-v <TYPE...>, --virtual-system-type <TYPE...>
 Change virtual system type(s) supported by this OVF/OVA package.
-p <PROFILE...>, --profiles <PROFILE...>
 Make hardware changes only under the given configuration profile(s). (default: changes apply to all profiles)
--delete-all-other-profiles
 Delete all configuration profiles other than those specified with the --profiles option
Computational hardware options
-c CPUS, --cpus CPUS
 Set the number of CPUs.
-m MEMORY, --memory MEMORY
 Set the amount of RAM. (Examples: "4096M", "4 GiB")
Network interface options
-n NICS, --nics NICS
 Set the number of NICs.
--nic-types <TYPE...>
 Set the hardware type(s) for all NICs. (default: do not change existing NICs, and new NICs added will match the existing type(s).)
--nic-names <NAME1...>
 Specify a list of one or more NIC names or patterns to apply to NIC devices. See Notes.
-N <NETWORK...>, --nic-networks <NETWORK...>
 Specify a series of one or more network names or patterns to map NICs to. See Notes.
--network-descriptions <NAME1...>
 Specify a list of one or more network descriptions or patterns to apply to the networks. See Notes.
-M <MAC1...>, --mac-addresses-list <MAC1...>
 Specify a list of MAC addresses for the NICs. If N MACs are specified, the first (N-1) NICs will receive the first (N-1) MACs, and all remaining NICs will use the Nth MAC
Serial port options
-s SERIAL_PORTS, --serial-ports SERIAL_PORTS
 Set the number of serial ports.
-S <URI1...>, --serial-connectivity <URI1...>
 Specify a series of connectivity strings (URIs such as "telnet://localhost:9101") to map serial ports to. If fewer URIs than serial ports are specified, the remaining ports will be unmapped.
Disk and disk controller options
--scsi-subtypes <TYPE...>
 Set resource subtype(s) (such as "lsilogic" or "virtio") for all SCSI controllers. If an empty string is provided, any existing subtype will be removed.
--ide-subtypes <TYPE...>
 Set resource subtype(s) (such as "virtio") for all IDE controllers. If an empty string is provided, any existing subtype will be removed.

Notes

The --nic-names, --nic-networks, and --network-descriptions options support the use of a wildcard value to automatically generate a series of consecutively numbered strings. The syntax for the wildcard option is '{' followed by a number to start incrementing from, followed by '}'. See examples below.

Examples

Create a new profile named "1CPU-8GB" with 1 CPU and 8 gigabytes of RAM

cot edit-hardware csr1000v.ova --output csr1000v_custom.ova \
    --profile 1CPU-4GB --cpus 1 --memory 8GB

Wildcard example - without caring about how many NICs are defined in the input OVA, rename all of the NICs in the output OVA as 'Ethernet0/10', 'Ethernet0/11', 'Ethernet0/12', etc., and map them to networks 'Ethernet0_10', 'Ethernet0_11', 'Ethernet0_12', etc., which are described as 'Data network 1', 'Data network 2', etc.

cot edit-hardware input.ova -o output.ova \
    --nic-names "Ethernet0/{10}" \
    --nic-networks "Ethernet0_{10}" \
    --network-descriptions "Data network {1}"

Combination of fixed and wildcarded names - rename the NICs in the output OVA as 'mgmt', 'eth0', 'eth1', 'eth2'...

cot edit-hardware input.ova -o output.ova --nic-names "mgmt" \
    "eth{0}"

Customizing OVF environment settings with cot edit-properties

Synopsis

cot edit-properties --help
cot <opts> edit-properties PACKAGE
                           [-p KEY1=VALUE1 [-p KEY2=VALUE2 ...]]
                           [-l LABEL1 [-l LABEL2 ...]]
                           [-d DESC1 [-d DESC2 ...]] [-c CONFIG_FILE]
                           [-u [USER_CONFIGURABLE]]
                           [-t TRANSPORT [TRANSPORT2 ...]]
                           [-o OUTPUT]
cot <opts> edit-properties PACKAGE [-u [USER_CONFIGURABLE]]
                           [-o OUTPUT]

Description

Configure environment properties of the given OVF or OVA. The user may specify keys and values as command-line arguments or may provide a config-file to read from. If neither --config-file, --properties, nor --transport are given, the program will run interactively.

Options

PACKAGE
OVF descriptor or OVA file to edit
General options
-h, --help Show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new OVF/OVA package to create instead of updating the existing OVF
Property setting options
-u <USER_CONFIGURABLE>, --user-configurable <USER_CONFIGURABLE>
 Update the 'userConfigurable' flag on all edited properties to True or the given value
-c CONFIG_FILE, --config-file CONFIG_FILE
 Read configuration CLI from this text file and generate generic properties for each line of CLI
-p <KEY1[=VALUE1][+TYPE1]...>, --properties <KEY1[=VALUE1][+TYPE1]...>
 Update or create the given property keys. A '=' delimits the optional value to set this key to. A '+' delimits the optional type to enforce for this key. This argument may be repeated as needed to specify multiple properties to edit.
-l <LABEL1...>, --labels <LABEL1...>
 Set the label(s) for the property(s) being edited. If this option is specified, the number of properties and the number of labels must be equal.
-d <DESC1...>, --descriptions <DESC1...>
 Set the description(s) for the property(s) being edited. If this option is specified, the number of properties and the number of descriptions must be equal.
-t <TRANSPORT...>, --transports <TRANSPORT...>
 Set the transport method(s) for properties. Known values are 'iso', 'vmware', and 'ibm', or an arbitrary URI may be specified.

Examples

Add configuration from a text file and mark the resulting properties as non-user-configurable.

cot edit-properties input.ovf -c config.txt -u=0

Add/update two properties, one a string with no default value and the other a boolean defaulting to true, and mark both properties as user-configurable.

cot edit-properties input.ovf -p string-property+string \
    -p bool-property=true+boolean --user-configurable

Update the label and description of two existing properties

cot edit-properties input.ovf -p hostname -l "Hostname" \
    -d "Hostname of this device" -p enable-ssh -l "Enable \
    remote SSH access" -d "Enable sshd and disable telnetd"

Embedding bootstrap configuration with cot inject-config

Synopsis

cot inject-config --help
cot <opts> inject-config PACKAGE [-o OUTPUT] [-c CONFIG_FILE]
                         [-s SECONDARY_CONFIG_FILE]
                         [-e EXTRA_FILE [EXTRA_FILE2 ...]]

Description

Add one or more "bootstrap" configuration file(s) to the given OVF or OVA. These files will be packaged into a virtual hard disk, or virtual CD-ROM, as appropriate to the target platform. Any specified primary and secondary config files will be renamed if necessary to meet expectations of the target platform, while any files provided with the --extra-files option will be included as-is and will not be renamed.

Options

PACKAGE
Package, OVF descriptor or OVA file to edit
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
 Name/path of new VM package to create instead of updating the existing package
-c CONFIG_FILE, --config-file CONFIG_FILE
 Text file to embed as primary configuration
-s SECONDARY_CONFIG_FILE, --secondary-config-file SECONDARY_CONFIG_FILE
 Text file to embed as secondary configuration (currently only used for IOS XR admin config)
-e <EXTRA_FILE...>, --extra-files <EXTRA_FILE...>
 Additional file(s) to include as-is

Deploying an OVF to create a VM with cot deploy

Synopsis

cot deploy --help
cot <opts> deploy PACKAGE esxi ...

Description

Deploy an OVF or OVA to create a virtual machine on a specified server.

Options

PACKAGE
OVF descriptor or OVA file
-h, --help show this help message and exit
Hypervisors
esxi
Deploy to ESXi, vSphere, or vCenter

Creating a VM on VMware vCenter/vSphere with cot deploy esxi

Synopsis

cot deploy PACKAGE esxi --help
cot <opts> deploy PACKAGE esxi LOCATOR [-u USERNAME] [-p PASSWORD]
                               [-c CONFIGURATION] [-n VM_NAME] [-P]
                               [-N OVF1=HOST1 [-N OVF2=HOST2 ...]]
                               [-S KIND1:VAL1[,OPTS1]
                               [-S KIND2:VAL2[,OPTS2] ...]]
                               [-d DATASTORE] [-o=OVFTOOL_ARGS]

Description

Deploy OVF/OVA to ESXi/vCenter/vSphere hypervisor

Options

LOCATOR
vSphere target locator. Examples: "192.0.2.100" (deploy directly to ESXi server), "192.0.2.101/mydatacenter/host/192.0.2.100" (deploy via vCenter server)
-h, --help show this help message and exit
-u USERNAME, --username USERNAME
 Server login username
-p PASSWORD, --password PASSWORD
 Server login password
-c CONFIGURATION, --configuration CONFIGURATION
 Use the specified configuration profile defined in the OVF. If unspecified and the OVF has multiple profiles, the user will be prompted or the default configuration will be used.
-n VM_NAME, --vm-name VM_NAME
 Name to use for the VM (if applicable) and any files created. If unspecified, the name of the OVF will be used.
-P, --power-on Power on the created VM to begin booting immediately.
-N <OVF_NET1=HOST_NET1...>, --network-map <OVF_NET1=HOST_NET1...>
 Map networks named in the OVF to networks (bridges, vSwitches, etc.) in the hypervisor environment. This argument may be repeated as needed to specify multiple mappings.
-S <K1:V1[,O1]...>, --serial-connection <K1:V1[,O1]...>
 Set connectivity for a serial port defined in the OVF. This argument may be repeated to specify more port connections. Each entry should be structured as 'kind:value' or 'kind:value,options'.
-d DATASTORE, --datastore DATASTORE
 ESXi datastore to use for the new VM
-o OVFTOOL_ARGS, --ovftool-args OVFTOOL_ARGS
 Quoted string describing additional CLI parameters to pass through to "ovftool". Examples: -o="--foo", --ovftool-args="--foo --bar"

Examples

Deploy to vSphere/ESXi server 192.0.2.100 with credentials admin/admin, creating a VM named 'test_vm' from foo.ova.

cot deploy foo.ova esxi 192.0.2.100 -u admin -p admin \
    -n test_vm

Deploy to vSphere/ESXi server 192.0.2.100, with username admin (prompting the user to input a password at runtime), creating a VM based on profile '1CPU-2.5GB' in foo.ova, and creating the serial port as a telnet server listening on port 10022 of the host

cot deploy foo.ova esxi 192.0.2.100 -u admin -c 1CPU-2.5GB \
    -S telnet://:10022,server

Deploy to vSphere server 192.0.2.1 which belongs to datacenter 'mydc' on vCenter server 192.0.2.100, and map the two NIC networks to vSwitches. Note that in this case -u specifies the vCenter login username.

cot deploy foo.ova esxi "192.0.2.100/mydc/host/192.0.2.1" \
    -u administrator -N "GigabitEthernet1=VM Network" \
    -N "GigabitEthernet2=myvswitch"

Deploy with passthrough arguments to ovftool.

cot deploy foo.ova esxi 192.0.2.100 -u admin -p password \
    --ovftool-args="--overwrite --acceptAllEulas"

Glossary

COT
Common OVF Tool
controller
hardware controller
A virtual hardware controller for hardware such as a disk device. In addition to its primary type (IDE, SCSI, etc.), a controller may also have a subtype, such as virtio or lsilogic. In an OVF package, a controller is represented by an XML Item element in the VirtualHardwareSection of the OVF descriptor. Typically each disk device must be associated with a controller.
disk description
disk element
disk reference
A description of a virtual disk included in a virtual machine. In an OVF descriptor, this is an XML Disk element in the DiskSection. This disk description may be associated with a file reference and/or disk file, or it may be a placeholder for a blank disk not yet created. Typically a disk description must be associated with a disk drive in order to actually be accessible by the guest OS.
disk device
disk drive
disk item
A hardware item describing a virtual CD-ROM, DVD-ROM, or hard disk drive. In an OVF package, this is an XML Item element in the VirtualHardwareSection of the OVF descriptor. This item may reference a disk reference or a file reference to map a filesystem to this drive. Typically a disk drive must be associated to a hardware controller.
disk file
disk image
A file such as a .vmdk, .iso, or .qcow2. May or may not be associated with a disk drive.
file element
file reference
A reference to a file, such as a disk file or any other file type, to be included in a virtual machine. In an OVF descriptor, this is an XML File element in the References section.
hardware element
hardware item
Generic term for any discrete piece of virtual machine hardware, including but not limited to the CPU(s), memory, disk drive, hardware controller, network port, etc.
OVF
Open Virtualization Format, an open standard.
OVF descriptor
An XML file, based on the OVF specification, which describes a virtual machine.

Change Log

All notable changes to the COT project will be documented in this file. This project adheres to Semantic Versioning.

2.1.0 - 2018-01-29

Fixed

  • The bin/cot helper script in this repository (not the script installed when installing COT normally) has had some $PATH issues fixed.
  • cot add-disk ... --controller now accepts sata as an additional option (#72).

2.0.5 - 2017-11-30

Fixed

  • Error in previous release due to COT using a user-specified relative file path in FileReference.create which implicitly assumed absolute paths.
    • Now correctly fixes up the path before constructing this FileReference.
    • To prevent similar errors in the future, if a FileReference is constructed with a non-absolute path, COT will log a warning and attempt to fix the path at that point.

2.0.4 - 2017-11-30

Fixed

  • COT now ensures that cloned Items have a higher InstanceID than the Item they were cloned from, avoiding unexpected out-of-order behavior (#70).
  • COT now produces valid manifests for OVF 2.x files (which use SHA256 instead of SHA1 as in OVF 0.x and 1.x).

Added

  • Official support for pypy3. (COT has already been usable under pypy3, but now it's actually part of our tests.)
  • COT checks the manifest when loading an OVF/OVA and warns if any checksum mismatches are seen.
  • FileReference class as superclass of FileInTAR and FileOnDisk.

Changed

  • Refactored how man page builds are triggered so that Sphinx is no longer a mandatory pre-requisite for building COT. This helps us to avoid breaking Travis-CI tests when a new version of Sphinx comes out that doesn't support older Python versions like 3.3.
  • Replaced add_disk.guess_drive_type_from_extension with DiskRepresentation.predicted_drive_type.

2.0.3 - 2017-04-03

Fixed

  • Fixed issue where UnboundLocalError would be raised during COT's attempt to clean up after a qemu-img error occurring while trying to convert a disk to VMDK (#67).
  • Fixed incorrect invocation of 'sudo mkdir' on Mac OS X.

2.0.2 - 2017-03-20

Fixed

  • Path normalization of output file resulted in COT failing to detect the case of self-overwriting an OVA, resulting in file corruption (#66). Improved detection of self-overwriting cases including relative vs. absolute paths, symlinks, and hardlinks.

2.0.1 - 2017-03-17

Fixed

  • One unit test case, COT.helpers.tests.test_helper.HelperGenericTest.test_download_and_expand_tgz, was failing after 2.0.0 was released, due to its assumptions about the contents of the master repo on GitHub no longer being valid. Fixed now.

2.0.0 - 2017-03-17

Fixed

  • Fixed a case that could result in a RuntimeError being thrown when using cot edit-hardware to simultaneously create NICs and define a new configuration profile (#64).
  • Update minimum QEMU version preferred for creation of streamOptimized VMDK files from 2.1.0 to 2.5.1, due to an incompatibility with VMware in VMDKs produced by versions prior to 2.5.1 (#65).
    • If vmdktool is not installed, but QEMU 2.1.0-2.5.0 is, then COT will fall back to usinq qemu-img, but will warn of this potential incompatibility with ESXi.
  • When cot info is invoked with multiple files as input, an error in loading one file will no longer cause the entire command to abort; COT will now log an error about the offending file and continue to the next.

Added

  • COT commands now check the available disk space in the temporary working directory against the temporary storage space the command estimates it will require, and if there is likely to be a problem, warns the user before continuing. (#63)
  • Additionally COT commands that write out an updated OVF/OVA now check the available disk space in the requested output location against the VM's predicted output size, and if there is likely to be a problem, warns the user before continuing.
  • Helper classes can now opt to cache their output to avoid repeatedly re-running the same command. Currently enabled for qemu-img info ... and isoinfo ... commands. (#62)
  • New modules and APIs in support of the above:
    • Command.working_dir_disk_space_required() instance method, to ask a command to estimate how much temporary storage it will require. This method is now automatically called from Command unit tests to validate its accuracy.
    • VMDescription.predicted_output_size() instance method, to ask a VM to estimate how much disk space it will require when written out.
    • COT.utilities module with functions available_bytes_at_path and tar_entry_size.

Removed

  • Discontinued support for Python 2.6 as it has been retired since 2013.
  • Removed cot --debug alias for cot --verbose --verbose as it is no longer correct after increased granularity of logging options, as described below.

Changed

  • With cot edit-hardware, the platform hardware validation is no longer a hard limit. Instead, if a value appears to be invalid, the user will be warned about the validation failure and given the option to continue or abort (#61). cot --force ..., as usual, can be used to auto-continue without prompting.
  • More granular logging levels (thanks to verboselogs; COT now requires version 1.6 of this package) and more self-consistency in message logging.
  • Revised format of log messages generated by COT.
  • Lots of API changes:
    • All commands ('submodules') are now grouped as a sub-package under COT.commands. Most classes in this package have been renamed.
    • Moved COT.ovf package to be a sub-package under COT.vm_description.
    • Moved UI and CLI classes into a COT.ui sub-package.
    • Moved the to_string function from COT.data_validation to COT.utilities.
    • Function COT.deploy_esxi.get_object_from_connection is now method PyVmomiVMReconfigSpec.lookup_object.
    • Function COT.cli.formatter is now class COT.ui.cli.CLILoggingFormatter.
    • The functionality of classes VMContextManager and VMFactory has been rolled into the core VMDescription class.
    • COT.disks module:
      • Function create_disk is now split into class methods DiskRepresentation.for_new_file (creates the disk file and returns a corresponding DiskRepresentation instance) and DiskRepresentation.create_file (creates disk file only).
      • Function convert_disk is now class method DiskRepresentation.convert_to
      • Function disk_representation_from_file is now class method DiskRepresentation.from_file
      • The DiskRepresentation constructor now only takes the path to a file as input - if you want to create a new file, use DiskRepresentation.for_new_file instead of calling the constructor directly.
    • COT.helpers module:
      • Static method Helper.cp has been renamed to Helper.copy_file.
    • COT.ovf module:
      • COT.ovf.ovf.byte_string has been moved and renamed to COT.utilities.pretty_bytes.
      • COT.ovf.ovf.byte_count has been moved and renamed to COT.vm_description.ovf.utilities.programmatic_bytes_to_int.
      • COT.ovf.ovf.factor_bytes has been moved and renamed to COT.vm_description.ovf.utilities.int_bytes_to_programmatic_units.
      • OVF.validate_and_update_file_references and OVF.validate_and_update_networks have been moved to private functions and are no longer part of the public API of the OVF class.
      • Hardware.update_existing_item_count_per_profile has been moved to a private method.
    • COT.platforms module:
      • Class GenericPlatform is now Platform.
      • Function platform_from_product_class is now class method Platform.for_product_string and returns an instance of a Platform class rather than the class object itself.
      • Most Platform APIs are now instance methods instead of class methods.
      • Function COT.platforms.is_known_product_class has been removed.

1.9.1 - 2017-02-21

Changed

  • Removed 32 GiB memory limitation on Cisco IOS XRv 9000 platform.

1.9.0 - 2017-02-13

Added

  • Support for Python 3.6
  • Support for brew package manager (#55).
  • Support for Cisco Nexus 9000v (NX-OSv 9000) platform (#60).

Fixed

  • Improved messaging when COT is unable to install a helper program (#57).

1.8.2 - 2017-01-18

Fixed

  • Issue (#58) where various multi-value CLI options for the edit-hardware and inject-config commands did not append properly.
  • Issue in which explicitly specified NIC names were being overwritten by names auto-derived from network names when attempting to set both NIC name and network names in a single cot edit-hardware call.
  • cot edit-properties again accepts property values containing the characters + and = (#59).

Added

  • COT can now detect the presence of an .ovf file in a TAR archive even if the archive is not named *.ova* and even if the .ovf file is not the first file in the archive as specified by the OVF specification. This allows (read-only at present) handling of VirtualBox Vagrant .box files, which are approximately equivalent to non-standards-compliant OVAs.

1.8.1 - 2016-11-12

Fixed

  • Under Python versions older than 2.7.9, explicitly require pyOpenSSL and ndg-httpsclient to avoid issues like hostname 'people.freebsd.org' doesn't match 'wfe0.ysv.freebsd.org' when installing vmdktool.

1.8.0 - 2016-11-08

Fixed

  • TypeError in find_item method (#54).
  • cot inject-config correctly handles OVAs with multiple empty CD-ROM drives to choose amongst (#54 also).
  • Cisco CSR1000v platform now supports 8 CPUs as a valid option.

Added

  • cot inject-config --extra-files parameter (#53).
  • Helper class for isoinfo (a companion to mkisofs).
  • Added glossary of terms to COT documentation.
  • Inline documentation (docstrings) are now validated using the Pylint docparams extension.

Changed

  • Refactored the monolithic COT/platforms.py file into a proper submodule.
  • create_iso() now adds Rock Ridge extensions by default.
  • Refactored COT.helpers into two modules - COT.helpers (now just for handling helper programs such as apt-get and mkisofs) and COT.disks (which uses the helpers to handle ISO/VMDK/QCOW2/RAW image files).
  • Inline documentation (docstrings) have been converted to "Google style" for better readability in the code. Sphinx rendering of documentation (for readthedocs.org, etc) now makes use of the napoleon extension to handle this style.

Removed

1.7.4 - 2016-09-21

Newer versions of Sphinx have dropped support for Python 2.6 and 3.3, so I have updated COT's installation requirements to use older Sphinx versions under these Python versions.

1.7.3 - 2016-09-06

Added

  • When adding NICs to an OVF, if no -nic-networks are specified, cot edit-hardware will now try to infer sequential naming of the Network elements and if successful, offer to create additional Networks as appropriate. (#18)

1.7.2 - 2016-08-17

Fixed

  • Issue #52 - OVFItemDataError raised when adding NICs to CSR1000V OVA, or more generally when cloning an OVFItem whose ElementName references its Connection.

1.7.1 - 2016-08-12

Fixed

  • cot deploy ... --serial-connection will create additional serial ports beyond those defined in the OVF, if requested. Previously it would ask the user for confirmation but not actually do anything about it. (#51)

1.7.0 - 2016-08-05

Added

  • Support for Python 3.5
  • Enhancements to cot edit-properties (#50):
    • Added --user-configurable option to set whether created/updated properties are marked as user-configurable in the OVF.
    • Added --labels and --descriptions options to set/update the labels and descriptions associated with properties.
    • It's now valid to set no default value for a property by omitting the =value, as in -p property-with-no-value, as well as the existing -p property-with-empty-value= syntax to set an empty string as the value.
    • Users can now optionally specify the property type to enforce for each property by using the delimiter +type, as in -p key=1+boolean.

Changed

  • Switched from statement coverage to branch coverage for better test analysis.
  • Moved from Coveralls to Codecov for test coverage tracking, since Coveralls does not support branch coverage reporting.

Fixed

  • When cot edit-hardware is used to create new NICs in an OVF that previously had none, and the user does not specify any corresponding Network entries, automatically create a 'VM Network' entry, because all NICs must be mapped to Networks for a valid OVF descriptor.

1.6.1 - 2016-07-07

Fixed

  • ValueMismatchError exceptions are properly caught by the CLI wrapper so as to result in a graceful exit rather than a stack trace.
  • cot remove-file now errors if the user specifies both file-id and file-path, one of which matches a file in the OVF, but the other does not match this or any other file.
  • Better handling of exceptions and usage of sudo when installing helpers.
  • Manual pages are now correctly included in the distribution. Oops!

1.6.0 - 2016-06-30

Added

  • cot edit-product --product-class option, to set or change the product class identifier (such as com.cisco.csr1000v).
  • Enabled additional code quality validation with Pylint, pep8-naming, and mccabe (#49).

Changed

  • Lots of refactoring to reduce code complexity as measured by Pylint and mccabe.

Fixed

  • COT now recognizes AllocationUnits values like megabytes.
  • COT no longer ignores the AllocationUnits value given for RAM.
  • COT.ovf.byte_string() now properly uses binary units (KiB rather than kB, etc.)

1.5.2 - 2016-06-17

Changed

  • Development requirement changes: The package pep8 has been renamed to pycodestyle, and pep257 has been renamed to pydocstyle. Updated configuration and documentation to reflect these changes. Also, flake8-pep257 does not presently handle these changes, so replaced it as a dependency with the more up-to-date flake8-docstrings package.

1.5.1 - 2016-06-07

Added

  • cot edit-hardware --network-descriptions option, to specify the descriptive string(s) associated with each network definition.

Fixed

  • #48 - NIC type not set when adding NICs to an OVF that had none before.
  • When updating NIC network mapping, COT now also updates any Description that references the network mapping.

1.5.0 - 2016-06-06

Added

  • #47 - Added cot remove-file subcommand.
  • #43 - add cot edit-properties --transport option to set environment transport type(s) - iso, VMWare Tools, etc.
    • cot info now has a new "Environment" section that displays the transport type
  • #45 - support for multiple values for --nic-types, --ide-subtypes, and --scsi-subtypes in cot edit-hardware.
  • COT now recognizes the Cisco IOS XRv 9000 platform identifier com.cisco.ios-xrv9000.
  • #21 - subcommand aliases (Python 3.x only):
    • cot edit-product aliases: cot set-product, cot set-version
    • cot edit-properties aliases: cot set-properties, cot edit-environment, cot set-environment
    • cot info alias: cot describe
    • cot inject-config alias: cot add-bootstrap
    • cot remove-file alias: cot delete-file
  • Support for tab-completion of CLI parameters using argcomplete.

Changed

  • cot edit-hardware options --nic-types, --ide-subtypes, and --scsi-subtypes are now validated and canonicalized by COT, meaning that:
    • cot edit-hardware --nic-type virtio-net-pci is now a valid command and will correctly create an OVF with ResourceSubType virtio (not virtio-net-pci)
    • cot edit-hardware --ide-subtype foobar will now fail with an error
  • cot info is now more self-consistent in how it displays property keys. They are now always wrapped in < >, whereas previously this was only sometimes the case.
  • cot info --verbose now displays file and disk ID strings under the "Files and Disks" section.

1.4.2 - 2016-05-11

Added

  • COT now supports xorriso as another alternative to mkisofs and genisoimage

Fixed

  • #42 - cot deploy esxi error handling behavior needed to be updated for requests release 2.8.
  • #44 - test case failure seen when running pyVmomi 6.0.0.2016.4.

Changed

  • Installation document now recommends installation via pip rather than installing from source.
  • #40 - Now uses faster Docker-based infrastructure from Travis CI for CI builds/tests.

1.4.1 - 2015-09-02

Fixed

  • #41 - symlinks were not dereferenced when writing out to OVA.

1.4.0 - 2015-09-01

Added

  • #24 - cot deploy esxi now creates serial ports after deployment using pyVmomi library.
    • Serial port connectivity must be specified either via entries in the OVF (which can be defined using cot edit-hardware ... -S) or at deployment time using the new -S / --serial-connection parameter to cot deploy.
    • The syntax for serial port connectivity definition is based on that of QEMU's --serial CLI option.
    • Currently only "telnet", "tcp", and "device" connection types are supported.
  • #38 - cot edit-product can now set product and vendor information.
  • flake8 validation now includes pep257 to validate docstring compliance to PEP 257 as well.
  • Added changelog file.
  • Added COT.file_reference submodule in support of #39.

Changed

  • Split ESXi-specific logic out of COT.deploy module and into new COT.deploy_esxi module.
  • UT for COT.deploy_esxi now requires mock (standard library in Python 3.x, install via pip on Python 2.x).

Fixed

  • #39 - avoid unnecessary file copies to save time and disk space.

1.3.3 - 2015-07-02

Fixed

  • #10 - When changing network mapping, delete no longer needed networks
  • #31 - Added --delete-all-other-profiles option to cot edit-hardware
  • #32 - cot edit-hardware network names can now use wildcards
  • #34 - cot add-disk can now be used to replace a CD-ROM drive with a hard disk, or vice versa.

1.3.2 - 2015-04-09

Fixed

  • Adapt to changes to the Travis-CI testing environment.

1.3.1 - 2015-04-09

Fixed

  • #30 - cot install-helpers can now install fatdisk and vmdktool under Python 3.

1.3.0 - 2015-03-27

Added

  • Installation of helper programs is now provided by a cot install-helpers subcommand rather than a separate script.
  • COT now has man pages (man cot, man cot-edit-hardware, etc.) The man pages are also installed by cot install-helpers.
  • Improved documentation of the CLI on readthedocs.org as well.

Changed

  • Refactored COT.helper_tools module into COT.helpers subpackage. This package has an API (COT.helpers.api) for the rest of COT to access it; the helper-specific logic (qemu-img, fatdisk, etc.) is split into individual helper modules that are abstracted away by the API.
  • Similarly, logic from COT.tests.helper_tools has been refactored and enhanced under COT.helpers.tests.
  • Renamed all test code files from "foo.py" to "test_foo.py" to facilitate test case discovery.
  • CLI help strings are dynamically rendered to ReST when docs are built, providing cleaner output for both readthedocs.org and the manpages.

Removed

  • COT no longer supports Python 3.2.
  • cot_unittest is no more - use tox or unit2 discover to run tests.
  • As noted above, the installation script check_and_install_helpers.py no longer exists - this functionality is now provided by the COT.install_helpers module.

1.2.4 - 2015-03-06

Fixed

  • #29 - cot edit-properties interactive mode was broken in v1.2.2

1.2.3 - 2015-02-19

Fixed

1.2.2 - 2015-02-19

Added

Changed

  • CLI adapts more intelligently to terminal width (fixes #28)
  • Submodules now use Python properties instead of get_value/set_value methods.

1.2.1 - 2015-02-03

Added

  • Now PEP 8 compliant - passes validation by flake8 code analysis.
  • Very preliminary support for OVF 2.x format
  • Now uses tox for easier test execution and coverage.py for code coverage analysis.
  • Code coverage reporting with Coveralls.

Changed

  • Now uses colorlog instead of coloredlogs for CLI log colorization, as this fits better with COT's logging model.
  • Greatly improved unit test structure and code coverage, including tests for logging.

1.2.0 - 2015-01-16

Added

  • Greatly improved logging (#26). COT now defaults to logging level INFO, which provides relatively brief status updates to the user. You can also run with --quiet to suppress INFO messages and only log WARNING and ERROR messages, --verbose to see VERBOSE messages as well, or --debug if you want to really get into the guts of what COT is doing.
  • Now integrated with Travis CI for automated builds and UT under all supported Python versions. This should greatly improve the stability of COT under less-common Python versions. (#12)

Changed

  • The CLI for cot deploy has been revised somewhat based on user feedback.
  • A lot of restructuring of the underlying code to make things more modular and easier to test in isolation.

Fixed

  • Various bugfixes for issues specific to Python 2.6 and 3.x - these environments should now be fully working again.

1.1.6 - 2015-01-05

Added

  • Added THANKS file recognizing various non-code contributions to COT.

Fixed

  • Bug fixes for cot inject-config and cot deploy, including issues #19 and #20 and a warning to users about serial ports and ESXi (issue eventually to be addressed by fixing #24).
  • More graceful handling of Ctrl-C interrupt while COT is running.

1.1.5 - 2014-11-25

Fixed

  • Fixed issue #17 (cot edit-hardware adding NICs makes an OVA that vCenter regards as invalid)
  • Removed several spurious WARNING messages

1.1.4 - 2014-11-12

Added

  • COT can at least be installed and run under CentOS/Python2.6 now, although the automated unit tests will complain about the different XML output that 2.6 produces.

Changed

  • Vastly improved installation workflow under Linuxes supporting apt-get or yum - included helper script can automatically install all helper programs except ovftool. Fixes #9.

Fixed

  • Improved cot deploy handling of config profiles - fixed #5 and #15

1.1.3 - 2014-10-01

Added

  • cot edit-hardware added --nic-names option for assigning names to each NIC
  • cot info now displays NIC names.

Fixed

  • Improved installation documentation
  • Some improvements to IOS XRv OVA support

1.1.2 - 2014-09-24

Added

  • Take advantage of QEMU 2.1 finally supporting the streamOptimized VMDK sub-format.
  • Can now create new hardware items without an existing item of the same type (issue #4)

Changed

  • Clearer documentation and logging messages (issue #8 and others)
  • Now uses versioneer for automatic version numbering.

Fixed

  • Fixed several Python 3 compatibility issues (issue #7 and others)

1.1.1 - 2014-08-19

Fixed

  • Minor bug fixes to cot deploy esxi.

1.1.0 - 2014-07-29

Added

  • cot deploy esxi subcommand by Kevin Keim (@kakeim), which uses ovftool to deploy an OVA to an ESXi vCenter server.

Changed

  • Removed dependencies on md5 / md5sum / shasum / sha1sum in favor of Python's hashlib module.
  • Nicer formatting of cot info output

Fixed

  • Miscellaneous fixes and code cleanup.

1.0.0 - 2014-06-27

Initial public release.

Contributing to COT

Please do contribute! We only have a few simple requirements for diffs and pull requests.

Follow coding guidelines

Logging level usage

COT uses logging levels (including the additional intermediate logging levels provided by the verboselogs package) as follows:

Level Usage guidelines Examples
ERROR

Something is definitely wrong, but COT is able to proceed, at least for the moment. COT may raise an exception at some later point due to this issue.

If continuing is already known to be impossible, you should raise an exception now instead of logging an ERROR message.
  • OVF descriptor is not following the specification.
  • User has provided invalid input, but can retry.
  • Expected data is missing.
  • Internal logic error in COT that should never be encountered.
WARNING

Something potentially wrong, or at least risky, happened that the user should be informed of.

This includes cases where, due to insufficient information provided by the user, COT had to make an uncertain choice on its own (or was unable to make such a decision but is continuing nonetheless).
  • COT would have prompted the user to provide input or confirm a risky operation, but has been instructed to run non-interactively.
  • COT is having to guess whether a given disk drive should be SCSI or IDE as the user didn't specify which when instructing COT to add the drive.
  • User-provided information was unused, such as providing the device type to set for a device that doesn't exist.
  • User has instructed COT to configure hardware settings that appear to be outside the supported range for the given platform.
NOTICE

Something noteworthy happened that is not necessarily a problem, but deserves the user's attention.

This is the lowest logging level enabled by default, so messages generated at and above this level should be succinct and meaningful to all users.
  • COT does not recognize the virtual platform described by a given OVF and so will be treating it generically.
  • COT is attempting to install a needed helper application.
  • COT is having to create parts of the OVF descriptor from scratch.
  • COT is replacing a file or disk that was previously included in this OVF with a new one.
INFO Status updates about normal operation of the software.
  • COT has successfully parsed an OVF or OVA and is ready to operate on it.
  • COT is beginning to write the updated OVF/OVA to disk.
VERBOSE Detailed information of interest to an expert or very curious user.
  • Individual task steps of editing an OVF.
  • COT's reasoning for making high-level decisions.
DEBUG Highly detailed information, probably only useful to a developer familiar with the code.
  • Information about temporary files and other internal state of COT itself.
  • Individual operations within a complex task.
SPAM Extremely detailed or repetitive info that even a developer will rarely want to see.
  • Data dumps of XML elements and other data structures.
  • Status updates on tasks that typically happen many times in a single COT run.

Coding style

We try to keep COT's code base compliant with Python coding standards including PEP 8 and PEP 257. We use the flake8 and Pylint tools and their extension packages to verify this as part of our test automation. To run coding style analysis independently of the other test automation, you can run tox -e flake8,pylint, or you can install these tools and run them directly:

cot/$ sudo pip install --upgrade flake8
cot/$ sudo pip install --upgrade pydocstyle
cot/$ sudo pip install --upgrade flake8-docstrings
cot/$ sudo pip install --upgrade pep8-naming
cot/$ sudo pip install --upgrade mccabe
cot/$ flake8
./COT/ovf/item.py:229:1: C901 'OVFItem.value_replace_wildcards' is too complex (11)
./COT/ovf/item.py:603:1: C901 'OVFItem.generate_items' is too complex (11)
./COT/ovf/ovf.py:461:1: C901 'OVF.validate_hardware' is too complex (14)

cot/$ sudo pip install --upgrade pylint
cot/$ pylint COT
************* Module COT.ovf.item
E:331,24: Instance of 'list' has no 'split' member (no-member)
R:334,16: Redefinition of value type from list to tuple (redefined-variable-type)
R:603, 4: Too many branches (13/12) (too-many-branches)
************* Module COT.ovf.ovf
C:  1, 0: Too many lines in module (2646/2600) (too-many-lines)
R:177, 0: Too many public methods (76/74) (too-many-public-methods)

Fix any errors and warnings these tools report, and run again until no errors are reported.

CLI consistency

When adding or modifying CLI, keep the following guidelines in mind:

  • All subcommands must support -h/--help. Never redefine -h to mean something other than "get help".
  • All commands that modify files must define -o/--output to specify a new file to create instead of overwriting any input file. If and only if the command does not modify the input file, you may use -o to mean something else (e.g., cot deploy ... has -o/--ovftool-args).
  • Best effort: use consistent argument names and short argument names unless there's an unavoidable conflict. Examples:
    • cot add-file and cot add-disk abbreviate --file-id as -f, but cot remove-file abbreviates it as -i because it uses -f to mean --file-path. In retrospect, using -p for --file-path would have allowed COT to be more self-consistenct.
    • cot edit-hardware uses -p/--profiles to specify one or more configuration profiles to operate on. cot deploy esxi uses -c/--configuration to specify the configuration profile to deploy. In this case, this is somewhat unavoidable as cot edit-hardware uses -c to mean --cpus (no reasonable alternative) and cot deploy esxi uses -p to mean --password (likewise).
  • In cases where short argument names unavoidably differ between commands, support the same set of long names if at all possible. For example, cot edit-hardware and cot deploy esxi should both support both --profile(s) and --configuration(s) as synonymous long names that can be used interchangeably to specify a configuration profile.

Add automated unit tests

Whether adding new functionality or fixing a bug, please add appropriate unit test case(s) under COT/tests/ or COT/<sub-package>/tests/ (as appropriate) to cover your changes. Your changes must pass all existing and new automated test cases before your code will be accepted.

You can run the COT automated tests under a single Python version by running python ./setup.py test.

For full testing under all supported versions as well as verifying code coverage for your tests, you should install tox (pip install tox) and coverage (pip install coverage) then run tox from the COT directory:

cot/$ tox
...
py27 runtests: commands[0] | coverage run --append setup.py test --quiet
...
py33 runtests: commands[0] | coverage run --append setup.py test --quiet
...
py34 runtests: commands[0] | coverage run --append setup.py test --quiet
...
py35 runtests: commands[0] | coverage run --append setup.py test --quiet
...
py36 runtests: commands[0] | coverage run --append setup.py test --quiet
...
pypy runtests: commands[0] | coverage run --append setup.py test --quiet
...
flake8 runtests: commands[0] | flake8
...
pylint runtests: commands[0] | pylint COT
...
docs runtests: commands[0] | sphinx-build -W -b html -d ...
...
stats runtests: commands[0] | coverage combine
stats runtests: commands[1] | coverage report -i
Name                                 Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------------------
COT/__init__.py                          5      0      0      0   100%
COT/add_disk.py                        168      3     66      3    97%
COT/add_file.py                         45      0     12      0   100%
COT/cli.py                             254     15     95      9    93%
COT/data_validation.py                 124      2     44      1    98%
COT/deploy.py                          154      6     62      6    94%
COT/deploy_esxi.py                     196      0     68      1    99%
COT/disks/__init__.py                   23      0     10      0   100%
COT/disks/disk.py                       56      1     20      1    97%
...
COT/vm_description.py                  166      4      4      0    98%
COT/vm_factory.py                       26      0      4      0   100%
COT/xml_file.py                        121      3     54      1    98%
----------------------------------------------------------------------
TOTAL                                 5122    114   1908    105    97%
stats runtests: commands[2] | coverage html -i
_______________ summary _______________
  setup: commands succeeded
  py27: commands succeeded
  py33: commands succeeded
  py34: commands succeeded
  py35: commands succeeded
  py36: commands succeeded
  pypy: commands succeeded
  flake8: commands succeeded
  pylint: commands succeeded
  docs: commands succeeded
  stats: commands succeeded
  congratulations :)

After running tox you can check the code coverage details by opening htmlcov/index.html in a web browser.

Update documentation

If you add or change any COT CLI or APIs, or add or remove any external dependencies, please update the relevant documentation.

Add yourself as a contributor

If you haven't contributed to COT previously, be sure to add yourself as a contributor in the COPYRIGHT.txt file.

Open a pull request

COT follows Vincent Driessen's A successful Git branching model. As such, please submit feature enhancement and non-critical bugfix requests to merge into the develop branch rather than master.

Credits

We would like to thank:

  • For evangelization, user feedback and bug reports:
    • Sean Adams
    • Arun Arunkumar
    • Mark Coverdill
    • Myles Dear
    • Chandu Gutti
    • Jeff Haag
    • Jeff Loughridge
    • Vincent Wenshi Lv
    • Roger Melton
    • Keith Mitchell
    • Jonathan Muslow
    • Scott O'Donnell
    • Rick Ogg
    • Anantha Padmanabha
    • Keerthi Rawat
    • David Rosenfeld
    • Rafal Skorka
    • Perumal Venkatesh
    • John Withington
  • For initial design review and comments:
  • For providing managerial support for the development and release of COT as open source:
    • Ray Romney
    • Sanjeev Tondale
    • Taskin Ucpinar
  • Rich Wellum, for creating "Build, Deploy, Execute OVA" (bdeo.sh), the precursor to COT.
  • Neil McGill, for creating and maintaining fatdisk
  • Brian Somers, for creating and maintaining vmdktool

COT package reference

The below documents describe in depth the code structure and APIs of COT. These are not generally of interest to the end users of the COT script, but are provided for reference of developers wishing to integrate the COT package directly into their code.

Package implementing the Common OVF Tool.

Utility modules

COT.data_validation Various helpers for data sanity checks.
COT.file_reference Wrapper classes to abstract away differences between file sources.
COT.utilities General-purpose utility functions for COT.
COT.xml_file Reading, editing, and writing XML files.

Sub-packages

COT.commands Package describing various operations COT can perform on a VM description.
COT.disks Package for handling various disk file types (VMDK, ISO, QCOW2, etc.).
COT.helpers Provides a common interface for interacting with various non-Python programs.
COT.platforms Package for identifying guest platforms and handling platform differences.
COT.ui Package providing common API for the COT user interface (UI) of whatever type.
COT.vm_description Support for various virtual machine description formats (OVF, OVA, etc.).

Note

The hierarchy of permissible imports between sub-packages is as follows:

COT.ui
   |
   +---> COT.commands
   |        |
   |        +---> COT.vm_description
   |        |        |
   |        |        +---> COT.platforms
   |        |        |
   |        +--------+---> COT.disks
   |        |                 |
   +--------+-----------------+---> COT.helpers

Thus, to avoid circular dependencies, none of the other sub-packages may import COT.ui - if they wish to interact with the UI in any way (e.g., COT.helpers prompting the user to confirm whether to try to install a helper program), this needs to be done with a callback object (e.g., COT.helpers.Helper.USER_INTERFACE) rather than an import of the other module.

Indices and tables