Developer Guide

Setup Development Environment

1. Install pip and tox:

   sudo apt-get install python-pip
   sudo pip install tox
   

2. Configure git pre-commit hook:

   sudo pip install flake8 pep8-naming
   flake8 --install-hook
   git config flake8.strict true
   

3. Configure your environment to run without root:

   sudo groupadd topology
   sudo usermod -a -G topology $USER
   sudo sh -c 'echo "%topology ALL = (root) NOPASSWD: /sbin/ip, /bin/mkdir -p /var/run/netns, /bin/rm /var/run/netns/*, /bin/ln -s /proc/*/ns/net /var/run/netns/*" > /etc/sudoers.d/topology'
   

   Now logout and login again. Confirm your you’re in the topology group with:

   $ id
   ...,1001(topology)
   

4. Download and install Docker:

   wget -qO- https://get.docker.com/ | sh
   

Building Documentation

tox -e doc

Output will be available at .tox/doc/tmp/html. It is recommended to install the webdev package:

sudo pip install webdev

So a development web server can serve any location like this:

$ webdev .tox/doc/tmp/html

Running Test Suite

tox -e py27,py34

topology_docker

topology_docker module entry point.

Submodules

topology_docker.networks

Docker networks management module.

Functions

• create_docker_network(): Create a Docker managed network with given configuration (netns, prefix) to
• create_platform_network(): Create a Topology managed network with given configuration (netns, prefix).

topology_docker.networks.create_docker_network(enode, category, config)

Create a Docker managed network with given configuration (netns, prefix) to be used with Topology framework.

Parameters:

• enode – The platform (a.k.a “engine”) node to configure.
• category (str) – Name of the panel category.
• config (dict) –

  Configuration for the network. A dictionary like:

  {
      'netns': 'mynetns',
      'managed_by': 'docker',
      'connect_to': 'somedockernetwork',
      'prefix': ''
  }
  

  This dictionary is taken from the node._get_network_config() result.

topology_docker.networks.create_platform_network(enode, category, config)

Create a Topology managed network with given configuration (netns, prefix).

Parameters:

• enode – The platform (a.k.a “engine”) node to configure.
• category (str) – Name of the panel category.
• config (dict) –

  Configuration for the network. A dictionary like:

  {
      'netns': 'mynetns',
      'managed_by': 'platform',
      'prefix': ''
  }
  

  This dictionary is taken from the node._get_network_config() result.

topology_docker.node

topology_docker base node module.

Classes

• DockerNode: An instance of this class will create a detached Docker container.

class topology_docker.node.DockerNode(identifier, image='ubuntu:latest', registry=None, command='bash', binds=None, network_mode='none', hostname=None, environment=None, privileged=True, tty=True, shared_dir_base='/tmp/topology/docker/', shared_dir_mount='/var/topology', create_host_config_kwargs=None, create_container_kwargs=None, **kwargs)

An instance of this class will create a detached Docker container.

This node binds the shared_dir_mount directory of the container to a local path in the host system defined in self.shared_dir.

Parameters:

• identifier (str) – Node unique identifier in the topology being built.
• image (str) – The image to run on this node, in the form repository:tag.
• registry (str) – Docker registry to pull image from.
• command (str) – The command to run when the container is brought up.
• binds (str) –

  Directories to bind for this container separated by a ; in the form:

  '/tmp:/tmp;/dev/log:/dev/log;/sys/fs/cgroup:/sys/fs/cgroup'
  

• network_mode (str) – Network mode for this container.
• hostname (str) – Container hostname.
• environment (list or dict) –

  Environment variables to pass to the container. They can be set as a list of strings in the following format:

  ['environment_variable=value']
  

  or as a dictionary in the following format:

  {'environment_variable': 'value'}
  

• privileged (bool) – Run container in privileged mode or not.
• tty (bool) – Whether to allocate a TTY or not to the process.
• shared_dir_base (str) – Base path in the host where the shared directory will be created. The shared directory will always have the name of the container inside this directory.
• shared_dir_mount (str) – Mount point of the shared directory in the container.
• create_host_config_kwargs (dict) – Extra kwargs arguments to pass to docker-py’s create_host_config() low-level API call.
• create_container_kwargs (dict) – Extra kwargs arguments to pass to docker-py’s create_container() low-level API call.

Read only public attributes:

Variables:

• image (str) – Name of the Docker image being used by this node. Same as the image keyword argument.
• container_id (str) – Unique container identifier assigned by the Docker daemon in the form of a hash.
• container_name (str) – Unique container name assigned by the framework in the form {identifier}_{pid}_{timestamp}.
• shared_dir (str) – Share directory in the host for this container. Always /tmp/topology/{container_name}.
• shared_dir_mount (str) – Directory inside the container where the shared_dir is mounted. Same as the shared_dir_mount keyword

_get_network_config()

Defines the network configuration for nodes of this type.

This method should be overriden when implementing a new node type to return a dictionary with its network configuration by setting the following components:

‘mapping’

This is a dictionary of dictionaries, each parent-level key defines one network category, and each category must have these three keys: netns, managed_by, and prefix, and can (optionally) have a connect_to key).

‘netns’

Specifies the network namespace (inside the docker container) where all the ports belonging to this category will be moved after their creation. If set to None, then the ports will remain in the container’s default network namespace.

‘managed_by’

Specifies who will manage different aspects of this network category depending on its value (which can be either docker or platform).

‘docker’

This network category will represent a network created by docker (identical to using the docker network create command) and will be visible to docker (right now all docker-managed networks are created using docker’s ‘bridge’ built-in network plugin, this will likely change in the near future).

‘platform’

This network category will represent ports created by the Docker Platform Engine and is invisible to docker.

‘prefix’

Defines a prefix that will be used when a port/interface is moved into a namespace, its value can be set to ‘’ (empty string) if no prefix is needed. In cases where the parent network category doesn’t have a netns (i.e. ‘netns’ is set to None) this value will be ignored.

‘connect_to’

Specifies a Docker network this category will be connected to, if this network doesn’t exists it will be created. If set to None, this category will be connected to a uniquely named Docker network that will be created by the platform.

‘default_category’

Every port that didn’t explicitly set its category (using the “category” attribute in the SZN definition) will be set to this category.

This is an example of a network configuration dictionary as expected to be returned by this funcition:

{
    'default_category': 'front_panel',
    'mapping': {
        'oobm': {
            'netns': 'oobmns',
            'managed_by': 'docker',
            'connect_to': 'oobm'
            'prefix': ''
        },
        'back_panel': {
            'netns': None,
            'managed_by': 'docker',
            'prefix': ''
        },
        'front_panel': {
            'netns': 'front',
            'managed_by': 'platform',
            'prefix': 'f_'
        }
    }
}

Returns:The dictionary defining the network configuration. Return type:dict

Inheritance

disable()

Disable the node.

In Docker implementation this pauses the container.

enable()

Enable the node.

In Docker implementation this unpauses the container.

notify_add_bilink(nodeport, bilink)

Get notified that a new bilink was added to a port of this engine node.

Parameters:

• nodeport ((pynml.nml.Node, pynml.nml.BidirectionalPort)) – A tuple with the specification node and port being linked.
• bilink (pynml.nml.BidirectionalLink) – The specification bidirectional link added.

notify_add_biport(node, biport)

Get notified that a new biport was added to this engine node.

Parameters:

• node (pynml.nml.Node) – The specification node that spawn this engine node.
• biport (pynml.nml.BidirectionalPort) – The specification bidirectional port added.

Return type:

str

Returns:

The assigned interface name of the port.

notify_post_build()

Get notified that the post build stage of the topology build was reached.

set_port_state(portlbl, state)

Set the given port label to the given state.

Parameters:

• portlbl (str) – The label of the port.
• state (bool) – True for up, False for down.

start()

Start the docker node and configures a netns for it.

stop()

Request container to stop.

topology_docker.nodes

topology_docker.nodes module entry point.

Submodules

topology_docker.nodes.host

Simple host Topology Docker Node using Ubuntu.

Classes

• HostNode: Simple host node for the Topology Docker platform engine.

class topology_docker.nodes.host.HostNode(identifier, image='ubuntu:14.04', **kwargs)

Simple host node for the Topology Docker platform engine.

This base host loads an ubuntu image (by default) and has bash as the default shell.

See topology_docker.node.DockerNode.

Inheritance

topology_docker.nodes.switch

Simple switch Topology Docker Node using Ubuntu.

Classes

• SwitchNode: Simple switch node for the Topology Docker platform engine.

class topology_docker.nodes.switch.SwitchNode(identifier, image='ubuntu:14.04', **kwargs)

Simple switch node for the Topology Docker platform engine.

This base switch loads an ubuntu image (by default) and creates a kernel-level bridge where it adds all of its front-panel interfaces to emulate the behaviour of a real “dumb” switch.

See topology_docker.node.DockerNode.

Inheritance

notify_post_build()

Get notified that the post build stage of the topology build was reached.

See DockerNode.notify_post_build() for more information.

topology_docker.platform

Docker engine platform module for topology.

Classes

• DockerPlatform: Plugin to build a topology using Docker.

class topology_docker.platform.DockerPlatform(timestamp, nmlmanager, **kwargs)

Plugin to build a topology using Docker.

See topology.platforms.platform.BasePlatform for more information.

Inheritance

add_bilink(nodeport_a, nodeport_b, bilink)

Add a link between two nodes.

See BasePlatform.add_bilink() for more information.

add_biport(node, biport)

Add a port to the docker node.

See BasePlatform.add_biport() for more information.

add_node(node)

Add a new DockerNode.

See BasePlatform.add_node() for more information.

destroy()

See BasePlatform.destroy() for more information.

post_build()

Ports are created for each node automatically while adding links. Creates the rest of the ports (no-linked ports)

See BasePlatform.post_build() for more information.

pre_build()

See BasePlatform.pre_build() for more information.

relink(link_id)

See BasePlatform.relink() for more information.

rollback(stage, enodes, exception)

See BasePlatform.rollback() for more information.

unlink(link_id)

See BasePlatform.unlink() for more information.

topology_docker.shell

Docker shell helper class module.

Classes

• DockerShell: Generic docker exec shell for unspecified interactive session.
• DockerBashShell: Specialized docker exec shell that will run and setup a bash

class topology_docker.shell.DockerShell(container, command, *args, **kwargs)

Generic docker exec shell for unspecified interactive session.

Inheritance

class topology_docker.shell.DockerBashShell(container, command, *args, **kwargs)

Specialized docker exec shell that will run and setup a bash interactive session.

Inheritance

topology_docker.utils

topology_docker utilities module.

Functions

• ensure_dir(): Ensure that a path exists.
• tmp_iface(): Return a valid temporal interface name.
• cmd_prefix(): Determine if the current user can run privileged commands and thus
• privileged_cmd(): Run a privileged command.
• get_iface_name(): Get interface name inside a container

topology_docker.utils.ensure_dir(path)

Ensure that a path exists.

Parameters:path (str) – Directory path to create.

topology_docker.utils.tmp_iface()

Return a valid temporal interface name.

Return type:str Returns:A random valid network interface name.

topology_docker.utils.cmd_prefix()

Determine if the current user can run privileged commands and thus determines the command prefix to use.

Return type:str Returns:The prefix to use for privileged commands.

topology_docker.utils.privileged_cmd(commands_tpl, **kwargs)

Run a privileged command.

This function will replace the tokens in the template with the provided keyword arguments, then it will split the lines of the template, strip them and execute them using the prefix for privileged commands, checking that the command returns 0.

Parameters:

• commands_tpl (str) – Single line or multiline template for commands.
• kwargs (dict) – Replacement tokens.

topology_docker.utils.get_iface_name(enode, netname)

Get interface name inside a container

This function will figure out the interface name inside the container for a given docker network.

Parameters:

• enode – The platform (“engine”) node to get the iface name from.
• netname (str) – The name of the docker network to which the interface we’re looking for is connected.

Docker Platform Engine for Topology

Docker based Platform Engine plugin for the Topology Modular Framework.

Documentation

Element attributes

Some element attributes are interpreted by this Platform Engine to perform several actions and setup. For example, the following option are taken into account in this Platform Engine.

[type=host image="ubuntu:latest"] host1 host2
[ipv4="192.168.20.20/24"] host1:veth0
[ipv4="192.168.20.21/24"] host2:veth0
[category="oobm"] host1:veth1
[identifier="mainlink" up=False] host1:veth0 -- host2:veth0

Nodes

type:

Type of this node. Node types can be extended using entry points:

entry_points={
    'topology_docker_node_10': [
        'host = topology_docker.nodes.host:HostNode',
        'openswitch = topology_docker.nodes.openswitch:OpenSwitchNode'
    ]
}

image:

Docker image to use for this node. You can run docker images to find the images that are available in your docker installation. To specify an image in this attribute, make sure to add the value for the REPOSITORY column and the value for the TAG column separated by a colon, like this: image="ubuntu:latest".

Ports

ipv4:IPv4 address to set to this port in the form 192.168.50.5/24. ipv6:IPv6 address to set to this port in the form 2001:0db8:0:f101::1/64. up:Bring up (the default) or down this port. category:The category this port belongs to. See topology_docker.node.DockerNode._get_network_config() for more information about network categories in a node.

Links

identifier:Unique idenfier of the link in the topology. It is required to explicitly set an identifier to a link to reference it in the topology_docker.platform.DockerPlatform.unlink() and topology_docker.platform.DockerPlatform.relink() functions. up:Bring up (the default) or down this link (implemented as bringing up or down both endpoints).

Engine Node extended interface

The Engine Nodes in this Platform Engine have the following additional methods:

• topology_docker.node.DockerNode.pause() and
• topology_docker.node.DockerNode.unpause()

This methods allow to pause and resume a node in the topology to test failover, replication, balancing or for simple management.

Node types provided by default

This Platform Engine provides two types of nodes by default: host and switch.

Host Node Type

This is a very simple host that can be used in topologies where there is a need to simulate a Linux-based host system. This node is based on an Ubuntu 14.04 image (by default) and uses bash for its shell.

It provides the two default network categories (as inherited from DockerNode): front_panel and oobm. The first one is used for ports that are going to be connected to other nodes and the second one is used to connect the node to a bridge network managed by docker itself (which means it’s not explicitly part of the topology).

Switch Node Type

This is a very simple switch that can be used in topologies where there is a need to simulate a switch that will just forward packets to the correct port based on the mac address that’s connected to it. This node is based on an Ubuntu 14.04 image (by default) and uses bash for its shell.

It provides the two default network categories (as inherited from DockerNode): front_panel and oobm. The first one is used for ports that are going to be connected to other nodes and the second one is used to connect the switch to a bridge network managed by docker itself (which means it’s not explicitly part of the topology).

All the ports in this node type (except the implicit one in the oobm category) are added to a kernel-level bridge named bridge0 which provides the switching functionality. Because of the way this type of bridges work, all interfaces added to it must be set to UP state, which means ports in nodes of this type will ignore the UP element attribute referenced above (all ports will be brought UP regardless of it).

Development

• Developer Guide.
• Internal Documentation Reference.
• Project repository.

License

Copyright (C) 2015-2016 Hewlett Packard Enterprise Development LP

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.