The GENI Developer Docs¶

A Short Background (Or, What Are the Limitations of the Internet Today?)¶

The landscape of the internet has changed dramatically since its beginnings as ARPANET in 1969. The devices using it have grown to include everything from PCs, laptops, and mobile phones to TVs, cars, and refrigerators. In addition to the range of devices, growing advanced in video and voice technology require greater bandwidth.

Although some of these applications were foreseeable, this is not what the current architecture of the internet was intended for.

Vint Cerf, widely considered a father of the internet gives a talk below, commenting on the internet’s beginnings and its inherent limitations.

A Global Environment for Network Innovations (GENI)¶

Experimentation and research into highspeed networks, new protocols, and distributed applications was previously limited to isolated labs. The GENI project was started to facilitate wide-scale experimentation — creating a virtual laboratory where engineers, researchers, and developers have access to:

• Programmable Networks: Routers can be programmed to handle packets in new ways; opening the door to experimenting with alternatives to the core protocols of the internet.
• Gigabit Networks: High speed networks that are 250 times faster than what is available today
• Virtual Machines: Provisionable environments for running applications that leverage these programmable, high-speed networks.

GENI refers to both the overall project and the suite of tools and services that have been developed around the 3 items above. This documentation exists to show you how, as a developer, you can get set up and write your own next-generation applications.

It’s the Side of Hacking You Haven’t Been Exposed To¶

How long have you taken TCP/IP for granted? How long have you build your web applications upon the decades-old work of others? With GENI, you are exposed to a new frontier of hacking.

Read more about what you can do with GENI.

Examples¶

Here are a few project ideas that are possible with GENI tools and resources.

Write an application that can utilize gigabit connections

Gigabit internet connections are the future. So what kind of network-reliant applications could you build that aren’t possible with the typical consumer internet speeds of today?

Test how different protocols are affected by network bottlenecks

If you wanted to compare, for instance, how TCP and UDP perform under different levels of network congestion, you could easily simulate that with GENI

Build distributed applications

You have considerable computing and network resources available via GENI, so you can develop and test your next-generation distributed applications over gigabit links.

Design an alternative to the IP protocol

Is IP holding you back? Define the rules that a router follows for forwarding packets, and experiment with what could be the internet’s future backbone protocol.

Going Forward¶

Are you ready to dig into GENI? Check out the prerequisites to make sure you can hit the ground running.

GENI Credentials¶

You’ll need GENI credentials before you can start experimenting. Click here to find out how to get set up with those.

Virtual Machines¶

We’ve mentioned that GENI is a virtual laboratory for running network experiments. As a GENI user, you are allowed to request and allocate resources around the world, such as virtual and physical machines.

It’s a good idea that you understand the basic difference between the different machine types. Sometimes it’s helpful to work with specific machine types in your experiments.

Linux and Relevant Utilities¶

It’s important that you are comfortable with using the Linux/UNIX command line. You will be using it heavily in the process of running your experiments, and it’s the only method you’ll have for controlling your resources.

In particular, make sure you’re well versed with:

1. A unix shell, such as bash
2. The ssh utility (used for connecting to your machines)
3. The ping utility
4. Setting up and using SSH keys (your only login method)
5. A scripting language, like ruby or python

Networking¶

What good would access to computing resources around the world be without a way to connect them? A basic understanding of computer networking concepts is important to have.

More specifically:

1. The layers of the OSI model
2. Internet Protocol, or IP, the backbone protocol of the internet
3. TCP and UDP
4. Connections over the mentioned protocols using socket APIs
5. How data packets are routed around the internet

When you feel comfortable with the topics above, you can move on to read about the terms and concepts used when discussing GENI.

GENI Setup: A Developer’s View¶

The basic work-flow of a developer looking to start building on GENI offers a nice view on the structure of the GENI project. So let’s consider Alice, a fictional GENI wannabe-experimenter.

From the top: Clearinghouse

In order to start interacting with GENI, Alice need to become a GENI user. Alice can obtain credentials from a GENI Clearinghouse. A GENI Clearing house can issue credentials that are federated, meaning that they are recognized by all GENI Management Authorities.

What’s in a Management Authority?

A Management Authority, or Manager, is an organization or institution that makes resources available to experimenters like Alice. These resources are made available via APIs. As a developer, Alice does not have to deal directly with these APIs because of GENI tools that are available to her.

What Tools?

Once Alice has GENI credentials, she can use tools like OMNI or Flack. Because Alice wants to dive in quickly, she opts for Flack, the visual web-based option.

Alice wants to start an experiment, so she needs to create an environment for her experiment, called a slice. The slice Alice creates via Flack will be recognized throughout by all of the GENI Management Authorities, which supply the resources and slivers that she needs.

What’s a Sliver?

Once Alice has her slice set up, she’ll want to request virtual machines from different Managers. The virtual machines, and any other resources that Alice might request are collectively referred to as slivers. They are called ‘slivers’ because they are often virtualized and shared components of a physical machine.

Whereas a slice is a global container for Alice’s experiments, the collection of resources she is using on any given site is referred to as a sliver. So if she has 2 nodes running in Utah, that’s one sliver, and her nodes running in Washington DC are in another sliver.

And that’s it

Now that you have a basic idea of how GENI is set up from a (very) high level, it’s best to move on the the more extensive GENI Terms and Concepts section.

Additional Resources¶

A glossary of the terms italicized in the document are available in the GENI Terms and Concepts section.

A more encompassing document with lower-level details on GENI architecture is available here.

Terms List¶

The list of terms below is organized in a way that the items further up on the list will help build your understanding of the terms toward the end of the list. Terms mentioned within definitions are in italics.

Resources

A resource is a generic term used to describe anything you can request and use on the GENi platform. It may include physical machine, virtual machines, network hardware and more. in the image of Flack above, the types of resources you can request from GENI are in the drop-down menu in the center of the screen (All types, firewall, etc).

Credentials

Your credentials are your keys to using the GENI environment. Once you have GENI credentials, you may begin setting up environments, requesting resources, and using the various tools that have been developed for GENI. Your credentials come in the form of a username, SSH key, and a password.

Flack

Flack is pictured at the top of this document. It’s is a visual tool for setting up experiment environments (known as a slice), requesting resources (such as a sliver), and setting up network paths between them. Much of your interfacing with GENI in experiments and examples will be through Flack. You can learn more about using Flack here.

Omni

Omni is a command-line tool for interacting with GENI. You can allocate resources like you can do with Flack, but without the visual interface. Omni is a powerful utility, which you can learn more about here.

Slice

A slice is an environment and conceptual container that your resources reside in. A single slice is recognized globally throughout the GENI environment. Within each slice, you can request resources from any GENI authority. Note that in the image of Flack above, A slice named “TestSlice” has been created.

Sliver

a sliver is a set of resources that are reserved in an AM for a slice, i.e. slivers are what a user has in specific AMs

Up until now, we have referred to the different computing and network components that you allocate within GENI as resources. On GENI, you may have resources in many different places. For example, maybe you have resources allocated in Utah, and you have other resources allocated in Washington DC.

In GENI terminology, a sliver refers to the set of resources in a single location. So that group of resources in Utah is one sliver, and the grouping of resources in Washington DC is another sliver.

Your slivers are all part of the same Slice.

In the Image of Flack above, you can see the sliver that is going to be allocated for the slice named ‘TestSlice’: 1 Virtual Machine (VM), 1 Physical Machine (PC), and network interfaces to link them (lan0).

Clearinghouse

A Clearinghouse is a GENI authority that is capable of signing users up to experiment with GENI. In the GENI architecture, there are multiple clearinghouses. If you are gaining credentials through the The Mozilla Ignite App Challenge, your clearinghouse will be located at pgeni.gpolab.bbn.com.

Federation

This is the general idea that once you receive your GENi credentials from a Clearinghouse, you can use them to request resources and slivers anywhere. In visual tools such as Flack, this is largely transparent.

Management Authority (Authority)

Resources around the world are managed by what is called a Management Authority. This is typically an organization or institution that manages the servers and virtual machines on their premises, and makes them available to GENI as a whole. In the image of Flack above, the “Managers” list on the far left are the Management Authorities.

Resource Specification (RSpec)

Also largely transparent to the end user when using tools such as Flack, a Resource Specification, more commonly called an RSpec, is an XML document that lists resources. It is used in two primary cases:

1. When querying a Management Authority on what types of resources it has available. Before you ask for a few machines for your experiment, you need to know what is available to you, right? This is called an Advertisement RSpec.
2. Once you get a list of resources available to you, and you decide what you need, you send an RSpec that describes the resources you are requesting to the appropriate Management Authorities.

Again, with a visual tool such as Flack, you don’t have a reason to build or read RSpecs manually. Flack does the work for you. If you are curious although, Flack allows you to both read the RSpecs it has created for you, and import RSpecs that you have written yourself or saved prior.

Ticket

When a request is made for resources (And an RSpec is sent out), the Management Authorities typically issues Tickets, which is essentially a promise to fulfill the request. Sometimes, requests cannot be fulfilled for various reasons.

Seattle

Seattle is another project under the GENI umbrella which allows you allocate virtual machines around the globe and run experiments. Some of these virtual machines reside on physical machine, host virtual machines, and even mobile devices. Because of the transient connectivity of some devices, it is very helpful for simulating what the internet is really like for experiments. You do not need GENI credentials to use Seattle. Anyone can sign up.

Additional Resources¶

This list is a modification of the original GENI glossary available at GENI.net. It has been modified to be of use specifically to developers. You are encouraged, however, to read as deeply into GENI as you see fit.

1. Geni.net GENI Glossary

See Your Gigabit App Through A Browser¶

If you have a box in Utah and another in DC, as we did in example 2, with a gigabit link in between, it may be nice to see your app through a browser.

For example, perhaps one box was intended to act as a client, and another as a web server, because you’re building a next-generation webapp that relies on a high speed connection. Using you application from home doesn’t take advantage of the high speed link the client box has, so you need to see your app through the eyes of the client, as if you were sitting in Utah.

X, commonly called X11 is a tool that can use to gain a remote desktop capability with your provisioned client. This is possible if:

1. The system you are remoting from (your Mac, Thinkpad, windows box, etc) supports X11
2. The system you are remoting to supports X11, and has SSH installed

In terms of requirement number 1, this is easiest is your laptop or desktop has a recent version of OSX or Ubuntu installed. It’s possible with a Windows computer too, with a little extra work.

Requirement #2 is easily satisfiable if the system you are remoting to has Ubuntu installed. From there, getting X11 and a Firefox browser installation is as simple as:

$ sudo apt-get update
$ sudo apt-get install xorg openbox ubuntu-desktop firefox

And then from your terminal:

$ ssh -X username@pcXXX.genihost.net

At that point, any windowed application you run in the terminal on the remote host will open a window on your local machine. Try something like:

$ firefox

And you should see a window appear, showing your Firefox browser running remotely.

Accessing Nodes Behind Firewalls¶

In a couple of the example projects, we set up nodes in Utah and Washington DC. In DC, the names of our nodes were prefixed with pg. For example, the name of the node might be something like pg502.

Nodes prefixed with pg are behind a firewall, and accessing them can only be done through another node networked with them. For that reason, it’s easiest to set up an SSH tunnel to reach the pg node.

This is a necessary part of completing Example 2, so it would be good to know how this works ahead of time.

$ ssh -p [pcxxx port] [username]@[pcxxx host] -L 2222:[pgxxx host]:[pgxxx ssh port]

$ ssh -p 22 katzgrau@pc100.emulab.net -L 2222:pg200.emulab.net:22

$ ssh -p [local tunnel port][username]@127.0.0.1

$ ssh -p 2222 katzgrau@127.0.0.1

Saving Configurations/RSpecs¶

If you’ve gone through the trouble of setting up a complex topology with specially configured nodes, it can be extremely helpful to save configuration of your experiment into an RSpec/text file. Next time that you want to set up the same experiment, you can load in the RSpec.

To see the RSpec that was generated while you were configuring your project, click the “View” dropdown on the upper left of the slice pane:

From there, click “Preview request documents”, and you should see:

You have the copy to copy this RSpec or save it to a file. It’s probably best to save it to a file so you can load it up later. When that time comes, click the “Import” dropdown.

When you import the RSpec, your experiment will be created with all the same settings and configurations it had before.

Tutorial¶

If you run into any problems with this tutorial, reach out to kenny at mozillafoundation dot org.

Step 1. Log In to Your Management Authority¶

For this tutorial we’re going to use Flack, the browser-based visual tool for interacting with GENI. The easiest way to authenticate with Flack is to simply be logged in to your clearinghouse’ website in one window or tab, and to have Flack open in another window or tab.

Figure 1: Logging in to a management authority (yours may differ)

Step 2. Open Flack In A New Tab¶

Flack is an in-browser tool written in Flash. It’s likely you already have flash installed, but if you don’t you can get it here:

Open Flack in a new tab or browser window. Flack can be found here: www.protogeni.net/trac/protogeni/wiki/Flack . You should see Flack initialize in the window, as in the figure below:

Figure 2: Flack Initialized

Step 3. Log In With Flack¶

Click the green “Log In” button above. Here, you are expected to paste your GENI private key. But because we are already logged into our management authority via another tab, we can complete this step easily and have our private key information inserted automatically.

Along the top of the prompt, there should be a button and dropdown box that says “Download from [Select authority]” as in the figure below.

Select your management authority’s domain from the dropdown. The correct domain to choose will match up with the domain where you logged in at Step 1.

Once the correct item is selected, click the “Download” button next to the option list. Your certificate (credentials) should appear, as in the figure below:

Finally, enter the private key passphrase that you used when you signed up at your management authority when you signed up. Click “OK” on the bottom left.

At this point, Flack will attempt to retrieve resources from the various authorities. When prompted to select where to list resources from, leave all of the options checked and click “Continue”. Flack will gather the resources lists, and display a map of North America, as pictured below:

Step 4. Create a New Slice¶

You’re now ready to create a GENI “slice.” Your slice is a GENI-wide environment that you can add resources to, and run experiments in. For more on GENI terminology, see the glossary.

Click on the “New” button in the top-left corner of Flack, and enter a name for this slice. Your slice name will have to be unique from all other GENI slices, so it cannot be the same name as in the example.

When you’re finished typing the name of your slice, click “OK”. Once your slice has been created (it may take a moment), you’ll see a blank pane with the name of your slice toward the top of your screen. The blank pane on the right is where we’ll set up our experiment.

Step 5. Set Up Two Machines and A Link¶

Now for the fun part!

In the image below, you’ll see:

• A: The area where new slices are created, and your existing slices are listed
• B: The area where the resources that you have available to you are listed (like VMs and PCs)
• C: The area where you can drag and drop resources

In area B, there will be an authority listed as ‘utahemulab.cm’. Drag the “VM” component immediately next to it to the area on the right. You should see:

Now, drag another VM onto the pane:

Lastly, drag a line between the two VMs, and click “Submit” on the bottom. Keep in mind, the ‘lan0’ device in the figure below will appear automatically.

At this point, you have two virtual machines on the GENI platform that are being provisioned. These two VMs are networked, and will have entries in their hosts file, (/etc/hosts) automatically created which point to each other. In other words, from this visual tool (Flack), you created two real virtual machines that are networked with each other on GENI.

When your VMs are ready, the background color of the right pane will be light green, as below.

Step 6. Send A Ping From One Node to the Other¶

At the last step in Step 5, you should see the connection info for each node listed to the right of it. For example, the hostname and port of the nodes in the example are.

• VM: Host pc533.emulab.net port 31290
• VM-0: Host pc533.emulab.net port 31291

If I open a terminal window, I should be able to connect to both without using a password:

Logging into VM

Logging into VM-0

As mentioned before, the hosts are already networked, and conveniently, there are already hosts for the appropriate IPs for VM and VM-0 in each host file:

At this point, we’re ready to ping one host from the other. Open the window for machine VM and type:

ping -c 2 VM-0

This will ping VM-0 twice. The -c flag allows us to limit the number of pings to 2. You should see something very similar to:

If you did, congratulation! You’ve just confirmed your two new GENI VMs can talk to each other! If you’re a bit handy with client/server apps, what might you be able to code up at this point?

If things didn’t go so well, see the troubleshooting section below and see if you can address any issues.

Troubleshooting¶

Did this experiment fail to go as planned? Ping kenny at mozillafoundation dot org and share the details. Or, see if your problem can be addressed below.

Considerations Before Proceeding¶

Important: Because GENI is a shared platform, it’s your responsibility as a developer to be mindful of the resources you are requesting. The resources for gigabit networks are limited, so you are encouraged to release those resources as soon as you are finished using them. This will make them available to other developers and experimenters.

It’s also highly recommended that you complete the first GENI example before beginning this one. This tutorial may reference introductory concepts that were explained previously.

Tutorial¶

The steps of this tutorial are very similar to the last tutorial, up until step number 4.

Step 1. Log In to Your Management Authority¶

Let’s start by logging into your clearinghouse if you aren’t already authenticated.

Step 2. Open Flack In A New Tab¶

Next, we’ll open Flack in a new browser window or tab: http://www.protogeni.net/trac/protogeni/wiki/Flack .

Figure 2: Flack Initialized

Step 4. Let’s Look At Our Gigabit Backbones¶

Let’s take a look at the map that is displayed when the authority list loading is complete. Do you see the pink lines running across the map? Those are gigabit backbones that are available to use.

In order to utilize a gigabit backbone, we’ll provision a machine at both ends of it. In this example, one end will be in Utah and the other will be in Washington DC.

In the image below, the Utah aggregate is black and has the number “633” on it. Washington DC is grey and has “136” on top of it.

Note: Because those numbers denote the number of resources available in those aggregates, the number fluctuates, and is likely different at the time that you are reading this.

Step 5. Create a New Slice¶

Let’s create a new slice for our gigabit experiment to live in. Give your slice a different name than the one in the figure below.

Click “Ok” and wait for your slice to initialize. When it’s done, you will see:

Step 6. Find A Machine With A Gigabit Link¶

Just as before, we’re at the fun part of the experiment. Let’s go hunting for a machine that’s hooked up to the Gigabit backbone in Utah. Click on the Map panel in the left part of the screen:

Click on the Utah aggregate. You’ll see a box pop up with a breakdown of the different authorities in that grouping. For these tutorials, we’ll stick with Utah/emulab, since they have the most resources available at the time of this writing.

Click on the box labeled “utahemulab.cm” to see the list of resources available. You’ll see:

Our goal here is to find a PC hooked up to a gigabit interface, which will be labeled in the right pane as “1 Gb/s (...)”. Find one, like we did in the figure below.

Note: It’s possible, but not highly likely, that all gig resources are unavailable, in which case they’ll have a red “x” in front of them instead of a checkmark. In this case, you may need to find another utahemulab.cm grouping on the map which has a gigabit-linked PC available.

Now that you’ve found a machine with a gigabit link, drag that top-level window off to the side, and bring your slice pane back up by clicking on the name of your slice on the left side.

Click into your slice, and drag that resource window over to the left so you can see your pane:

Drag the Gigabit PC we found over to the pane, like so:

Step 7. Find Another Machine With A Gigabit Link¶

We’ve now got a machine in Utah ready to go in our Gigabit setup. But now we need a machine on the other end of a network link. Back to the map!

It was our stated intention at the beginning of this example to get a link going from Utah to Washington DC. As mentioned before, resources on GENI are limited, and it’s possible that resources aren’t available in DC when you run this experiment. If that turns out to be the case, pick another grouping of resources on the map that have a pink line running from Utah.

So let’s look at Washington DC’s grouping of resources, since the pink line representing a gigabit link runs directly from Utah.

We can see in the figure above that we have 8 resources for our use in utahemulab.cm. Click into that, and you’ll likely see something similar to the figure below.

Remote computing resources are prefixed with “pg” as opposed to “pc” like we saw in Utah. pg40 in the figure above is hooked up to a gigabit link — just what we need! We’ll take the one resource available in the window above, and drag it to our slice as before.

At this point, our Utah and DC machines should be sitting in our slice:

Step 8. Set Up a Gigabit Network!¶

We’re now ready to hook up our network. Toward the upper left part of the slice pane, a LAN resource is available for us to drag into the slice.

Before:

After:

From here, drag a line from each machine to the lan. Your final lan should look like:

This is all very similar to how we set up our network in example 1. However, this time we need to explicitly set the speed of the lan to be 1 Gb/s.

Click the blue “i”, or “information” icon on the lan. The information pane for the lan will appear. We want to change the properties of the lan, so we’ll click on the “properties” tab:

In the capacity field, enter “1000000”. The unit of the capacity field is in kb/s, so we enter one million to specify 1 gb/s.

Click “Apply” at the bottom of the screen. If you’ve made it this far, congratulations! It’s time to reserve your gigabit network! Click “Submit” at the bottom of the slice pane, and confirm.

Note: This can sometimes be a process that takes up to 15 minutes, and it occasionally fails or hangs. If it does fail, you may try again until it succeeds. If it continues to fail, it may be possible that your resource was granted to another experimenter while you were setting up your lan. You can click the “refresh” icon above the “Managers” list on the far left side of Flack to get the latest resource lists.

When your new machines and networks have been provisioned, you’ll see a green background:

Author’s Note: In the process of setting this up, I did in fact run into the provisioning failure I said might happen. You’ll notice that I had to replace pc236, which was no longer available, with pc500 from the same Utah grouping.

You now have a full provisioned Gigabit lan!

Step 9. Log in to the machines¶

Let’s now open two terminals windows - one for each box we’ve provisioned, and one for an SSH tunnel that we’ll need. Let’s start with logging into the box prefixed with pc. You should specify the address of the pc, and be sure to include the port number of needed with the -p flag.

Note: At the same time we’re going to log into the first box, we’ll set up an SSH tunnel that we can use to log into the other box. pg40 is behind a firewall, and we won’t be able to access it directly.

Your SSH login command should look like:

ssh -p [pc 1 port] [pc 1 host] -L 2224:[pc 2 host]:[pc 2 ssh port]

So with the machines we’ve been working with, we’ll do just that:

Great. Now for logging into the second box (prefixed ‘pg’). Since we have a tunnel open on port 2224 locally, we’ll connect to that. You can type the same exact command, replacing [username] with your username:

$ ssh -p 2224 [username]@127.0.0.1

Step 10. Install iperf¶

Now, let’s check our network connection. Run this in the terminal with the connection to the machine prefixed with pc:

$ cat /etc/hosts

You’ll see something similar to:

The line highlighted in the image above is the alias to the remote host. It will be prefixed by pg. Let’s keep that in mind.

Next, run this command in the window with the pc prefix. This will download and install iperf:

$ wget http://iperf.googlecode.com/files/iperf-3.0b4.tar.gz && tar -xf iperf-3.0b4.tar.gz && cd iperf-3.0b4 && ./configure && make && sudo make install ; cd

You’ll see something similar to:

[katzgrau@pc500 ~]$ wget http://iperf.googlecode.com/files/iperf-3.0b4.tar.gz && tar -xf iperf-3.0b4.tar.gz && cd iperf-3.0b4 && ./configure && make && sudo make install ; cd
--2012-08-23 14:31:23--  http://iperf.googlecode.com/files/iperf-3.0b4.tar.gz
Resolving iperf.googlecode.com... 173.194.79.82, 2607:f8b0:400e:c01::52
Connecting to iperf.googlecode.com|173.194.79.82|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 309547 (302K) [application/x-gzip]
Saving to: `iperf-3.0b4.tar.gz.1'

100%[========>] 309,547     1.31M/s   in 0.2s

2012-08-23 14:31:23 (1.31 MB/s) - `iperf-3.0b4.tar.gz.1' saved [309547/309547]

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
[truncated for brevity]
make[2]: Leaving directory `/users/katzgrau/iperf-3.0b4'
make[1]: Leaving directory `/users/katzgrau/iperf-3.0b4'

Great! We’ve just installed iperf on our first box.

Open a new, temporary terminal window. Copy and paste this command:

wget http://iperf.googlecode.com/files/iperf-3.0b4.tar.gz && scp -P 2224 iperf-3.0b4.tar.gz 127.0.0.1:~/

This will download iperf, and send it to our box in DC, pg40.

Lastly, open the terminal tab of the box prefixed with pg. Run:

tar -xf iperf-3.0b4.tar.gz && cd iperf-3.0b4 && ./configure && make && sudo make install ; cd

You now have iperf installed on both machines!

Step 11. Run iperf¶

In the terminal window that is connected to pgXXX, start the iperf server:

$ /usr/local/bin/iperf3 -s

You’ll see something similar to:

In the terminal window that is connected to pcXXX, kick off the iperf client. Be sure to enter the appropriate host for your experiment:

$ /usr/local/bin/iperf3 -c

Awesome! You’ve just gigabit data transfer from Utah to DC!

In the figure above, we can see that data was transferred at about 300 Mbits/s. That data transfer was over TCP, which is slower than UDP due to its mechanisms for avoiding data loss. We’d test with UDP, but at the time of this writing, the -u flag for iperf3 is broken!

Imagine the possibilities. You have a gigabit network to play on. What could you hack on, or create?

But remember: If you don’t need the resources you provisioned right now, it’s better that you delete your machines/slivers and release them into the pool of available resources for other hackers and experimenters.

Lastly, if you found yourself tripped up by any part of this tutorial, be sure to reach out to katzgrau@gmail.com with feedback or questions.

Tutorial¶

Step 1 - Set Up Our Network¶

Open up Flack as we did in the previous example, and create a new slice, giving it a unique name:

When you’ve submitted the new slice request, and the slice is created, click into it so you have a blank pane in front of you:

Drag three PC nodes from utahemulab.cm on the pane:

Before:

After:

Important: The node in the middle will be our network switch.

The next thing we’ll want to do is network our three machines together. For this, we’ll drag two LAN components onto the pane. We want to have two distinct networks, so we’ll network the switch and the node on the left first, and secondly, the node on the right and the switch.

Firstly:

Secondly:

Now we’re ready to configure out individual boxes.

Step 2 - Specify Ubuntu As the Operating System and Submit¶

Let’s configure all three nodes so that they have Ubuntu installed. For the node on the left, click on the information icon:

In the pane that appears, head to the “Disk Image” field, and find the option for “Ubuntu 12.04 LTS with KVM installed”. Select that:

Click “Apply” at the bottom, and do the same for the other two nodes in our network. When you’re done, each node should be marked as having the Ubuntu OS configured:

We’ve got our network set up! We’re ready to submit our request to provision these nodes. Click “Submit” on the bottom of the screen. If you hit any errors while provisioning, you can submit again.

$ ssh pc209.emulab.net
$ sudo bash

$ apt-get update
$ apt-get install openvswitch-datapath-source bridge-utils
$ module-assistant auto-install openvswitch-datapath
$ apt-get install openvswitch-brcompat openvswitch-common
$ apt-get install curl traceroute

$ nano /etc/default/openvswitch-switch

# BRCOMPAT=no

# BRCOMPAT=yes

$ /etc/init.d/openvswitch-switch restart

Step 4 - Configure Open vSwitch to Use our Ethernet Interfaces¶

This next step truly depends on the box that you’re working on. You need to know which interfaces you’re going to be working with for the next steps. To do that, run:

$ ifconfig

In the output, you’ll see a few different interfaces. The ones you need are prefixed by ‘eth’, and end with anything between 1 and 5. In the example, below, the two interfaces relevant to us are eth2 and eth4. Take note of your interfaces, and apply them to the instructions below, along with the appropriate IP addresses.

So we have interfaces eth2 and eth4. We want to configure Open vSwitch to use those interfaces as if they were the interfaces of a switch. We do this by setting up a virtual bridge interface. Type, or copy and paste these lines into your terminal:

ovs-vsctl add-br br-int
ovs-vsctl add-port br-int eth2
ifconfig eth2 0
ifconfig br-int 10.10.1.2 netmask 255.255.255.0
route add -net 10.10.1.0 netmask 255.255.255.0 dev br-int

ovs-vsctl add-br br-int2
ovs-vsctl add-port br-int2 eth4
ifconfig eth4 0
ifconfig br-int2 10.10.2.1 netmask 255.255.255.0
route add -net 10.10.2.0 netmask 255.255.255.0 dev br-int2

At this point, running another:

$ ifconfig

Should show your new bridge interfaces:

Your switch is ready to go! Now you just need an OpenFlow controller.

Step 5 - Set Up Floodlight, an OpenFlow Controller¶

Your OpenFlow switch is controlled by an OpenFlow controller. An OpenFlow Controller is just a software package that interfaces with the switch via the OpenFlow API, and pushes routing rules.

In a situation where you had a hardware-based openflow switch, the controller would reside on a separate host. We’re using a software based switch in the example, and for convenience, we’re going to install the controller on the same box as our software switch.

Again, we’ll install a package and run a series of commands:

$ apt-get install build-essential default-jdk ant python-dev uml-utilities git
$ git clone git://github.com/floodlight/floodlight.git
$ cd floodlight
$ make
$ java -jar target/floodlight.jar

We’ve just started our Floodlight controller. This controller will act as our interface to the switch. Since we just started the controller, we’ll see log messages appearing on the screen:

Step 6 - Test Our Switch¶

Because Floodlight is running in our terminal, we’re going to open up another terminal window and ssh into our switch so we can keep working. We’ll also tell the switch to listen to the Floodlight controller while we’re at it:

$ ssh pc209.emulab.net
$ sudo bash
$ ovs-vsctl set-controller br-int tcp:127.0.0.1:6633

Additionally, we’re going to open up a connection to the machine in our topology labeled PC. Your address will be different. Check your topology pane for the correct address:

$ ssh pc536.emulab.net
$ sudo bash

Now let’s install a package of utilities we can use to test switch connectivity:

$ apt-get install uml-utilities traceroute

Next, From PC, let’s ping PC-0 to make the boxes on both ends of the switch can talk to each other:

$ ping pc-1

This should yield something similar too:

Great! Right now let’s test the route our packets are taking from PC to PC-0. We want to make sure our packets are flowing through the switch we set up. We can double-check by running:

$ route -n

Which would yield:

In the output above, you can see that our route from PC to PC-1 is indeed through the switch we set up. If you would like, you can also test with traceroute, in the manner of:

$ traceroute pc-1

http://pcXXX.emulab.net:8080/ui/index.html

.. image:: ../assets/images/ex3-floodlight-ui.png

Step 8 - Put Some Routing Rules!¶

Now for the programmable networks part of our openflow experiment. Let’s get back in the terminal window that is connected to our switch. We’ll now add a rule to our switch through Floodlight that will drop any packets coming from PC (10.10.2.2). Be sure to adjust your IPs below if they don’t match up. Also, replace [DPID] with the DPID you found above:

$ curl -d '{"switch": "[DPID]", "name":"drop-flow", "src-ip": "10.10.2.2", "dst-ip":"10.10.1.1","active":"true"}' http://127.0.0.1:8080/wm/staticflowentrypusher/json

Let’s make sure that rule was pushed correctly, by issuing:

$ curl http://127.0.0.1:8080/wm/staticflowentrypusher/list/[DPID]/json | json_pp -t dumper

And you’ll see something similar to:

$ curl -X DELETE -d '{"name":"drop-flow"}' http://127.0.0.1:8080/wm/staticflowentrypusher/json

Additional Resources¶

This tutorial was adapted from this blog post for GENI: http://networkstatic.net/2012/06/openflow-openvswitch-lab/

For more on the Floodlight REST API, and what methods are available: http://www.openflowhub.org/display/floodlightcontroller/REST+API

Logging In to Flack¶

The easiest way to authenticate with Flack is to simply be logged in to your clearinghouse’s website in one window or tab, and to have Flack open in another window or tab.

Open Flack in a new tab or browser window. You should see Flack initialize in the window, as in the figure below:

Click the green “Log In” button above. Because we are already logged into our management authority via another tab, we can complete this step easily and have our private key information inserted automatically.

Along the top of the prompt, there should be a button and dropdown box that says “Download from [Select authority]” as in the figure below.

Select your management authority’s domain from the dropdown. The correct domain to choose will match up with the domain where you logged in at Step 1.

Once the correct item is selected, click the “Download” button next to the option list. Your certificate (credentials) should appear, as in the figure below:

Finally, enter the private key passphrase that you used when you signed up at your management authority when you signed up. Click “OK” on the bottom left.

At this point, Flack will attempt to retrieve resources from the various authorities. When prompted to select where to list resources from, leave all of the options checked and click “Continue”. Flack will gather the resources lists, and display a map of North America, as pictured below:

Creating a Slice¶

Click on the “New” button in the top-left corner of Flack, and enter a name for this slice. Your slice name will have to be unique from all other GENI slices, so it cannot be the same name as in the example.

When you’re finished typing the name of your slice, click “OK”. Once your slice has been created (it may take a moment), you’ll see a blank pane with the name of your slice toward the top of your screen. The blank pane on the right is where we’ll set up our experiment.

Looking For Resources¶

In the figure below, we have a new slice aptly named “tutorialslice”, which is listed in box A. In box B, there is a list of the resource types that are available to our topology.

The dropdown control at the top of box B will filter the resource types shown. You can select “raw-pc”, which will show only physical computers, emu-vmz, which will show only virtual machines, and other types of resources.

When you find a resource you’d like to use, drag it over to box C.

Configuring Compute Resources¶

In order to configure a computer resource like a VM or PC that is already in your topology, click the “information” icon of the node:

You will see a configuration pane appear, with an option to select the “Disk Image”. This is essentially the base image, and more importantly, the operating system that will be installed to your resource. At this time selecting the disk image is limited to PCs:

For both PCs and VMs, you are able to add “install services”. If you have the URL of a tar’d and gzip’d file with package source code (as most packages are distributed), you can add it here so it will be installed automatically when the machine is brought up:

Configuring Network Resources¶

You can set certain properties of your lan resources, such as the requested link capacity, latency, and packet loss rate. Do that by clicking the lan’s information icon:

And setting the appropriate values in the fields:

Additional Resources¶

Official Flack Manual: http://www.protogeni.net/trac/protogeni/wiki/FlackManual

Additional Resources¶

An emulab tutorial with more detail on firewalls: http://www.uky.emulab.net/tutorial/docwrapper.php3?docname=firewall.html

Install A Specific Operating System¶

By default, GENI nodes get Fedora Core installed on them when created. You can override this by clicking into the properties of a PC and setting its operating system. Your choices are fairly standard: Ubuntu, Redhat, FreeBSD, or Fedora. Additionally, there are distros that come pre-packages with tools like KVM (Kernel Virtual Machine), which might be handy if you plan to host virtual machines, for instance.

Install Packages Automatically¶

If you have the URL to a package you would like to have installed for you, you can add an “install service”. When the machine is created, the package will be installed in the directory that you specify. This option is available in the properties pane of the PC.

Changing Network Characteristics¶

In the image below, we have two nodes that have yet to be networked together:

In the resource list drop down, we’re going to select the ‘delay’ option:

From there, we’ll want to drag a delay node (or two) onto the pane

Finally, let’s drag links between the nodes so the LANs are set up automatically:

At this point, let’s click the i icon on the delay node. We’ll see a properties pane appear with areas to set capacity, latency, and packet loss on either incoming or outgoing traffic. At this time, capacity is in kb/s, latency in milliseconds, and packet loss as a number between 0 and 1.

Apply your settings to finish it off.

Changing Network Speed Artificially¶

In the image below, we have two nodes that have yet to be networked together:

In the resource list drop down, we’re going to select the ‘delay’ option:

From there, we’ll want to drag a delay node (or two) onto the pane

Finally, let’s drag links between the nodes so the LANs are set up automatically:

At this point, let’s click the i icon on the delay node. We’ll see a properties pane appear with an area to set the link capacity.

Apply your settings, and you can now build and run your app as if you’re on a gig network.

Requesting Gigabit Links On GENI¶

When you’re setting up your experiment architecture, you have the ability to set the bandwidth properties of the network that the nodes are in. There are three important steps in making sure you get a gigabit link:

1. Select nodes with have a gigabit interface
2. Select nodes that are on a gigabit backbone (if applicable)
3. Request that network link between the nodes is for 1000000 kb/s

1. Selecting the Right Nodes¶

In the image below, we already have a slice set up, and we’re selecting nodes for our experiment. By navigating to the map, and clicking into an aggregate, we get a list of nodes available to us:

Clicking through, we finally find one available that has a 1 Gb/s connection. This is the type of node we’ll want to use. The next step would be simply dragging that node to our project pane (along with any others that we need).

2. Getting on A Gig Backbone¶

Now that we’ve got those gigabit interfaces available, we want to use them! If we’re running a multi-site experiment, we need to find nodes on another site that is linked on a gigabit backbone to the first node(s) we selected.

Looking at our Flack map, we see that we have resources connected to out first node’s aggregate by a pink/magenta line. In this case, we’re focused on an aggregate in Washington DC.

Clicking into into that aggregate, we again get a list of resources. Our goal is to find another machine with a gig interface.

Now that we’ve found one, we’ll drag that resource onto our map too, and create a network between them:

And:

And:

We’re ready to finalize our gigabit network now.

3. Requesting the Right Bandwidth¶

Now that we’ve nearly got our network set up, we’re ready to request gigabit speed between our nodes. We’ll click the information icon on our lan. We’ll see a pane that opens up. Click the “properties” tab. We want to specify that there should be a 1000000 kb/s connection on the network, like so:

When we click “Apply,” our project will reflect our requested speed in the experiment pane:

You’re now ready to submit your requested topology, and use the gig network.

Introduction¶

There is sometimes a need for network architects (or developers like you) to define the behavior of their networks in a custom manner. For example, the architect may want a network switch where they can control how packets are routed, or even define a custom protocol.

Historically, this was possible via closed, proprietary hardware that can be prohibitively expensive or impossible to obtain by researchers and experimenters. Yet, the need for this functionality exists in order to run wide-scale projects implementing new, experimental protocols, or even routing rules for basic network architecture needs.

OpenFlow is an API that a growing number of switch manufacturers (like IBM, or HP and its Procurve switch) are implementing, and it’s the leading architecture for Software defined Networking (SDN). It’s an open API that allows external software to define the routing rules, or “control plane” of routing hardware.

Igor Gashinsky gives an excellent overview on the need for SDN from an industry perspective below, but also illustrates the need to separate the control plane of a switch from the physical hardware.

Igor Gashinksy of Yahoo! At OpenFlow Symposium from Stephen Foskett on Vimeo.

OpenFlow is also implemented by Open vSwitch, a software-based switch you can install on an ordinary machine.

On the GENI network, you can request access to a physical OpenFlow-enabled Procurve, or simply install Open vSwitch on a machine and configure it from there.

Before going much further, you may want to know: What exactly can I do with OpenFlow?

Installing Open vSwitch¶

The preferred method for experimenters looking to play with OpenFlow is option #1. Open vSwitch is a software package which you can install on an ordinary node and have it act as a switch which implements OpenFlow.

This is also the easiest option with the lowest barrier to entry. The third tutorial, Programming Networks with OpenFlow will walk you through the exact steps required to make this work.

Using the GENI Mesoscale Infrastructure¶

GENI’s Mesoscale environment is a perfect place to run a large-scale OpenFLow experiment beyond a software-based option like Open vSwitch. Because this option requires coordination with the GENi Project Office (GPO), you may want to get you experiment running with Open vSwitch first, if that’s an option.

From there, reach out to help@geni.net for more information on obtaining a subnet for your experiment. A third-party, but detailed guide on how you might get your project running on the Mesoscale infrastructure is available on Github.

Reserve an HP Switch and install OpenFlow¶

This is an advanced option for getting up and running on OpenFlow. If you’re ambitious enough to reserve HP OpenFlow hardware, or simply have the need, reach out to help@geni.net for more information.

Additional Resources¶

• A guide on installing Open vSwitch: http://networkstatic.net/2012/06/openflow-openvswitch-lab/
• A guide on using GENI Mesoscale: https://github.com/aaronorosen/GeniTutorial/wiki/Geni-Mesoscale-Tutorial