Not all APIs are created equally.

Automation is starting to become the new catch phrase in the networking Industry. It seems like 2014 is the year that marketing groups from different vendors have been touting APIs on their products. Skeptical networking engineers however have claimed that an API does not mean that their jobs are getting easier. In fact it’s been making their jobs a little harder.

Before a networking engineer could strictly focus on pure networking. But now, network engineers are increasingly required to know more and more on how to code or at least know how to read code.

Just because you have an API doesn’t mean all the devices can and will play nice with each other. 

We can see this by looking at three different platforms. 

OpenStack         Contrail        Junos

Now let’s look at their API for data retrieval/configuration

REST                  REST           Netconf

Ok now you have two platforms that use one type of API and a third platform that uses a different API.

Let's look at the resulting Data Structure response

JSON                JSON XML

Again we have two platforms that have the same Data Structure and a third with a different Data Structure.

You might say, ok, at least two of these platforms have the same API and with the same data structure things should be good for both of them right? Actually as they say, the devil is in the details.

I can illustrate this just by looking at a simple IPv4 subnet. 

On Openstack the data abstracted looks like this

{ "networks": [ { "contrail:subnet_ipam":  [  { "subnet_cidr": "12.1.1.0/24",  } ] } ] }

On Contrail it looks like this

{"virtual-network":{ "network_ipam_refs":[ { "attr": { "ipam_subnets": [ { "subnet": { "ip_prefix": "12.1.1.0", "ip_prefix_len": 24 }, } ] }, } ], } }

You can see that one platform combines the subnet with the mask while the other one separates it. For a DevOps engineers and Network Engineers this is annoying. It’s like having to learn different Network Operating systems. The goal of an API should be to allow a simplified abstraction layer.

APIs need to be standardized. Openflow is a good attempt at this. Openflow requires the underlay to have a common protocol in order to allow a controller to programmatically configure them. The networking industry has done a great job at standardizing protocols but a sorry job at creating a common API standard. Maybe the IETF needs to jump in on this. A standardized API could ultimately make our jobs that much more easier.

How to use cURL to spin up an OpenStack VM

I know this is a little long but there are dependencies on how this works. Hopefully you can understand it. The last cURL command is the piece that puts it all together and spins up a VM. The last command will spin up a instance called "instance1", use a flavor and an cirros image and place it in the "VN2" network

LAPTOP:SCRIPTS$cat openstack-create.sh
TOKEN=""

#Do a POST to GET the AUTH TOKEN
TOKEN=$(curl -i \
  -H "Content-Type: application/json" \
  -d '
{ "auth": {
    "identity": {
      "methods": ["password"],
      "password": {
        "user": {
          "name": "admin",
          "domain": { "id": "default" },
          "password": "password",
          "tenantName": "admin"
        }
      }
    },
    "scope": {
      "project": {
        "name": "demo",
        "domain": { "id": "default" }
      }
    }
  }
}' \
  http://192.168.1.4:5000/v3/auth/tokens | grep X-Subject-Token: | sed -e "s/X-Subject-Token: //")

#STORE TOKEN
header='X-Auth-Token: '$TOKEN
#echo "$header"

#NOW USE the AUTH TOKEN TO CHECK TENANTS
curl -X GET http://192.168.1.4:5000/v2.0/tenants/ -H "$header" | python -m json.tool

#CHECK IMAGES FOR TENANT/PROJECT CALLED DEMO
# TENANT ID FOR DEMO = 6d0ebb466f01454e897e518289b3b785
echo "IMAGES"
curl -H "X-Auth-Project-Id: demo" -H "$header" -X GET http://192.168.1.4:8774/v2/6d0ebb466f01454e897e518289b3b785/images | python -m json.tool


# CHECK FLAVORS
echo "FLAVORS"

curl -H "X-Auth-Project-Id: demo" -H "$header" -X GET http://192.168.1.4:8774/v2/6d0ebb466f01454e897e518289b3b785/flavors | python -m json.tool

# CHECK SERVERS
echo "SERVERS"

curl -H "X-Auth-Project-Id: demo" -H "$header" -X GET http://192.168.1.4:8774/v2/6d0ebb466f01454e897e518289b3b785/servers | python -m json.tool

# CHECK NETWORK ID - THIS IS NEUTRON AND HAS IT'S OWN PORT NUMBER 9696
echo "NETWORKS"

curl -H "Content-Type: application/json" -H "$header" -X GET http://192.168.1.4:9696/v2.0/networks | python -m json.tool

#CREATE VM
echo "CREATE VM"

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -H "X-Auth-Project-Id: demo" -H "$header" -X POST http://192.168.1.4:8774/v2/6d0ebb466f01454e897e518289b3b785/servers -d '{"server": {"name": "instance1", "imageRef": "d5f30886-e8ce-4e89-9709-c49a40717b5b", "flavorRef": "2", "max_count": 1, "min_count": 1,"key_name": "xyz", "networks": [{"network": "bd912f99-c345-4f3e-8026-88e1dde42255", "uuid": "bd912f99-c345-4f3e-8026-88e1dde42255"}]}}'

-----------------------

I don’t execute the script and print it out as it would not look too pretty.

The most annoying part of this is all the curly brackets I had to dig around for the formatting of "networks" part as this was the hardest part.

I found out the formatting was like this:

networks: [{"port": String, "fixed_ip": String, "uuid": String, "network": String}, {"port": String, "fixed_ip": String, "uuid": String, "network": String}, ...]

Here's the api that I found useful:

http://developer.openstack.org/api-ref-compute-v2.html

Below is how it would look like if you executed the commands directly on the compute node. Note how image-list has the id for "imageRef"

root@openstack:/# nova flavor-list
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
| 1  | m1.tiny   | 512       | 1    | 0         |      | 1     | 1.0         | True      |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         | True      |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+
root@openstack:/# nova image-list
+--------------------------------------+--------------+--------+--------------------------------------+
| ID                                   | Name         | Status | Server                               |
+--------------------------------------+--------------+--------+--------------------------------------+
| d894be99-6c35-46be-b0bf-01149d724aec | c1           | ACTIVE |                                      |
| d5f30886-e8ce-4e89-9709-c49a40717b5b | cirros       | ACTIVE |                                      |
| 10a33e93-84da-43cb-862b-fbf0b0ea9d64 | ubuntu-cloud | ACTIVE |                                      |
+--------------------------------------+--------------+--------+--------------------------------------+
root@openstack:/# nova keypair-list
+------+-------------------------------------------------+
| Name | Fingerprint                                     |
+------+-------------------------------------------------+
| xyz  | 88:c3:9d:32:d0:33:c9:1d:c3:9a:ed:cc:51:0c:55:30 |
+------+-------------------------------------------------+
root@openstack:/# nova network-list
+--------------------------------------+-----------------------------+------+
| ID                                   | Label                       | Cidr |
+--------------------------------------+-----------------------------+------+
| 4c626c1c-26e2-45a3-8fa6-336fb297ffea | west-network                | None |
| 3d608137-9e84-452a-9b9b-dd055f21a095 | right-nw                    | None |
| cfcd7fd5-4dae-4b0d-bf77-41df699e7bfb | vn-blue                     | None |
| c72cad3d-8f41-48a9-8ebc-73f6f4d8be87 | VN1                         | None |
| 80d71671-0b00-422e-911f-cb599e5b462a | external-floating-ip        | None |
| a8d51e11-34a7-4e72-bd91-7e51d7fbdc6c | default-virtual-network     | None |
| c9a0a923-d583-4513-b2b6-c2c6d805ff18 | management-network-poc-demo | None |
| f7681c05-30e9-4951-b2fb-1796a0b7c41f | left-vn                     | None |
| 3a156ca5-54de-4654-b344-5e75a1a4369c | ip-fabric                   | None |
| bd912f99-c345-4f3e-8026-88e1dde42255 | VN2                         | None |
| a77a4bec-51d8-4a6f-8e93-01d8e2d21512 | management                  | None |
| d4b022f2-88ff-4d63-84cb-c79b76153799 | network1                    | None |
| 632269aa-74ac-4306-820d-3b27835f5951 | __link_local__              | None |
| f2bc533f-ba1d-412d-a788-875670883e9b | svc-vn-mgmt                 | None |
| 1072c78f-f464-4a6f-91a5-0caac6cf2f26 | east-network                | None |
| 77c40c19-07d1-4416-86d5-1a00b2545a88 | public-fip                  | None |
+--------------------------------------+-----------------------------+------+
root@openstack:/#

Using python to connect remotely to Openstack

In order to beef up my DevOps skills I decided to see if I could connect through a REST API using python. I found out that there are some python modules built specifically for this.

One such module is called novaclient. You would need pip on your machine to do this.

sudo easy_install pip

Then you can use pip to install the nova client.

sudo pip install python-novaclient

If you want to look at the details of the python module it can be found here:

https://github.com/openstack/python-novaclient

This is where I look more into the code and found out it’s capabilities

You can pass parameters to the “client” function.

def __init__(self, username=None, api_key=None, project_id=None,

auth_url=None, insecure=False, timeout=None,

proxy_tenant_id=None, proxy_token=None, region_name=None,

endpoint_type='publicURL', extensions=None,

service_type='compute', service_name=None,

volume_service_name=None, timings=False, bypass_url=None,

os_cache=False, no_cache=True, http_log_debug=False,

auth_system='keystone', auth_plugin=None, auth_token=None,

cacert=None, tenant_id=None, user_id=None,

connection_pool=False, session=None, auth=None,

completion_cache=None):

First I built a login function that will pass these parameters which I can call from my main script.

login.py

-----------

def get_nova_credentials():

 cred =  {}

 cred['username'] = "admin"

 cred['api_key'] = “password”

 cred['auth_url'] = "http://<openstack-ip>:5000/v2.0"

 cred['project_id'] = "demo"

 cred['service_type'] = "compute"

 return cred

Now in my main script I can import the client and my credentials

server-list.py

-----------------

#!/usr/bin/env python

import novaclient

from novaclient.v1_1 import client

from login import get_nova_credentials

credentials = get_nova_credentials()

#Pass credentials to the client function.

nova = client.Client(**credentials)

#grab the list of servers and print out the id, names and status

vms = nova.servers.list(detailed=True)

for vm in vms:

  print vm.id, vm.name, vm.status

—————

script in action.

laptop$ python server-list.py 

f5801333-5d81-496c-b257-e589ca36e944 Cirros-VM2 ACTIVE

098270e0-e5fb-4ea6-a1f1-2dfca11a409d Cirros-VM1 ACTIVE

So what's the big deal? Why do this when you can use the Horizon web ui?

Now that I have a basic understanding of this module, I can start automating things such as building VMs, virtual networks, etc in a scaled and precise manner.

How Contrail communicates with the underlay

Contrail typically consists of a cluster of nodes. The three main nodes are the config, control and compute. The config node is where the Openstack Horizon and Contrail Controller exists. The Control node is used to form a MP-BGP session to a gateway router. The Compute node hosts all the VMs and virtual networks.

You can think of Contrail as a PE router as pretty much this is what a gateway router perceives the other end of the connection. Contrail uses a vRouter and when you configure virtual networks you have the ability to add a route-target to that virtual network. On the Gateway router you would create VRFs to associate with the corresponding virtual networks and prefixes can be exchanged. Data plane traffic will traverse through an MPLS tunnel between Contrail and the Gateway router. It's at the gateway router where you would "leak" the received Contrail virtual network into the main routing instance of the gateway router.

Here I use an Juniper MX as the gateway router. When I first setup contrail I used the testbed.py script to add the mx gateway router.

It's called ext_router = [ip address]

Then in contrail webui I should see the BGP session. You can however add this post contrail installation.

On the MX, I configure an iBGP session to connect with the Contrail control node.

user@router# show protocols
mpls {
    interface all;
}
bgp {
    group IBGP-CONTRAIL {
        type internal;
        local-address 192.168.10.11;
        family inet-vpn {
            unicast;
        }
        neighbor 192.168.10.2;
    }
}

Then in Contrail config node I create a virtual network and add a route target.

I create a corresponding VRF on the MX with the route target.

user@router# show routing-instances
VRF1 {
    instance-type vrf;
    interface lt-3/0/0.3;
    route-distinguisher 1.1.1.1:101;
    vrf-target target:64512:101;
    routing-options {
        static {
            route 0.0.0.0/0 next-hop 192.168.12.1;
        }
    }
} 

I check to see the BGP session established.

user@router# run show bgp summary                                     
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0         
                       8          8          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.2          64512       5724       6292       0       3 1d 23:08:56 Establ
  bgp.l3vpn.0: 8/8/8/0
  VRF1.inet.0: 3/3/3/0

The Virtual Network IP addresses for the VMs will be sent.

user@router# run show route receive-protocol bgp 192.168.10.2 

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
* 11.1.1.1/32             192.168.10.3                 100        ?
* 11.1.1.5/32             192.168.10.3                 100        ?
* 11.1.1.7/32             192.168.10.3                 200        ?

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
  192.168.10.3:7:11.1.1.1/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.5/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.7/32                   
*                         192.168.10.3                 200        ?

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
1                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
2                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
13                 *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
299904             *[VPN/170] 1d 23:14:00
                    > to 192.168.11.1 via lt-3/0/0.1, Pop     
299936             *[VPN/170] 1d 12:47:38
                      receive table VRF1.inet.0, Pop     
299952             *[VPN/170] 1d 12:47:38
                    > to 192.168.12.1 via lt-3/0/0.3, Pop     

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.10.3:7:11.1.1.1/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
192.168.10.3:7:11.1.1.5/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
192.168.10.3:7:11.1.1.7/32               
                   *[BGP/170] 1d 12:26:14, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28


Note the dynamic MPLS GRE tunnel is created. You will need to create one on the MX.

user@router# show chassis
fpc 3 {
    pic 0 {
        tunnel-services;
    }
}

user@router# show routing-options
static {
    route 0.0.0.0/0 next-hop 10.161.1.1;
}
autonomous-system 64512;
dynamic-tunnels {
    dynamic_overlay_tunnels {
        source-address 192.168.10.11;
        gre;
        destination-networks {
            192.168.10.0/24;
        }
    }
}



PoC-Demo.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 1d 13:05:36
                    > to 192.168.12.1 via lt-3/0/0.3
11.1.1.1/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
11.1.1.5/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
11.1.1.7/32        *[BGP/170] 1d 12:44:12, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28
192.168.12.0/24    *[Direct/0] 1d 13:05:36
                    > via lt-3/0/0.3
192.168.12.2/32    *[Local/0] 1d 13:05:36
                      Local via lt-3/0/0.3

LT interfaces are created to allow the virtual network traffic to communicate between the VRF and the main routing instance. You could also use RIB groups and Policies to do the same thing.

    lt-3/0/0 {
        unit 2 {
            encapsulation ethernet;
            peer-unit 3;
            family inet {
                address 192.168.12.1/24;
            }
        }
        unit 3 {
            encapsulation ethernet;
            peer-unit 2;
            family inet {
                address 192.168.12.2/24;
            }
        }
    }

You then need to make sure the interface that is connecting to the Contrail network is using MPLS.


interfaces {

    ge-3/1/1 {
        unit 0 {
            family inet {
                address 192.168.10.11/24;
            }
            family mpls;
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 1.1.1.1/32;
            }
            family iso {
                address 49.0002.0010.0100.1001.00;
            }
        }
    }
}

One thing you should be aware of is the next-hop of the route advertised by contrail points to the IP address of the Compute Node and not the control node.
user@router# run show route 11.1.1.1/32 detail

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
11.1.1.1/32 (1 entry, 1 announced)
        *BGP    Preference: 170/-101
                Route Distinguisher: 192.168.10.3:7  <<< Contrail's RD
                Next hop type: Indirect
                Address: 0x94f4a28
                Next-hop reference count: 3
                Source: 192.168.10.2
                Next hop type: Router, Next hop index: 660
                Next hop: via gr-3/0/0.32770, selected
                Label operation: Push 24
                Label TTL action: prop-ttl
                Session Id: 0xd
                Protocol next hop: 192.168.10.3  <<<< IP of compute node
                Push 24
                Indirect next hop: 0x9574410 1048574 INH Session ID: 0xe
                State: <Secondary Active Int Ext ProtectionCand>
                Local AS: 64512 Peer AS: 64512
                Age: 1d 13:54:24     Metric2: 0
                Validation State: unverified
                Task: BGP_64512.192.168.10.2+34735
                Announcement bits (1): 1-KRT
                AS path: ?
                Communities: target:64512:101   << RT from contrail
                Import Accepted
                VPN Label: 24
                Localpref: 100
                Router ID: 192.168.10.2         <<<< IP of control node
                Primary Routing Table bgp.l3vpn.0

How to access a VM in OpenStack + Contrail

When you first spin up VMs in an Openstack + Contrail environment you don’t have many choices on how to access your VM. You can go through the Openstack Webui to access via the console. 

But that method is not ideal as you cannot do things such as copy and paste.

Another method is to access the VM from the Contrail compute node. All VMs are stored on the compute node of the Contrail cluster. 

Or you can inject an ssh-key into your VM so you can access the VM from the compute node.

First you create your ssh-key.

Select the Access and Security tab in Openstack.

Choose Keypairs

 Then create a key pair.

You will then be able to download the "key" to your local computer. You will need to copy this key onto the Compute node.

MyComputer$ scp ssh-key.pem root@172.16.100.104:~/.ssh

I chose to place this key in the .ssh directory of the Compute node.

Compute-Node:~/.ssh$ ls

authorized_keys  id_rsa  id_rsa.pub  known_hosts  ssh-key.pem

Next you create your VM.

Make sure you choose the ssh-key you created.

You can see if your ssh-key was injected from the horizon dashboard.

Note: One of the problems I see in Horizon is the inability of injecting the ssh-key after the VM was created. There is no was to edit and do this post-creation. Pretty annoying especially if there are many ssh-keys and you forget to choose the correct one.

Next go back to your compute node.

Issue the netstat -rn command.

When a VM is created the Compute node generates a 169.254.x.x link local address. It's similar to a loopback address and is only accessible on the local machine and cannot be accessed over the network.

Compute-Node$ netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         172.16.100.1    0.0.0.0         UG        0 0          0 vhost0

169.254.0.3     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.4     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.5     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.6     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.7     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.8     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

169.254.0.9     0.0.0.0         255.255.255.255 UH        0 0          0 vhost0

172.16.100.0    0.0.0.0         255.255.255.0   U         0 0          0 vhost0

Then you can ssh into the vm of your choice with -i parameter and path-to/ssh-key. For example:

Compute-Node$ssh -i ~/.ssh/ssh-key.pem cirros@169.254.0.3

Cirros-VM1$ whoami

cirros

You can see the ssh-key authorized hosts file in the .ssh directory of the VM

$ more authorized_keys 

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCe9TjQfiVTiidtt2qUICwK/7DArACsjLWDkx7Esvu6vWS8MahyrlgNkeQtaFDx7wub5LaHesqq6wj2pKDX07RWxylAkxShqy2+ZIoOgJqMBr0vfq4xpp2qU7fPiAq4YV3CdqTwOggnNHQNeGgpM6406IJSJcVYJVqTC3/3SsFgxzva4UqNgA3mjRQxSsmVxc6jVHVKfAYQ8+fDFNniNjY+q9qvihtAXwmLGfv/gxE/N01aMC+MH1b5cmj2o7WNpbt5qGDyrf3jB6rqz5CI95XS0MjvaScTWKb5ul0yuWTkp1zcPY4vzDaFjc0Fl7627Lm2IuQZg76dgKl3rnnPtXbv Generated by Nova

How to install Contrail on a single node Ubuntu system.

I wanted to install and test out a contrail set on a single node system.

I first started with a clean Ubuntu server. Contrail seems to have support on 12.04LTS version of code (12.04.03 to be precise). So I downloaded the amd64-iso file and did a fresh install on a system.

After it came up I did some package adds to get the machine prepped.

First I made sure I was root.
sudo su
passwd
I gave root a password. This is to be used later when Contrail tries to install packages using root.

This are the files I noticed while looking at different blogs on the OpenContrail.org website. I'm not sure if all these dependencies are needed. This is what worked for me. (I'm pretty sure git isn't needed here, but I use git for other projects so I did it anyways.)

apt-get install  -y git-core ant build-essential pkg-config linux-headers-3.2.0-35-virtual
apt-get install -y scons git python-lxml wget gcc patch make unzip flex bison g++ libssl-dev autoconf automake libtool pkg-config vim python-dev python-setuptools python-paramiko

apt-get update


Then I went onto the Juniper website and downloaded the software.

You DON'T have to install Openstack first to get this going.

This package has Contrail plus the Openstack Havana version built in.

I ftp or scp this file onto my ubuntu server.

I place it in the /tmp directory.

Next I install the package.

dpkg -i contrail-install-packages_1.05.1-234~havana_all.deb

The packages get placed in a contrail directory.

cd /opt/contrail/contrail_packages

Then I run the setup shell script.

./setup.sh

After this, you'll need to create or modify a testbed.py script. This tells contrail how install the compute, storage and control nodes. Since this is an all-in-one system, I going to clone the single box example and modify it.

cd /opt/contrail/utils/fabfile/testbeds/

cp testbed_singlebox_example.py testbed.py

Next I edit the file.

vi testbed.py

In order to execute this file, you need to back up a few directories (I know it's lame, the command doesn't seem to execute in the correct directory.)

cd /opt/contrail/utils

Next you will issue the fabric command

fab install_contrail

fab setup_all

After a few minutes, the scripts will run and more packages will be installed and configured to the specifications of your testbed file.

The server will reboot automatically.

After it comes back up sudo su when you login again

Then you need to source your authentication files.

keystonerc  and openstackrc  are located in /etc/contrail

source /etc/contrail/keystonerc
source /etc/contrail/openstackrc

Here are some commands  you can issue after installation to check the state of openstack

openstack-status

nova-manage service list

root@ubuntu:/home/user# openstack-status
== Nova services ==
openstack-nova-api:           active
openstack-nova-compute:       active
openstack-nova-network:       inactive (disabled on boot)
openstack-nova-scheduler:     active
openstack-nova-volume:        inactive (disabled on boot)
openstack-nova-conductor:     active
== Glance services ==
openstack-glance-api:         active
openstack-glance-registry:    active
== Keystone service ==
openstack-keystone:           active
== Cinder services ==
openstack-cinder-api:         active
openstack-cinder-scheduler:   active
openstack-cinder-volume:      inactive (disabled on boot)
== Support services ==
mysql:                        active
libvirt-bin:                  active
rabbitmq-server:              inactive (disabled on boot)
memcached:                    inactive (disabled on boot)
== Keystone users ==

+----------------------------------+---------+---------+---------------------+
|                id                |   name  | enabled |        email        |
+----------------------------------+---------+---------+---------------------+
| 2fa7b037efe1437a9045eab35f446511 |  admin  |   True  |  admin@example.com  |
| 6bb262f464a546b391b30879f1e8f10b |  cinder |   True  |  cinder@example.com |
| fdedd2c1a26d44ab9e83f000383cedf3 |   demo  |   True  |   demo@example.com  |
| cdc338c569af42cf87ba7bc7e7e161a8 |  glance |   True  |  glance@example.com |
| 2e28f88537064d97bfed64ad62f2fc66 | neutron |   True  | neutron@example.com |
| c09776b2f84744198d46c0361bfc1070 |   nova  |   True  |   nova@example.com  |
+----------------------------------+---------+---------+---------------------+
== Glance images ==
ID                                   Name                           Disk Format          Container Format     Size         
------------------------------------ ------------------------------ -------------------- -------------------- --------------
== Nova instance flavors ==
m1.medium: Memory: 4096MB, VCPUS: 2, Root: 40GB, Ephemeral: 0Gb, FlavorID: 3, Swap: 0MB, RXTX Factor: 1.0, public, ExtraSpecs {}
m1.tiny: Memory: 512MB, VCPUS: 1, Root: 1GB, Ephemeral: 0Gb, FlavorID: 1, Swap: 0MB, RXTX Factor: 1.0, public, ExtraSpecs {}
m1.large: Memory: 8192MB, VCPUS: 4, Root: 80GB, Ephemeral: 0Gb, FlavorID: 4, Swap: 0MB, RXTX Factor: 1.0, public, ExtraSpecs {}
m1.xlarge: Memory: 16384MB, VCPUS: 8, Root: 160GB, Ephemeral: 0Gb, FlavorID: 5, Swap: 0MB, RXTX Factor: 1.0, public, ExtraSpecs {}
m1.small: Memory: 2048MB, VCPUS: 1, Root: 20GB, Ephemeral: 0Gb, FlavorID: 2, Swap: 0MB, RXTX Factor: 1.0, public, ExtraSpecs {}
== Nova instances ==

root@ubuntu:/home/user# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth ubuntu                               internal         enabled    :-)   2014-09-18 17:26:26
nova-console     ubuntu                               internal         enabled    :-)   2014-09-18 17:26:26
nova-scheduler   ubuntu                               internal         enabled    :-)   2014-09-18 17:26:26
nova-conductor   ubuntu                               internal         enabled    :-)   2014-09-18 17:26:27
nova-compute     ubuntu                               nova             enabled    :-)   2014-09-18 17:26:26


You are now ready to login to the Horizon Dashboard to configure your Openstack platform.

http://x.x.x.x/horizon

To log into your contrail controller change the port number to 8143

https://x.x.x.x:8143/

Next time I'll show how create a MP-BGP connection to a MX gateway to all traffic from the virtual network to connect to a physical network.

Another way to delete a VM instance in Openstack

For some reason I could not delete a VM in my Openstack Cloud Platform. I tried to terminate the instance in the Horizon GUI but that did not work. I also tried to do it from the cli via:

# nova delete <uuid>

That did not work either. So I researched in on the web and noticed that the instances are kept in two places. The first place is in the nova directory:

/var/lib/nova/instances
[root@sdnnode2 instances]# ls
51426435-9f72-4077-91a4-6c89ea53894f 58fd794e-048a-4135-b064-17fb6bb3682b  bbc44d49-62f7-4cdc-8e7f-564190c6f043  locks _base d0ce9950-752c-406f-91e0-3bd0ff0f488d  snapshots

looking into those uuid directory names contained the following:

[root@sdnnode2 51426435-9f72-4077-91a4-6c89ea53894f]# ls
console.log  disk  libvirt.xml

However deleting these files still would not delete the VM from appearing in the Horizon GUI. These are the files that Openstack would use to spin up the VM.

The vm existed in MySQL.

I found a handy sql_guide to brush me up on SQL commands. Then I had to access the SQL server with the correct credentials. If you don't remember it you can find it in the nova.conf file located in:

/etc/nova/nova.conf

[root@sdnnode2 nova]# cat nova.conf | grep sql
sql_connection = mysql://nova:<mypassword>@10.161.38.4/nova

Then to access the SQL server:
[root@sdnnode2 nova]# sudo mysql -u nova -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 166
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

show databases gives us.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| nova               |
| test               |
+--------------------+



mysql> use nova;

 Database changed
mysql> show tables;
+--------------------------------------------+
| Tables_in_nova                             |
+--------------------------------------------+
| agent_builds                               |
| aggregate_hosts                            |
| aggregate_metadata                         |
| aggregates                                 |
| block_device_mapping                       |
| bw_usage_cache                             |
<TRUNCATED>
| volume_usage_cache                         |
| volumes                                    |
+--------------------------------------------+
115 rows in set (0.00 sec)


Here we can then find the instance uuid. First you could figure out the fields in a particular table:


mysql> describe key_pairs;
+-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| created_at  | datetime     | YES  |     | NULL    |                |
| updated_at  | datetime     | YES  |     | NULL    |                |
| deleted_at  | datetime     | YES  |     | NULL    |                |
| id          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name        | varchar(255) | YES  |     | NULL    |                |
| user_id     | varchar(255) | YES  | MUL | NULL    |                |
| fingerprint | varchar(255) | YES  |     | NULL    |                |
| public_key  | mediumtext   | YES  |     | NULL    |                |
| deleted     | int(11)      | YES  |     | NULL    |                |
+-------------+--------------+------+-----+---------+----------------+
9 rows in set (0.00 sec)



Then you can list the objects that are in the field you are searching.

mysql> select name from key_pairs;
+----------+
| name     |
+----------+
| my-key   |
| root-key |
+----------+
2 rows in set (0.00 sec)


I found out there are 5 tables you need to modify to get delete the VM and the should be done in this order:
security_group_instance_association
block_device_mapping
instance_info_caches
fixed_ips
instances

So I needed to find the uuid of the VM I was going to delete.

mysql> select uuid,hostname from instances where deleted=0;
+--------------------------------------+-----------+
| uuid                                 | hostname  |
+--------------------------------------+-----------+
| bbc44d49-62f7-4cdc-8e7f-564190c6f043 | right-vm1 |
| d0ce9950-752c-406f-91e0-3bd0ff0f488d | left-vm1  |
| 51426435-9f72-4077-91a4-6c89ea53894f | test-vm    |
| 58fd794e-048a-4135-b064-17fb6bb3682b | left-vm2  |
+--------------------------------------+-----------+
4 rows in set (0.00 sec)

I then started my process. I checked to see if the instance was not already marked deleted by checking for the condition of "deleted=0";

mysql> select instance_uuid from security_group_instance_association where deleted=0;
Empty set (0.00 sec)

I didn't have a security group association so I proceeded with the next step.

mysql>  DELETE FROM block_device_mapping WHERE instance_uuid = "51426435-9f72-4077-91a4-6c89ea53894f";


Query OK, 1 row affected (0.00 sec)


I then followed the other steps

mysql> select instance_uuid from instance_info_caches where deleted=0;
DELETE FROM instance_info_caches WHERE instance_uuid = "51426435-9f72-4077-91a4-6c89ea53894f";
Query OK, 1 row affected (0.00 sec)

mysql>select instance_uuid from fixed_ips where deleted=0;
Empty set (0.00 sec)

I didn't have a fixed_ip so I went to the last step, which was to changed the "deleted" field.
You need to make sure the deleted number matches the id.

mysql> select id,deleted,uuid from instances;
+----+---------+--------------------------------------+
| id | deleted | uuid                                 |
+----+---------+--------------------------------------+
| 31 |      31 | 028ff14b-55f2-4aa2-8fd9-37d70ff26a3b |
| 11 |      11 | 4c60dd0a-ae81-45e0-931b-c996e89b10d7 |
| 47 |      47 | 4ca2687f-35a4-4248-9777-24d9317e4c20 |
| 25 |      25 | 512747be-b608-423e-ac81-54f2ebcd7e58 |
| 35 |      0 | 51426435-9f72-4077-91a4-6c89ea53894f |

mysql> UPDATE instances SET deleted = 40 where uuid = "51426435-9f72-4077-91a4-6c89ea53894f";Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select id,deleted,uuid from instances;
+----+---------+--------------------------------------+
| id | deleted | uuid                                 |
+----+---------+--------------------------------------+
| 31 |      31 | 028ff14b-55f2-4aa2-8fd9-37d70ff26a3b |
| 11 |      11 | 4c60dd0a-ae81-45e0-931b-c996e89b10d7 |
| 47 |      47 | 4ca2687f-35a4-4248-9777-24d9317e4c20 |
| 25 |      25 | 512747be-b608-423e-ac81-54f2ebcd7e58 |
| 35 |      35 | 51426435-9f72-4077-91a4-6c89ea53894f |


Finally the instance was no longer in my Openstack server.

[root@sdnserver1 nova]# nova show 51426435-9f72-4077-91a4-6c89ea53894f
ERROR: No server with a name or ID of '51426435-9f72-4077-91a4-6c89ea53894f' exists.

Using CURL to access Openstack

So I'm exploring the API for Openstack to see how I can extract information from an Openstack platform.

I found the examples can be found here.  While the API is located here.

This could be useful as in the future I could script a networking device (Switch, Router, Firewall, etc) to automate and provision services whenever a new tenant is built.

The idea is to self provision itself. For example when a new tenant is created, a switch can access Openstack, find out what network name was configured, which vlan id assigned and a what ip subnet configured.

So the first step is to authenticate to the Openstack Server and retrieve a temporary token.

host1#curl -d '{"auth":{"passwordCredentials":{"username": "admin","password": "mypassword"},"tenantName": "admin"}}' -H "Content-Type: application/js
on" http://x.x.x.x:5000/v2.0/tokens | python -mjson.tool

What comes back is a json response that needs to be parsed.  I piped the python module "python -mjson.tool"  to parse the contents.

What I get in return looks similar to this:

{
    "access": {
        "metadata": {
            "is_admin": 0,
            "roles": [
                "3e2b3771096e4a95b39f3832c19679df"
            ]
        },
        "serviceCatalog": [
            {
                "endpoints": [
                    {

             ....
            }
        ],
 "token": {
            "expires": "2014-07-10T19:31:10Z",
            "id": "14213fjgiod2341vcrt46b9674f",
            "issued_at": "2014-07-09T19:31:10.700433",
            "tenant": {
                "description": "admin tenant",
                "enabled": true,
                "id": "3a10de8a82444118865a6398b336ee68",
                "name": "admin"
            }
        },
        "user": {
            "id": "be246bdf3e01493d8d75bf3938e1bffc",
            "name": "admin",
            "roles": [
                {
                    "name": "admin"
                }
            ],
            "roles_links": [],
            "username": "admin"
        }
    }
}

Now you can see that the token "expires" and this will mean that you will need to request tokens every so often for security purposes.

After this you'll need the token id so you can make further requests.

Next assign your token to a variable.

host1# MyToken=14213fjgiod2341vcrt46b9674f

host1# echo $MyToken
14213fjgiod2341vcrt46b9674f

Then you can use the token to request further information:
host1#curl -s  -H "X-Auth-Token: $MyToken"  http://x.x.x.x:9696/v2.0/networks | python -mjson.tool
{
    "networks": [
        {
            "admin_state_up": true,
            "id": "0b6ed891-a9ae-4c5a-a7f9-36e851bf1d48",
            "name": "Network1",
            "provider:network_type": "local",
            "provider:physical_network": null,
            "provider:segmentation_id": null,
            "router:external": false,
            "shared": false,
            "status": "ACTIVE",
            "subnets": [
                "d4bf516f-135c-4c7f-ba7e-363cb6c7d307"
            ],
            "tenant_id": "3a10de8a82444118865a6398b336ee68"
        }
}

With this information I can do a get on the subnet:

/v2.0/subnets/​{subnet_id}

 # curl -s  -H "X-Auth-Token: $MyToken"  http://x.x.x.x:9696/v2.0/subnets/d4bf516f-135c-4c7f-ba7e-363cb6c7d307 | python -mjson.tool
{
    "subnet": {
        "allocation_pools": [
            {
                "end": "20.20.0.20",
                "start": "20.20.0.10"
            }
        ],
        "cidr": "20.20.0.0/24",
        "dns_nameservers": [
            "198.6.1.1"
        ],
        "enable_dhcp": true,
        "gateway_ip": "20.20.0.1",
        "host_routes": [],
        "id": "d4bf516f-135c-4c7f-ba7e-363cb6c7d307",
        "ip_version": 4,
        "name": "Network1",
        "network_id": "0b6ed891-a9ae-4c5a-a7f9-36e851bf1d48",
        "tenant_id": "3a10de8a82444118865a6398b336ee68"
    }
}

So with this information I can basically extract the cidr and  gateway_ip to populate a gateway router.