Wednesday, September 24, 2014

How Contrail communicates with the underlay

Contrail typically consists of a cluster of nodes. The three main nodes are the config, control and compute. The config node is where the Openstack Horizon and Contrail Controller exists. The Control node is used to form a MP-BGP session to a gateway router. The Compute node hosts all the VMs and virtual networks.

You can think of Contrail as a PE router as pretty much this is what a gateway router perceives the other end of the connection. Contrail uses a vRouter and when you configure virtual networks you have the ability to add a route-target to that virtual network. On the Gateway router you would create VRFs to associate with the corresponding virtual networks and prefixes can be exchanged. Data plane traffic will traverse through an MPLS tunnel between Contrail and the Gateway router. It's at the gateway router where you would "leak" the received Contrail virtual network into the main routing instance of the gateway router.

Here I use an Juniper MX as the gateway router. When I first setup contrail I used the testbed.py script to add the mx gateway router.

It's called ext_router = [ip address]

Then in contrail webui I should see the BGP session. You can however add this post contrail installation.


On the MX, I configure an iBGP session to connect with the Contrail control node.

user@router# show protocols
mpls {
    interface all;
}
bgp {
    group IBGP-CONTRAIL {
        type internal;
        local-address 192.168.10.11;
        family inet-vpn {
            unicast;
        }
        neighbor 192.168.10.2;
    }
}

Then in Contrail config node I create a virtual network and add a route target.




I create a corresponding VRF on the MX with the route target.

user@router# show routing-instances
VRF1 {
    instance-type vrf;
    interface lt-3/0/0.3;
    route-distinguisher 1.1.1.1:101;
    vrf-target target:64512:101;
    routing-options {
        static {
            route 0.0.0.0/0 next-hop 192.168.12.1;
        }
    }


I check to see the BGP session established.

user@router# run show bgp summary                                     
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0         
                       8          8          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.2          64512       5724       6292       0       3 1d 23:08:56 Establ
  bgp.l3vpn.0: 8/8/8/0
  VRF1.inet.0: 3/3/3/0

The Virtual Network IP addresses for the VMs will be sent.

user@router# run show route receive-protocol bgp 192.168.10.2 

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
* 11.1.1.1/32             192.168.10.3                 100        ?
* 11.1.1.5/32             192.168.10.3                 100        ?
* 11.1.1.7/32             192.168.10.3                 200        ?

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
  192.168.10.3:7:11.1.1.1/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.5/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.7/32                   
*                         192.168.10.3                 200        ?

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
1                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
2                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
13                 *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
299904             *[VPN/170] 1d 23:14:00
                    > to 192.168.11.1 via lt-3/0/0.1, Pop     
299936             *[VPN/170] 1d 12:47:38
                      receive table VRF1.inet.0, Pop     
299952             *[VPN/170] 1d 12:47:38
                    > to 192.168.12.1 via lt-3/0/0.3, Pop     

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.10.3:7:11.1.1.1/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
192.168.10.3:7:11.1.1.5/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
192.168.10.3:7:11.1.1.7/32               
                   *[BGP/170] 1d 12:26:14, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28


Note the dynamic MPLS GRE tunnel is created. You will need to create one on the MX.

user@router# show chassis
fpc 3 {
    pic 0 {
        tunnel-services;
    }
}

user@router# show routing-options
static {
    route 0.0.0.0/0 next-hop 10.161.1.1;
}
autonomous-system 64512;
dynamic-tunnels {
    dynamic_overlay_tunnels {
        source-address 192.168.10.11;
        gre;
        destination-networks {
            192.168.10.0/24;
        }
    }
}



PoC-Demo.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 1d 13:05:36
                    > to 192.168.12.1 via lt-3/0/0.3
11.1.1.1/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
11.1.1.5/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
11.1.1.7/32        *[BGP/170] 1d 12:44:12, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28
192.168.12.0/24    *[Direct/0] 1d 13:05:36
                    > via lt-3/0/0.3
192.168.12.2/32    *[Local/0] 1d 13:05:36
                      Local via lt-3/0/0.3

LT interfaces are created to allow the virtual network traffic to communicate between the VRF and the main routing instance. You could also use RIB groups and Policies to do the same thing.

    lt-3/0/0 {
        unit 2 {
            encapsulation ethernet;
            peer-unit 3;
            family inet {
                address 192.168.12.1/24;
            }
        }
        unit 3 {
            encapsulation ethernet;
            peer-unit 2;
            family inet {
                address 192.168.12.2/24;
            }
        }
    }

You then need to make sure the interface that is connecting to the Contrail network is using MPLS.


interfaces {

    ge-3/1/1 {
        unit 0 {
            family inet {
                address 192.168.10.11/24;
            }
            family mpls;
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 1.1.1.1/32;
            }
            family iso {
                address 49.0002.0010.0100.1001.00;
            }
        }
    }
}

One thing you should be aware of is the next-hop of the route advertised by contrail points to the IP address of the Compute Node and not the control node.
user@router# run show route 11.1.1.1/32 detail

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
11.1.1.1/32 (1 entry, 1 announced)
        *BGP    Preference: 170/-101
                Route Distinguisher: 192.168.10.3:7  <<< Contrail's RD
                Next hop type: Indirect
                Address: 0x94f4a28
                Next-hop reference count: 3
                Source: 192.168.10.2
                Next hop type: Router, Next hop index: 660
                Next hop: via gr-3/0/0.32770, selected
                Label operation: Push 24
                Label TTL action: prop-ttl
                Session Id: 0xd
                Protocol next hop: 192.168.10.3  <<<< IP of compute node
                Push 24
                Indirect next hop: 0x9574410 1048574 INH Session ID: 0xe
                State: <Secondary Active Int Ext ProtectionCand>
                Local AS: 64512 Peer AS: 64512
                Age: 1d 13:54:24     Metric2: 0
                Validation State: unverified
                Task: BGP_64512.192.168.10.2+34735
                Announcement bits (1): 1-KRT
                AS path: ?
                Communities: target:64512:101   << RT from contrail
                Import Accepted
                VPN Label: 24
                Localpref: 100
                Router ID: 192.168.10.2         <<<< IP of control node
                Primary Routing Table bgp.l3vpn.0

No comments:

Post a Comment