Showing posts with label mp-bgp. Show all posts
Showing posts with label mp-bgp. Show all posts

Wednesday, September 24, 2014

How Contrail communicates with the underlay

Contrail typically consists of a cluster of nodes. The three main nodes are the config, control and compute. The config node is where the Openstack Horizon and Contrail Controller exists. The Control node is used to form a MP-BGP session to a gateway router. The Compute node hosts all the VMs and virtual networks.

You can think of Contrail as a PE router as pretty much this is what a gateway router perceives the other end of the connection. Contrail uses a vRouter and when you configure virtual networks you have the ability to add a route-target to that virtual network. On the Gateway router you would create VRFs to associate with the corresponding virtual networks and prefixes can be exchanged. Data plane traffic will traverse through an MPLS tunnel between Contrail and the Gateway router. It's at the gateway router where you would "leak" the received Contrail virtual network into the main routing instance of the gateway router.

Here I use an Juniper MX as the gateway router. When I first setup contrail I used the testbed.py script to add the mx gateway router.

It's called ext_router = [ip address]

Then in contrail webui I should see the BGP session. You can however add this post contrail installation.


On the MX, I configure an iBGP session to connect with the Contrail control node.

user@router# show protocols
mpls {
    interface all;
}
bgp {
    group IBGP-CONTRAIL {
        type internal;
        local-address 192.168.10.11;
        family inet-vpn {
            unicast;
        }
        neighbor 192.168.10.2;
    }
}

Then in Contrail config node I create a virtual network and add a route target.




I create a corresponding VRF on the MX with the route target.

user@router# show routing-instances
VRF1 {
    instance-type vrf;
    interface lt-3/0/0.3;
    route-distinguisher 1.1.1.1:101;
    vrf-target target:64512:101;
    routing-options {
        static {
            route 0.0.0.0/0 next-hop 192.168.12.1;
        }
    }


I check to see the BGP session established.

user@router# run show bgp summary                                     
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
bgp.l3vpn.0         
                       8          8          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.2          64512       5724       6292       0       3 1d 23:08:56 Establ
  bgp.l3vpn.0: 8/8/8/0
  VRF1.inet.0: 3/3/3/0

The Virtual Network IP addresses for the VMs will be sent.

user@router# run show route receive-protocol bgp 192.168.10.2 

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
* 11.1.1.1/32             192.168.10.3                 100        ?
* 11.1.1.5/32             192.168.10.3                 100        ?
* 11.1.1.7/32             192.168.10.3                 200        ?

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
  192.168.10.3:7:11.1.1.1/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.5/32                   
*                         192.168.10.3                 100        ?
  192.168.10.3:7:11.1.1.7/32                   
*                         192.168.10.3                 200        ?

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
1                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
2                  *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
13                 *[MPLS/0] 1w1d 04:10:08, metric 1
                      Receive
299904             *[VPN/170] 1d 23:14:00
                    > to 192.168.11.1 via lt-3/0/0.1, Pop     
299936             *[VPN/170] 1d 12:47:38
                      receive table VRF1.inet.0, Pop     
299952             *[VPN/170] 1d 12:47:38
                    > to 192.168.12.1 via lt-3/0/0.3, Pop     

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.10.3:7:11.1.1.1/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
192.168.10.3:7:11.1.1.5/32               
                   *[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
192.168.10.3:7:11.1.1.7/32               
                   *[BGP/170] 1d 12:26:14, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28


Note the dynamic MPLS GRE tunnel is created. You will need to create one on the MX.

user@router# show chassis
fpc 3 {
    pic 0 {
        tunnel-services;
    }
}

user@router# show routing-options
static {
    route 0.0.0.0/0 next-hop 10.161.1.1;
}
autonomous-system 64512;
dynamic-tunnels {
    dynamic_overlay_tunnels {
        source-address 192.168.10.11;
        gre;
        destination-networks {
            192.168.10.0/24;
        }
    }
}



PoC-Demo.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0          *[Static/5] 1d 13:05:36
                    > to 192.168.12.1 via lt-3/0/0.3
11.1.1.1/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 24
11.1.1.5/32        *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 18
11.1.1.7/32        *[BGP/170] 1d 12:44:12, localpref 200, from 192.168.10.2
                      AS path: ?, validation-state: unverified
                    > via gr-3/0/0.32770, Push 28
192.168.12.0/24    *[Direct/0] 1d 13:05:36
                    > via lt-3/0/0.3
192.168.12.2/32    *[Local/0] 1d 13:05:36
                      Local via lt-3/0/0.3

LT interfaces are created to allow the virtual network traffic to communicate between the VRF and the main routing instance. You could also use RIB groups and Policies to do the same thing.

    lt-3/0/0 {
        unit 2 {
            encapsulation ethernet;
            peer-unit 3;
            family inet {
                address 192.168.12.1/24;
            }
        }
        unit 3 {
            encapsulation ethernet;
            peer-unit 2;
            family inet {
                address 192.168.12.2/24;
            }
        }
    }

You then need to make sure the interface that is connecting to the Contrail network is using MPLS.


interfaces {

    ge-3/1/1 {
        unit 0 {
            family inet {
                address 192.168.10.11/24;
            }
            family mpls;
        }
    }
    lo0 {
        unit 0 {
            family inet {
                address 1.1.1.1/32;
            }
            family iso {
                address 49.0002.0010.0100.1001.00;
            }
        }
    }
}

One thing you should be aware of is the next-hop of the route advertised by contrail points to the IP address of the Compute Node and not the control node.
user@router# run show route 11.1.1.1/32 detail

VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
11.1.1.1/32 (1 entry, 1 announced)
        *BGP    Preference: 170/-101
                Route Distinguisher: 192.168.10.3:7  <<< Contrail's RD
                Next hop type: Indirect
                Address: 0x94f4a28
                Next-hop reference count: 3
                Source: 192.168.10.2
                Next hop type: Router, Next hop index: 660
                Next hop: via gr-3/0/0.32770, selected
                Label operation: Push 24
                Label TTL action: prop-ttl
                Session Id: 0xd
                Protocol next hop: 192.168.10.3  <<<< IP of compute node
                Push 24
                Indirect next hop: 0x9574410 1048574 INH Session ID: 0xe
                State: <Secondary Active Int Ext ProtectionCand>
                Local AS: 64512 Peer AS: 64512
                Age: 1d 13:54:24     Metric2: 0
                Validation State: unverified
                Task: BGP_64512.192.168.10.2+34735
                Announcement bits (1): 1-KRT
                AS path: ?
                Communities: target:64512:101   << RT from contrail
                Import Accepted
                VPN Label: 24
                Localpref: 100
                Router ID: 192.168.10.2         <<<< IP of control node
                Primary Routing Table bgp.l3vpn.0

Thursday, July 31, 2014

EVPN for Layer 2 stretch between Data Centers Pt.1

EVPN (Enhanced VPN or Ethernet VPN) is a great technology for stretching Layer 2 between Data Centers (aka Data Center Interconnect or DCI). It uses MP-BGP for control plane exchange of Tenant information and mac-addresses. Data Plane traffic is tunneled inside a tunneling protocol such as MPLS, VXLAN or PBB. EVPN is used in lieu of VPLS because it provides better control over BUM traffic (Broadcast, Unknown Unicast, and Multicast). It also supports the ability to forward traffic over multiple active paths and Multihoming. EVPN used over MPLS provides the benefits of traffic engineering and fast convergence.

In part I, I've created a small single-homed setup to show how this works.


The first step is to create the trunk port facing the Leaf switch. The leaf switch is a standard TOR switch with no special config.

set interfaces et-2/2/1 description TO-LEAF1
set interfaces et-2/2/1 flexible-vlan-tagging
set interfaces et-2/2/1 encapsulation flexible-ethernet-services
set interfaces et-2/2/1 unit 100 encapsulation vlan-bridge
set interfaces et-2/2/1 unit 100 vlan-id 100
 
I created a sub-interface and placed it into a routing instance.

set routing-instances evpn100 instance-type evpn
set routing-instances evpn100 vlan-id 100
set routing-instances evpn100 interface et-2/2/1.100
set routing-instances evpn100 route-distinguisher 4.4.4.4:100
set routing-instances evpn100 vrf-target target:65000:100
set routing-instances evpn100 protocols evpn interface et-2/2/1.100
set routing-instances evpn100 protocols evpn label-allocation per-instance


Instance configuration looks like a normal VPLS configuration except for the instance-type and evpn protocol parameters.

Next I configure BGP to exchange control plane info.

set protocols bgp group IBGP type internal
set protocols bgp group IBGP local-address 4.4.4.4
set protocols bgp group IBGP family inet unicast
set protocols bgp group IBGP family evpn signaling
set protocols bgp group IBGP neighbor 5.5.5.5

A new address family is used called evpn.
After that, the normal MPLS, your flavor of MPLS signaling and IGP protocol configuration is used as well as the Core MPLS facing interfaces.

set protocols mpls interface all
set protocols mpls interface fxp0.0 disable
set protocols mpls interface lo0.0
set protocols isis interface all
set protocols isis interface fxp0.0 disable
set protocols isis interface lo0.0 passive
set protocols ldp interface all
set protocols ldp interface fxp0.0 disable
set protocols ldp interface lo0.0

set interfaces et-2/0/0 description TO-CORE1
set interfaces et-2/0/0 unit 0 family inet address 192.168.24.4/24
set interfaces et-2/0/0 unit 0 family iso
set interfaces et-2/0/0 unit 0 family mpls


Once configured, MP-BGP exchanges "control plane" information.

# run show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0              
                       0          0          0          0          0          0
bgp.evpn.0          
                       2          2          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
5.5.5.5               65000        137        136       0       0       57:42 Establ
  inet.0: 0/0/0/0
  bgp.evpn.0: 2/2/2/0
  evpn100.evpn.0: 2/2/2/0

  __default_evpn__.evpn.0: 0/0/0/0


# run show route receive-protocol bgp 5.5.5.5

inet.0: 24 destinations, 24 routes (24 active, 0 holddown, 0 hidden)

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)

iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)

mpls.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)

bgp.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
  2:5.5.5.5:100::100::00:00:05:ed:ae:01/304                  
*                         5.5.5.5                      100        I
  3:5.5.5.5:100::100::5.5.5.5/304                  
*                         5.5.5.5                      100        I

evpn100.evpn.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
  Prefix          Nexthop           MED     Lclpref    AS path
  2:5.5.5.5:100::100::00:00:05:ed:ae:01/304                  
*                         5.5.5.5                      100        I
  3:5.5.5.5:100::100::5.5.5.5/304                  

You can also check the status of the EVPN and it's mac table


# run show evpn mac-table

MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
           SE - statistics enabled, NM - non configured MAC, R - remote PE MAC)


Ethernet switching table : 2 entries, 2 learned
Routing instance : evpn100
    Vlan                MAC                 MAC         Age    Logical                NH        RTR
    name                address             flags              interface              Index     ID
    __evpn100__         00:00:05:ed:ad:49   D             -   et-2/2/1.100        
    __evpn100__         00:00:05:ed:ae:01   DC            -   pip-13.010010000000    1048577   1048577


This shows you locally learned macs and macs learned over the WAN.

# run show evpn statistics   
Instance: evpn100
   Local interface: et-2/2/1.100, Index: 338
     Broadcast packets:                     1
     Broadcast bytes  :                    60
     Multicast packets:                     0
     Multicast bytes  :                     0
     Flooded packets  :                  4240
     Flooded bytes    :               6341604
     Unicast packets  :               3292539
     Unicast bytes    :            3528822524
     Current MAC count:                     1 (Limit 0)

In Part II I'll go more into configuring Gateway information to prevent the trombone effect.


Thursday, May 22, 2014

Test driving Open Contrail

So I went to the hands on Open Contrail MeetUp. It started off as a semi Q&A overview of the architecture and business use case.

There is a need from Service providers to create services for customers that could

a) be deployed in a timely matter (seconds rather than months

b) Lower Op Ex through automation

c) reduce Operational complexity with template configs

Contrail is a module that can be currently added to an Openstack or Cloudstack platform. I'll talk about the Openstack implementation as I know more about this Cloud platform. The Contrail Controller works entirely in the overlay. It does not know anything about the underlay except for the gateway so it expects the physical network to already be in place. This means it can interoperate with any existing switch vendor network. When you install Contrail as a Neutron plugin into Openstack, it will create a vRouter which is bound to the hypervisor. Currently KVM is the one it works with but I hear that it was tested on VMware as a vm but not directly onto ESXi.  The vRouter is important because you wouldn't use Open vSwitch (OVS). The main function of Contrail is creating the overlay network within Openstack.

Here's how this works. Let's look at this simple logical topology



First you would go to the Contrail Controller (Web GUI) and create the two networks.


Then you would go to the Openstack Horizon Web UI and spin up your VM instances. Under the networking tab you should be able to associate the network to the VM.



Next you would need to create a policy so that each network can talk to each other. Think of it as creating a firewall ACL.



Last you need to attach the policy to each network.

Contrail will automatically assign the VM an ip address and point the default route of the VM to the vRouter.


This will allow you to access the Back End Server.

Overall provisioning time would be roughly 2-3 minutes depending on how fast the VMs can spin up.

Now if you want to provide access to anything outside of the Data Center you'll need to go through a gateway router. AKA Data Center Interconnect (DCI)

To understand this more you will need a Gateway router that speaks MP-BGP can can support GRE or VXLAN. A GRE tunnel will be created from the Contrail vRouter (Virtual) to the physical Gateway Router. If you want to create multi-tenency, you will have to put each tunnel into a separate VRF on the Gateway Router. This provisioning is a manual process of the Gateway Router. Contrail does not manage the Gateway Router at all, however you may be able to automate this function using scripting.
The vRouter will exchange routes via MP-BGP and there is a setting in Contrail to create the Router-Target. The BGP  family types vRouter supports is inet-vpn (or vpnv4) and evpn.

Overall the provisioning is fairly simple. There are a few things I would like "improved" in the product, such as being able to attach a policy from the networking overview "tab" instead of having to drill down into each network and doing it there. There could be a more excel feel to this view as you should be able to make modifications from this view using pulldowns or directly adding the ip subnets.
Also the manual process of creating VRFs on the Gateway router is a bit tedious. I'm investigating whether Openstack has the ability run a script that can make a netconf rpc call to the gateway router.

An alternative is to use slax and curl on a Juniper MX router to extract the details from the Contrail controller.