Contrail typically consists of a cluster of nodes. The three main nodes are the config, control and compute. The config node is where the Openstack Horizon and Contrail Controller exists. The Control node is used to form a MP-BGP session to a gateway router. The Compute node hosts all the VMs and virtual networks.
You can think of Contrail as a PE router as pretty much this is what a gateway router perceives the other end of the connection. Contrail uses a vRouter and when you configure virtual networks you have the ability to add a route-target to that virtual network. On the Gateway router you would create VRFs to associate with the corresponding virtual networks and prefixes can be exchanged. Data plane traffic will traverse through an MPLS tunnel between Contrail and the Gateway router. It's at the gateway router where you would "leak" the received Contrail virtual network into the main routing instance of the gateway router.
Here I use an Juniper MX as the gateway router. When I first setup contrail I used the testbed.py script to add the mx gateway router.
It's called ext_router = [ip address]
Then in contrail webui I should see the BGP session. You can however add this post contrail installation.
On the MX, I configure an iBGP session to connect with the Contrail control node.
user@router# show protocols
mpls {
interface all;
}
bgp {
group IBGP-CONTRAIL {
type internal;
local-address 192.168.10.11;
family inet-vpn {
unicast;
}
neighbor 192.168.10.2;
}
}
Then in Contrail config node I create a virtual network and add a route target.
I create a corresponding VRF on the MX with the route target.
user@router# show routing-instances
VRF1 {
instance-type vrf;
interface lt-3/0/0.3;
route-distinguisher 1.1.1.1:101;
vrf-target target:64512:101;
routing-options {
static {
route 0.0.0.0/0 next-hop 192.168.12.1;
}
}
}
I check to see the BGP session established.
user@router# run show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
bgp.l3vpn.0
8 8 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.2 64512 5724 6292 0 3 1d 23:08:56 Establ
bgp.l3vpn.0: 8/8/8/0
VRF1.inet.0: 3/3/3/0
The Virtual Network IP addresses for the VMs will be sent.
user@router# run show route receive-protocol bgp 192.168.10.2
VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
* 11.1.1.1/32 192.168.10.3 100 ?
* 11.1.1.5/32 192.168.10.3 100 ?
* 11.1.1.7/32 192.168.10.3 200 ?
bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
192.168.10.3:7:11.1.1.1/32
* 192.168.10.3 100 ?
192.168.10.3:7:11.1.1.5/32
* 192.168.10.3 100 ?
192.168.10.3:7:11.1.1.7/32
* 192.168.10.3 200 ?
mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0 *[MPLS/0] 1w1d 04:10:08, metric 1
Receive
1 *[MPLS/0] 1w1d 04:10:08, metric 1
Receive
2 *[MPLS/0] 1w1d 04:10:08, metric 1
Receive
13 *[MPLS/0] 1w1d 04:10:08, metric 1
Receive
299904 *[VPN/170] 1d 23:14:00
> to 192.168.11.1 via lt-3/0/0.1, Pop
299936 *[VPN/170] 1d 12:47:38
receive table VRF1.inet.0, Pop
299952 *[VPN/170] 1d 12:47:38
> to 192.168.12.1 via lt-3/0/0.3, Pop
bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.168.10.3:7:11.1.1.1/32
*[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 24
192.168.10.3:7:11.1.1.5/32
*[BGP/170] 1d 12:46:01, localpref 100, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 18
192.168.10.3:7:11.1.1.7/32
*[BGP/170] 1d 12:26:14, localpref 200, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 28
Note the dynamic MPLS GRE tunnel is created. You will need to create one on the MX.
user@router# show chassis
fpc 3 {
pic 0 {
tunnel-services;
}
}
user@router# show routing-options
static {
route 0.0.0.0/0 next-hop 10.161.1.1;
}
autonomous-system 64512;
dynamic-tunnels {
dynamic_overlay_tunnels {
source-address 192.168.10.11;
gre;
destination-networks {
192.168.10.0/24;
}
}
}
PoC-Demo.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 1d 13:05:36
> to 192.168.12.1 via lt-3/0/0.3
11.1.1.1/32 *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 24
11.1.1.5/32 *[BGP/170] 1d 13:03:59, localpref 100, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 18
11.1.1.7/32 *[BGP/170] 1d 12:44:12, localpref 200, from 192.168.10.2
AS path: ?, validation-state: unverified
> via gr-3/0/0.32770, Push 28
192.168.12.0/24 *[Direct/0] 1d 13:05:36
> via lt-3/0/0.3
192.168.12.2/32 *[Local/0] 1d 13:05:36
Local via lt-3/0/0.3
LT interfaces are created to allow the virtual network traffic to communicate between the VRF and the main routing instance. You could also use RIB groups and Policies to do the same thing.
lt-3/0/0 {
unit 2 {
encapsulation ethernet;
peer-unit 3;
family inet {
address 192.168.12.1/24;
}
}
unit 3 {
encapsulation ethernet;
peer-unit 2;
family inet {
address 192.168.12.2/24;
}
}
}
You then need to make sure the interface that is connecting to the Contrail network is using MPLS.
interfaces {
ge-3/1/1 {
unit 0 {
family inet {
address 192.168.10.11/24;
}
family mpls;
}
}
lo0 {
unit 0 {
family inet {
address 1.1.1.1/32;
}
family iso {
address 49.0002.0010.0100.1001.00;
}
}
}
}
One thing you should be aware of is the next-hop of the route advertised by contrail points to the IP address of the Compute Node and not the control node.
user@router# run show route 11.1.1.1/32 detail
VRF1.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
11.1.1.1/32 (1 entry, 1 announced)
*BGP Preference: 170/-101
Route Distinguisher: 192.168.10.3:7 <<< Contrail's RD
Next hop type: Indirect
Address: 0x94f4a28
Next-hop reference count: 3
Source: 192.168.10.2
Next hop type: Router, Next hop index: 660
Next hop: via gr-3/0/0.32770, selected
Label operation: Push 24
Label TTL action: prop-ttl
Session Id: 0xd
Protocol next hop: 192.168.10.3 <<<< IP of compute node
Push 24
Indirect next hop: 0x9574410 1048574 INH Session ID: 0xe
State: <Secondary Active Int Ext ProtectionCand>
Local AS: 64512 Peer AS: 64512
Age: 1d 13:54:24 Metric2: 0
Validation State: unverified
Task: BGP_64512.192.168.10.2+34735
Announcement bits (1): 1-KRT
AS path: ?
Communities: target:64512:101 << RT from contrail
Import Accepted
VPN Label: 24
Localpref: 100
Router ID: 192.168.10.2 <<<< IP of control node
Primary Routing Table bgp.l3vpn.0
Showing posts with label mp-bgp. Show all posts
Showing posts with label mp-bgp. Show all posts
Wednesday, September 24, 2014
Thursday, July 31, 2014
EVPN for Layer 2 stretch between Data Centers Pt.1
EVPN (Enhanced VPN or Ethernet VPN) is a great technology for stretching Layer 2 between Data Centers (aka Data Center Interconnect or DCI). It uses MP-BGP for control plane exchange of Tenant information and mac-addresses. Data Plane traffic is tunneled inside a tunneling protocol such as MPLS, VXLAN or PBB. EVPN is used in lieu of VPLS because it provides better control over BUM traffic (Broadcast, Unknown Unicast, and Multicast). It also supports the ability to forward traffic over multiple active paths and Multihoming. EVPN used over MPLS provides the benefits of traffic engineering and fast convergence.
In part I, I've created a small single-homed setup to show how this works.
set interfaces et-2/2/1 encapsulation flexible-ethernet-services
set interfaces et-2/2/1 unit 100 encapsulation vlan-bridge
set interfaces et-2/2/1 unit 100 vlan-id 100
set routing-instances evpn100 interface et-2/2/1.100
set routing-instances evpn100 route-distinguisher 4.4.4.4:100
set routing-instances evpn100 vrf-target target:65000:100
set routing-instances evpn100 protocols evpn interface et-2/2/1.100
set routing-instances evpn100 protocols evpn label-allocation per-instance
set protocols bgp group IBGP family inet unicast
set protocols bgp group IBGP family evpn signaling
set protocols bgp group IBGP neighbor 5.5.5.5
A new address family is used called evpn.
set protocols mpls interface lo0.0
set protocols isis interface all
set protocols isis interface fxp0.0 disable
set protocols isis interface lo0.0 passive
set protocols ldp interface all
set protocols ldp interface fxp0.0 disable
set protocols ldp interface lo0.0
set interfaces et-2/0/0 description TO-CORE1
set interfaces et-2/0/0 unit 0 family inet address 192.168.24.4/24
set interfaces et-2/0/0 unit 0 family iso
set interfaces et-2/0/0 unit 0 family mpls
Once configured, MP-BGP exchanges "control plane" information.
# run show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
bgp.evpn.0
2 2 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
5.5.5.5 65000 137 136 0 0 57:42 Establ
inet.0: 0/0/0/0
bgp.evpn.0: 2/2/2/0
evpn100.evpn.0: 2/2/2/0
__default_evpn__.evpn.0: 0/0/0/0
# run show route receive-protocol bgp 5.5.5.5
inet.0: 24 destinations, 24 routes (24 active, 0 holddown, 0 hidden)
inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
mpls.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
bgp.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
2:5.5.5.5:100::100::00:00:05:ed:ae:01/304
* 5.5.5.5 100 I
3:5.5.5.5:100::100::5.5.5.5/304
* 5.5.5.5 100 I
evpn100.evpn.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
2:5.5.5.5:100::100::00:00:05:ed:ae:01/304
* 5.5.5.5 100 I
3:5.5.5.5:100::100::5.5.5.5/304
You can also check the status of the EVPN and it's mac table
# run show evpn mac-table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : evpn100
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
__evpn100__ 00:00:05:ed:ad:49 D - et-2/2/1.100
__evpn100__ 00:00:05:ed:ae:01 DC - pip-13.010010000000 1048577 1048577
This shows you locally learned macs and macs learned over the WAN.
# run show evpn statistics
Instance: evpn100
Local interface: et-2/2/1.100, Index: 338
Broadcast packets: 1
Broadcast bytes : 60
Multicast packets: 0
Multicast bytes : 0
Flooded packets : 4240
Flooded bytes : 6341604
Unicast packets : 3292539
Unicast bytes : 3528822524
Current MAC count: 1 (Limit 0)
In Part II I'll go more into configuring Gateway information to prevent the trombone effect.
In part I, I've created a small single-homed setup to show how this works.
The first step is to create the trunk port facing the Leaf switch. The leaf switch is a standard TOR switch with no special config.
set interfaces et-2/2/1 description TO-LEAF1
set interfaces et-2/2/1 flexible-vlan-taggingset interfaces et-2/2/1 encapsulation flexible-ethernet-services
set interfaces et-2/2/1 unit 100 encapsulation vlan-bridge
set interfaces et-2/2/1 unit 100 vlan-id 100
I created a sub-interface and placed it into a routing instance.
set routing-instances evpn100 instance-type evpn
set routing-instances evpn100 vlan-id 100set routing-instances evpn100 interface et-2/2/1.100
set routing-instances evpn100 route-distinguisher 4.4.4.4:100
set routing-instances evpn100 vrf-target target:65000:100
set routing-instances evpn100 protocols evpn interface et-2/2/1.100
set routing-instances evpn100 protocols evpn label-allocation per-instance
Instance configuration looks like a normal VPLS configuration except for the instance-type and evpn protocol parameters.
Next I configure BGP to exchange control plane info.
set protocols bgp group IBGP type internal
set protocols bgp group IBGP local-address 4.4.4.4set protocols bgp group IBGP family inet unicast
set protocols bgp group IBGP family evpn signaling
set protocols bgp group IBGP neighbor 5.5.5.5
A new address family is used called evpn.
After that, the normal MPLS, your flavor of MPLS signaling and IGP protocol configuration is used as well as the Core MPLS facing interfaces.
set protocols mpls interface all
set protocols mpls interface fxp0.0 disableset protocols mpls interface lo0.0
set protocols isis interface all
set protocols isis interface fxp0.0 disable
set protocols isis interface lo0.0 passive
set protocols ldp interface all
set protocols ldp interface fxp0.0 disable
set protocols ldp interface lo0.0
set interfaces et-2/0/0 unit 0 family inet address 192.168.24.4/24
set interfaces et-2/0/0 unit 0 family iso
set interfaces et-2/0/0 unit 0 family mpls
Once configured, MP-BGP exchanges "control plane" information.
# run show bgp summary
Groups: 1 Peers: 1 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
bgp.evpn.0
2 2 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
5.5.5.5 65000 137 136 0 0 57:42 Establ
inet.0: 0/0/0/0
bgp.evpn.0: 2/2/2/0
evpn100.evpn.0: 2/2/2/0
__default_evpn__.evpn.0: 0/0/0/0
# run show route receive-protocol bgp 5.5.5.5
inet.0: 24 destinations, 24 routes (24 active, 0 holddown, 0 hidden)
inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
iso.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
mpls.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
bgp.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
2:5.5.5.5:100::100::00:00:05:ed:ae:01/304
* 5.5.5.5 100 I
3:5.5.5.5:100::100::5.5.5.5/304
* 5.5.5.5 100 I
evpn100.evpn.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
2:5.5.5.5:100::100::00:00:05:ed:ae:01/304
* 5.5.5.5 100 I
3:5.5.5.5:100::100::5.5.5.5/304
You can also check the status of the EVPN and it's mac table
# run show evpn mac-table
MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static, C - Control MAC
SE - statistics enabled, NM - non configured MAC, R - remote PE MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : evpn100
Vlan MAC MAC Age Logical NH RTR
name address flags interface Index ID
__evpn100__ 00:00:05:ed:ad:49 D - et-2/2/1.100
__evpn100__ 00:00:05:ed:ae:01 DC - pip-13.010010000000 1048577 1048577
This shows you locally learned macs and macs learned over the WAN.
# run show evpn statistics
Instance: evpn100
Local interface: et-2/2/1.100, Index: 338
Broadcast packets: 1
Broadcast bytes : 60
Multicast packets: 0
Multicast bytes : 0
Flooded packets : 4240
Flooded bytes : 6341604
Unicast packets : 3292539
Unicast bytes : 3528822524
Current MAC count: 1 (Limit 0)
In Part II I'll go more into configuring Gateway information to prevent the trombone effect.
Thursday, May 22, 2014
Test driving Open Contrail
So I went to the hands on Open Contrail MeetUp. It started off as a semi Q&A overview of the architecture and business use case.
There is a need from Service providers to create services for customers that could
a) be deployed in a timely matter (seconds rather than months
b) Lower Op Ex through automation
c) reduce Operational complexity with template configs
Contrail is a module that can be currently added to an Openstack or Cloudstack platform. I'll talk about the Openstack implementation as I know more about this Cloud platform. The Contrail Controller works entirely in the overlay. It does not know anything about the underlay except for the gateway so it expects the physical network to already be in place. This means it can interoperate with any existing switch vendor network. When you install Contrail as a Neutron plugin into Openstack, it will create a vRouter which is bound to the hypervisor. Currently KVM is the one it works with but I hear that it was tested on VMware as a vm but not directly onto ESXi. The vRouter is important because you wouldn't use Open vSwitch (OVS). The main function of Contrail is creating the overlay network within Openstack.
Here's how this works. Let's look at this simple logical topology
First you would go to the Contrail Controller (Web GUI) and create the two networks.
Then you would go to the Openstack Horizon Web UI and spin up your VM instances. Under the networking tab you should be able to associate the network to the VM.
Next you would need to create a policy so that each network can talk to each other. Think of it as creating a firewall ACL.
Last you need to attach the policy to each network.
Contrail will automatically assign the VM an ip address and point the default route of the VM to the vRouter.
This will allow you to access the Back End Server.
Overall provisioning time would be roughly 2-3 minutes depending on how fast the VMs can spin up.
Now if you want to provide access to anything outside of the Data Center you'll need to go through a gateway router. AKA Data Center Interconnect (DCI)
To understand this more you will need a Gateway router that speaks MP-BGP can can support GRE or VXLAN. A GRE tunnel will be created from the Contrail vRouter (Virtual) to the physical Gateway Router. If you want to create multi-tenency, you will have to put each tunnel into a separate VRF on the Gateway Router. This provisioning is a manual process of the Gateway Router. Contrail does not manage the Gateway Router at all, however you may be able to automate this function using scripting.
The vRouter will exchange routes via MP-BGP and there is a setting in Contrail to create the Router-Target. The BGP family types vRouter supports is inet-vpn (or vpnv4) and evpn.
Overall the provisioning is fairly simple. There are a few things I would like "improved" in the product, such as being able to attach a policy from the networking overview "tab" instead of having to drill down into each network and doing it there. There could be a more excel feel to this view as you should be able to make modifications from this view using pulldowns or directly adding the ip subnets.
Also the manual process of creating VRFs on the Gateway router is a bit tedious. I'm investigating whether Openstack has the ability run a script that can make a netconf rpc call to the gateway router.
An alternative is to use slax and curl on a Juniper MX router to extract the details from the Contrail controller.
There is a need from Service providers to create services for customers that could
a) be deployed in a timely matter (seconds rather than months
b) Lower Op Ex through automation
c) reduce Operational complexity with template configs
Contrail is a module that can be currently added to an Openstack or Cloudstack platform. I'll talk about the Openstack implementation as I know more about this Cloud platform. The Contrail Controller works entirely in the overlay. It does not know anything about the underlay except for the gateway so it expects the physical network to already be in place. This means it can interoperate with any existing switch vendor network. When you install Contrail as a Neutron plugin into Openstack, it will create a vRouter which is bound to the hypervisor. Currently KVM is the one it works with but I hear that it was tested on VMware as a vm but not directly onto ESXi. The vRouter is important because you wouldn't use Open vSwitch (OVS). The main function of Contrail is creating the overlay network within Openstack.
Here's how this works. Let's look at this simple logical topology
First you would go to the Contrail Controller (Web GUI) and create the two networks.
Then you would go to the Openstack Horizon Web UI and spin up your VM instances. Under the networking tab you should be able to associate the network to the VM.
Next you would need to create a policy so that each network can talk to each other. Think of it as creating a firewall ACL.
Last you need to attach the policy to each network.
Contrail will automatically assign the VM an ip address and point the default route of the VM to the vRouter.
This will allow you to access the Back End Server.
Overall provisioning time would be roughly 2-3 minutes depending on how fast the VMs can spin up.
Now if you want to provide access to anything outside of the Data Center you'll need to go through a gateway router. AKA Data Center Interconnect (DCI)
To understand this more you will need a Gateway router that speaks MP-BGP can can support GRE or VXLAN. A GRE tunnel will be created from the Contrail vRouter (Virtual) to the physical Gateway Router. If you want to create multi-tenency, you will have to put each tunnel into a separate VRF on the Gateway Router. This provisioning is a manual process of the Gateway Router. Contrail does not manage the Gateway Router at all, however you may be able to automate this function using scripting.
The vRouter will exchange routes via MP-BGP and there is a setting in Contrail to create the Router-Target. The BGP family types vRouter supports is inet-vpn (or vpnv4) and evpn.
Overall the provisioning is fairly simple. There are a few things I would like "improved" in the product, such as being able to attach a policy from the networking overview "tab" instead of having to drill down into each network and doing it there. There could be a more excel feel to this view as you should be able to make modifications from this view using pulldowns or directly adding the ip subnets.
Also the manual process of creating VRFs on the Gateway router is a bit tedious. I'm investigating whether Openstack has the ability run a script that can make a netconf rpc call to the gateway router.
An alternative is to use slax and curl on a Juniper MX router to extract the details from the Contrail controller.
Subscribe to:
Posts (Atom)








