How to load balance switched traffic over a spanning tree network using Juniper switches

This is a pretty simple setup. With normal spanning tree one of the redundant links will be blocked. So in order to make use of the blocked port, you can load balance traffic using MSTP (Multiple Spanning Tree).  First the basic setup.

user@SW-A# show interfaces ge-0/0/42 

description to-SW-B-0/0/42;

unit 0 {

    family ethernet-switching {

        port-mode trunk;

        vlan {

            members all;

        }

    }

}

user@SW-A# show interfaces ge-0/0/47    

description to-SW-B-ge-0/0/47;

unit 0 {

    family ethernet-switching {

        port-mode trunk;

        vlan {

            members all;

        }

    }

}

user@SW-A# show vlans 

v100 {

    vlan-id 100;

}

v200 {

    vlan-id 200;

}

user@SW-A# show protocols mstp 

msti 1 {

    vlan 100-199;

}

msti 2 {

    vlan 200-299;

}

user@SW-A# run show spanning-tree interface 

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32768.50c58dab9081     20000  FWD    DESG 

ge-0/0/47.0    128:560      128:560  32768.50c58dab9081     20000  FWD    DESG 

Spanning tree interface parameters for instance 1

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32769.50c58dab9081     20000  FWD    DESG 

ge-0/0/47.0    128:560      128:560  32769.50c58dab9081     20000  FWD    DESG 

Spanning tree interface parameters for instance 2

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32770.50c58dab9081     20000  FWD    DESG 

ge-0/0/47.0    128:560      128:560  32770.50c58dab9081     20000  FWD    DESG 

user@SW-B# show interfaces ge-0/0/42                              

description to-SW-A-0/0/42;

unit 0 {

    family ethernet-switching {

        port-mode trunk;

        vlan {

            members all;

        }

    }

}

user@SW-B# show interfaces ge-0/0/47    

description to-SW-A-ge-0/0/47;

unit 0 {

    family ethernet-switching {

        port-mode trunk;

        vlan {

            members all;

        }

    }

}

user@SW-B# show protocols mstp 

msti 1 {

    vlan 100-199;

}

msti 2 {

    vlan 200-299;

}

user@SW-B# run show spanning-tree interface 

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32768.50c58dab9081     20000  FWD    ROOT 

ge-0/0/47.0    128:560      128:560  32768.50c58dab9081     20000  BLK    ALT  

Spanning tree interface parameters for instance 1

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32769.50c58dab9081     20000  FWD    ROOT   <<<<< NOTE

ge-0/0/47.0    128:560      128:560  32769.50c58dab9081     20000  BLK    ALT  

Spanning tree interface parameters for instance 2

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32770.50c58dab9081     20000  FWD    ROOT 

ge-0/0/47.0    128:560      128:560  32770.50c58dab9081     20000  BLK    ALT  

SW-A is root bridge because of lowest mac address.

As you can see on SW-B, port ge-0/0/47 is blocked for all tagged and untagged traffic. 

There are two ways to influence spanning tree to change the forwarding state.

First, to affect the forwarding state of a particular spanning tree instance, you can use priorities on the root switch. This is useful to load balance traffic for under utilized links.

user@SW-A# set protocols mstp msti 1 interface ge-0/0/42 priority 240

On the remote switch SW-B

user@SW-B# run show spanning-tree interface    

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32768.50c58dab9081     20000  FWD    ROOT 

ge-0/0/47.0    128:560      128:560  32768.50c58dab9081     20000  BLK    ALT  

Spanning tree interface parameters for instance 1

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      240:555  32769.50c58dab9081     20000  BLK    ALT      <<<<<<<< Here we block traffic 

ge-0/0/47.0    128:560      128:560  32769.50c58dab9081     20000  FWD    ROOT  <<<< now traffic will be forwarded on this interface for vlans in the range 100-199

Spanning tree interface parameters for instance 2

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32770.50c58dab9081     20000  FWD    ROOT 

ge-0/0/47.0    128:560      128:560  32770.50c58dab9081     20000  BLK    ALT  

The second method is to affect the traffic by adjusting the cost locally

user@SW-B# set protocols mstp msti 2 interface ge-0/0/47 cost 10000 

user@SW-B# commit 

configuration check succeeds

commit complete

user@SW-B# run show spanning-tree interface    

Spanning tree interface parameters for instance 0

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32768.50c58dab9081     20000  FWD    ROOT 

ge-0/0/47.0    128:560      128:560  32768.50c58dab9081     20000  BLK    ALT  

Spanning tree interface parameters for instance 1

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      240:555  32769.50c58dab9081     20000  BLK    ALT  

ge-0/0/47.0    128:560      128:560  32769.50c58dab9081     20000  FWD    ROOT 

Spanning tree interface parameters for instance 2

Interface    Port ID    Designated      Designated         Port    State  Role

                         port ID        bridge ID          Cost

ge-0/0/42.0    128:555      128:555  32770.50c58dab9081     20000  BLK    ALT  

ge-0/0/47.0    128:560      128:560  32770.50c58dab9081     10000  FWD    ROOT  <<<<<<<< HERE

Now only untagged traffic will forward over the ge-0/0/42 interface.

Use (EVPN) Ethernet Virtual Private Network for Data Center Interconnections (DCI)

As Enterprises build Data Centers at different locations for disaster recovery and traffic distribution, there is a need to interconnect them transparently. Stretching Layer 2 across a WAN poses some challenges.

1) Workload Mobility aka VM migration from one DC to another.

2) Fast convergence in a multi homed environment.

3) Load balancing across multiple active paths between data centers.

The Trombone effect when migrating VMs across a WAN.

When VM1 is moved from one Hypervisor in DC1 to the other Hypervisor in DC2, the default GW for VM1 still resides on DC1. When VM1 sends traffic to VM2, the traffic will traverse the core before tromboning back to DC2.

EVPN solves this.  EVPN is a similar technology to VPLS except that mac addresses are learned and exchanged through the control plane using BGP as the transport protocol.  A new BGP family is introduced called EVPN.

bgp {
    group IBGP {
        local-address 1.1.1.1;        
        family evpn {
            signaling;  
        }
        neighbor 2.2.2.2;
    }
}

First an understanding of how EVPN works.

In a multi-tenant environment, each tenant will correspond to an EVPN instance (EVI). Route Distinguishers are used to distinguish between each EVI and Route Targets are used to share learned mac addresses between EVIs.

For mac learning, each PE router snoops for DHCP and/or ARP(IPv4)/ND(IPv6) packets for a particular EVI. The PE can then advertise the locally learned MAC address to remote PE nodes through MP-iBGP. MAC addresses are aggregated and a MAC prefix is advertised rather than advertising every single MAC address, thus allowing the ability to scale thousands of MAC addresses.  When a remote PE receives this bgp update it will extract the mac address and build a table with the next-hop pointing to the LSP of the advertising PE. Because this is BGP, policies can be created to filter and manipulate forwarding decisions.

When a local PE router sees an ARP request for an IP address and if the PE router has the MAC address binding for that IP address across the wan, the PE router performs a proxy ARP and responds to the ARP Request and can make the forwarding decision locally.  This reduces (BUM) flooding (Broadcast, Unknown Unicast and Multicast) across WAN links.

Gateway IP and MAC addresses syncing in EVPN allows the host to use the nearest gateway to route traffic. You do this by creating IRBs on both PEs using different GW IP addresses. To accomplish this IRBs (IP  + MAC addresses) are advertised using a BGP extended community. When VM1 migrates to DC2, it sends packets to the mac address associated to GW IP address of DC1. The IRB in DC2 notices that the destination mac address for these packets is across the WAN, so it does the routing locally. When the arp entry for the GW in VM1 expires, the VM will arp again and the IRB in DC2 will send a reply to VM1 with it's updated mac address.

Another thing that happens when VM migration is performed in an EVPN network, the MAC address of the VM is now advertised in DC2, the PE in DC2 updates their mac table table while the PE in DC1 withdraws the entry.

To address fast convergence in a multi homed environment, a concept called an Ethernet Segment is introduced. The set of links connecting to two or more local PE routers are called an Ethernet Segment. Each segment has an unique identifier called an ESI. An ethernet tag is also used to identify each broadcast domain such a vlan. When an Ethernet segment fails, the local PE withdraws the corresponding Ethernet "route" from BGP which triggers all remote PE routers to update their forwarding tables to update the corresponding next-hop to the backup PE.

EVPN introduces Split Horizon. BUM flooding aka, Broadcast, Unknown unicast or Multicast traffic are encapsulated in a MPLS packet with the Ethernet Segment Identifier. This allows the Egress PE to make a forwarding decision and prevents loops, because the PEs know where the packet originated from.
This in turn makes it possible to forward traffic over multiple active links through the WAN and allows for the ability to load balance.

With these advantages EVPN makes it a viable choice for interconnecting Data Centers.