Last week I have received following question from one of my reader …
I
came to your blog post
http://blog.igics.com/2014/05/dell-force10-vlt-virtual-link-trunking.html and I
am really happy that you shared this information with us. However I was
wondering if you have tested a scenario with 4 S4810 with VLT configured on 2 x
2 and connected together (somewhere called mLAG). How do you continue to add
VLT couples to the setup? I would be really happy if you could provide any info
regarding such setup.
So let’s deep dive into VLT port-channel between two Force10
VLT Domains also known as mVLT. Please note that VLT can be configured not only
between two Force10 VLT domains but also between Force10 VLT domain and other
multi chassis port-channel technology like for instance CISCO virtual Port
Channel (vPC). However, this blog post is focused to single vendor solution mVLT
on DELL S-Series Switches (previously known as Force10 S-Series).
If you are not familiar with DELL Force10 VLT technology read my
previous blog post where is VLT described in detail. It is really important to understand VLT before you try to understand mVLT (Multi-domain VLT). By the way mVLT is called eVLT (Enhanced VLT) in Force10 documentation so it might be little bit confusing. Anyway mVLT is nothing else then regular virtual port channel (VLT) between two VLT domains. Therefore mVLT is quite good term if you ask me.
mVLT Logical Design
mVLT logical design is pretty straight forward. It is
required to achieve stretched L2 over two datacenters without any loops. This
topology is often called loop free topology and it is depicted on figure below
from spanning tree (STP) point of view.
However we would like to have hardware and link redundancy therefore multi chassis
port-channel technology (Force10 VLT in our particular case) is used to still have
simple loop free topology from spanning tree point of view but with switch unit
and physical link redundancy. Force10 mVLT solution is logically depicted on
figure below.
Please note, that each single VLT Domain act in spanning tree as a single logical switch.
DELL highly recommends using four links between VLT domains
because of higher redundancy and optimal data flow. However, sometimes your are constraint with links between sites. Two links DCI is also supported design but
not recommended because there is obviously lower link redundancy and therefore
higher probability of communication over VLTi which adds hop and therefore latency.
Two links mVLT DCI also known as square design is depicted on figure below.
Even the topology is loop free and from logical view we have just one switch on each datacenter spanning tree protocol should be enabled and configured just in case of human error or VLT domain failure or split. Rapid Spanning Tree (RSTP) protocol is good enough therefore used later in physical configurations.
mVLT Physical Design and Configuration
Physical design below shows connectivity of four (2x two) Force10
S4810 switches leveraging four links for DCI port-channel (mVLT).
Physical design for just two links DCI is depicted on following
schema.
And switch configuration snippets for four links mVLT are listed
below for completeness. Two link DCI is just variation of similar
configurations so you can simply reuse and slightly change four link configuration.
DCA-SWCORE-A – acts
as primary Root Bridge in RSTP in case of loop
!
hostname DCA-SWCORE-A
!
protocol spanning-tree rstp
no disable
hello-time 1
max-age 6
forward-delay 4
bridge-priority 4096
!
vlt domain 1
peer-link port-channel
128
back-up destination 172.16.201.2
primary-priority 1
system-mac mac-address 02:00:00:00:00:01
unit-id 0
peer-routing
!
proxy-gateway lldp
peer-domain-link
port-channel 127
!
interface TenGigabitEthernet 0/46
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface TenGigabitEthernet 0/47
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface fortyGigE 0/56
no ip address
mtu 12000
no shutdown
!
interface fortyGigE 0/60
no ip address
mtu 12000
no shutdown
!
interface ManagementEthernet 0/0
ip address 172.16.201.1/24
no shutdown
!
interface Port-channel 127
description "mVLT
- interconnect link"
no ip address
mtu 12000
switchport
vlt-peer-lag
port-channel 127
no shutdown
!
interface Port-channel 128
description "VLTi
- interconnect link"
no ip address
mtu 12000
channel-member
fortyGigE 0/56,60
no shutdown
!
DCA-SWCORE-B – acts as secondary Root Bridge in RSTP in
case of loop
!
hostname DCA-SWCORE-B
!
protocol spanning-tree rstp
no disable
hello-time 1
max-age 6
forward-delay 4
bridge-priority 8192
!
vlt domain 1
peer-link port-channel
128
back-up destination
172.16.201.1
primary-priority 8192
system-mac mac-address 02:00:00:00:00:01
unit-id 1
peer-routing
!
proxy-gateway lldp
peer-domain-link
port-channel 127
!
interface TenGigabitEthernet 0/46
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface TenGigabitEthernet 0/47
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface fortyGigE 0/56
no ip address
mtu 12000
no shutdown
!
interface fortyGigE 0/60
no ip address
mtu 12000
no shutdown
!
interface ManagementEthernet 0/0
ip address 172.16.201.2/24
no shutdown
!
interface Port-channel 127
description "mVLT
- interconnect link"
no ip address
mtu 12000
switchport
vlt-peer-lag
port-channel 127
no shutdown
!
interface Port-channel 128
description "VLTi
- interconnect link"
no ip address
mtu 12000
channel-member
fortyGigE 0/56,60
no shutdown
!
DCB-SWCORE-A – acts
as tertiary Root Bridge in RSTP in case of loop
!
hostname DCB-SWCORE-A
!
protocol spanning-tree rstp
no disable
hello-time 1
max-age 6
forward-delay 4
bridge-priority 12288
!
vlt domain 2
peer-link port-channel
128
back-up destination 172.16.202.2
primary-priority 1
system-mac mac-address 02:00:00:00:00:02
unit-id 0
peer-routing
!
proxy-gateway lldp
peer-domain-link
port-channel 127
!
interface TenGigabitEthernet 0/46
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface TenGigabitEthernet 0/47
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface fortyGigE 0/56
no ip address
mtu 12000
no shutdown
!
interface fortyGigE 0/60
no ip address
mtu 12000
no shutdown
!
interface ManagementEthernet 0/0
ip address 172.16.202.1/24
no shutdown
!
interface Port-channel 127
description "mVLT
- interconnect link"
no ip address
mtu 12000
switchport
vlt-peer-lag
port-channel 127
no shutdown
!
interface Port-channel 128
description "VLTi
- interconnect link"
no ip address
mtu 12000
channel-member
fortyGigE 0/56,60
no shutdown
!
DCB-SWCORE-B – acts
as quaternary Root Bridge in RSTP in case of loop
!
hostname DCB-SWCORE-B
!
protocol spanning-tree rstp
no disable
hello-time 1
max-age 6
forward-delay 4
bridge-priority 16384
!
vlt domain 2
peer-link port-channel
128
back-up destination 172.16.202.1
primary-priority 8192
system-mac mac-address 02:00:00:00:00:02
unit-id 1
peer-routing
!
proxy-gateway lldp
peer-domain-link
port-channel 127
!
interface TenGigabitEthernet 0/46
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface TenGigabitEthernet 0/47
no ip address
mtu 12000
!
port-channel-protocol
LACP
port-channel 127 mode
active
dampening 10 100 1000
60
no shutdown
!
interface fortyGigE 0/56
no ip address
mtu 12000
no shutdown
!
interface fortyGigE 0/60
no ip address
mtu 12000
no shutdown
!
interface ManagementEthernet 0/0
ip address 172.16.202.2/24
no shutdown
!
interface Port-channel 127
description "mVLT
- interconnect link"
no ip address
mtu 12000
switchport
vlt-peer-lag
port-channel 127
no shutdown
!
interface Port-channel 128
description "VLTi
- interconnect link"
no ip address
mtu 12000
channel-member
fortyGigE 0/56,60
no shutdown
!
Conclusion
Force10 mVLT is great technology for loop free L2 network topology. It can be leveraged for local loop free topologies inside single datacenter or as L2 extension between datacenters. However our networks are usually built to support IP traffic therefore L3
considerations has to be addressed as well. Just think about default IP gateway
behavior and potential DCI potential trombone. That’s where other VLT features peer-routing and proxy-gateway come in to play and mitigate DCI trombone issue. You can see these technologies configured in VLT configurations above. But that’s another topic for another blog post.
To be absolutely honest I personally don't recommend L2 interconnects between datacenters without any good justification. I strongly recommend L3 datacenter interconnects and when stretched L2 is needed then some network overlay technology can be leveraged. L3 will guarantee independent availability zones and splitting L2 failure domain. But on the other hand such network overlay needs some other bits and pieces which in some cases increase complexity and cost. Therefore mVLT can be seriously considered for cost effective datacenter L2 extensions. That's a typical "it depends" scenario where these two design decision options has to be compared and final decision clearly justified.
If you want to know more about these technologies or use cases just ask and we can go deeper or
broader. And as always any feedback and/or comment is highly appreciated.
