Not only vSAN but also vMotion, NFS and other types of traffic can benefit from Jumbo Frames configured on an ethernet network as the network traffic should consume fewer CPU cycles and achieve higher throughput.
Jumbo Frames must be configured end-to-end, therefore we should start the configuration in the network core on Physical Switches, then continue to Virtual Switches and finish on VMkernel ports (vmk). These three configuration places are depicted on schema below.
Physical Switch
Jumbo Frames on physical switches can be configured per the whole switch or per switch ports. It depends on a particular physical switch but my Force10 switch supports configuration only per switch ports as shown on the screenshot below. The configuration per the whole switch would be easier with less configuration and as far as I know, some Cisco switches support it.
If you have more physical switches, all ports in the path must be configured for Jumbo Frames.
Virtual Switch
On the screenshot below you can see the Jumbo Frame configuration on my VMware Virtual Distributed Switch.
VMkernel port
And last but not least, the configuration on VMkernel port, in this case, the vmk interface used for vSAN traffic.
Final test
After any implementation, we should do the test that implementation was successful and all is working as expected. We should log in to ESXi host via ssh and use following ping command
vmkping -I vmk5 -s 8972 -d 192.168.26.122
-d set DF bit (IPv4) or disable fragmentation (IPv6)
-I outgoing interface
-s set the number of ICMP data bytes to be sent.
The default is 56, which translates to a 64 byte
ICMP frame when added to the 8 byte ICMP header.
(Note: these sizes does not include the IP header).
and here is the result in case everything is configured correctly.
In case the message is longer than configured MTU we would see the following ...
You can ask why we use size 8972 and not 9000?
The reason for the 8972 on *nix devices is that the ICMP/ping implementation doesn’t encapsulate the 28 byte ICMP (8) + IP (20) (ping + standard internet protocol packet) header – thus we must take the 9000 and subtract 28 = 8972. [source & credits for the answer]
Hope this helps.
Jumbo Frames must be configured end-to-end, therefore we should start the configuration in the network core on Physical Switches, then continue to Virtual Switches and finish on VMkernel ports (vmk). These three configuration places are depicted on schema below.
Jumbo Frames on physical switches can be configured per the whole switch or per switch ports. It depends on a particular physical switch but my Force10 switch supports configuration only per switch ports as shown on the screenshot below. The configuration per the whole switch would be easier with less configuration and as far as I know, some Cisco switches support it.
If you have more physical switches, all ports in the path must be configured for Jumbo Frames.
Virtual Switch
On the screenshot below you can see the Jumbo Frame configuration on my VMware Virtual Distributed Switch.
VMkernel port
And last but not least, the configuration on VMkernel port, in this case, the vmk interface used for vSAN traffic.
Final test
After any implementation, we should do the test that implementation was successful and all is working as expected. We should log in to ESXi host via ssh and use following ping command
vmkping -I vmk5 -s 8972 -d 192.168.26.122
-d set DF bit (IPv4) or disable fragmentation (IPv6)
-I
-s
The default is 56, which translates to a 64 byte
ICMP frame when added to the 8 byte ICMP header.
(Note: these sizes does not include the IP header).
and here is the result in case everything is configured correctly.
In case the message is longer than configured MTU we would see the following ...
You can ask why we use size 8972 and not 9000?
The reason for the 8972 on *nix devices is that the ICMP/ping implementation doesn’t encapsulate the 28 byte ICMP (8) + IP (20) (ping + standard internet protocol packet) header – thus we must take the 9000 and subtract 28 = 8972. [source & credits for the answer]
Hope this helps.
No comments:
Post a Comment