VMware Ports and Protocols

VMware recently released a very interesting tool. The tool documents all network ports and protocols required for communication from/to some VMware products. At the moment, there are the following products

• vSphere
• vSAN
• NSX for vSphere
• vRealize Network Insight
• vRealize Operations Managers
• vRealize Automation

I believe other products will follow. See the screenshot of the tool below.

The tool is available at https://ports.vmware.com/

I'm pretty sure technical designers and implementation engineers will love this tool.

VMware vSphere 6.7 Update 3 is GA

VMware vSphere 6.7 Update 3 is GA as of August 20, 2019.

The most interesting new feature is the possibility to change the Primary Network Identifier (PNID) of vCenter Server Appliance

With vCenter Server 6.7 Update 3, you can change the Primary Network Identifier (PNID) of your vCenter Server Appliance. You can change the vCenter Server Appliance FQDN or hostname, and also modify the IP address configuration of the virtual machine Management Network (NIC 0).

I wrote about PNIC last year here on this blog post What is vCenter PNID?
It is a great improvement that you can change vCenter FQDN, however, the process has some

For further operational details on how to change vCenter Server's FQDN/PNID read the blog post - Changing your vCenter Server’s FQDN.

There are a few more new features and enhancements

• vCenter Server 6.7 Update 3 supports a dynamic relationship between the IP address settings of a vCenter Server Appliance and a DNS server by using the Dynamic Domain Name Service (DDNS). The DDNS client in the appliance automatically sends secure updates to DDNS servers on scheduled intervals.
• With vCenter Server 6.7 Update 3, you can publish your .vmtx templates directly from a published library to multiple subscribers in a single action instead of performing a synchronization from each subscribed library individually. The published and subscribed libraries must be in the same linked vCenter Server system, regardless if on-premises, on cloud, or hybrid. Work with other templates in content libraries does not change.
• With vCenter Server 6.7 Update 3, if the overall health status of a vSAN cluster is Red, APIs to configure or extend HCI clusters throw InvalidState exception to prevent further configuration or extension. This fix aims to resolve situations when mixed versions of ESXi host in a HCI cluster might cause vSAN network partition.
• ixgben driver enhancements
• VMXNET3 enhancements
• NVIDIA virtual GPU (vGPU) enhancements
• bnxtnet driver enhancements
• QuickBoot support enhancements
• Configurable shutdown time for the sfcbd service

For further information read ESXi Release Notes here, vCenter Release Notes here and VMware Update Manager here.

I also recommend reading the blog post Announcing vSphere 6.7 Update 3 available at https://blogs.vmware.com/vsphere/2019/08/announcing-vsphere-6-7-update-3.html

VMware vSAN 6.7 U3 has been released

VMware vSAN 6.7 U3 is GA as of August 20, 2019!

This is a great release. I was waiting mainly for Native support for Windows Server Failover Clusters which is now officially supported so no more vSAN iSCSI targets and in-Guest iSCSI for shared disks across the WSFC as vSAN VMDKs now support SCSI-3 persistent reservations. This is a great improvement and significant simplification for vSphere / vSAN architects, technical designers, implementers, and operational guys. For further details see blog post "Native SQL Server Cluster support on vSAN" 

The second feature I like a lot is the performance improvement. Performance improvements and better performance predictability are always welcome. More predictable application performance is achieved in vSAN 6.7 U3 by destaging data from the write buffer to the capacity tier. This makes vSAN more efficient and consistent with increased throughput for sequential writes. The increased consistency in destaging operations results in a smaller deviation between high and low latency. This increased throughput also reduces the amount of time necessary for resyncs from repairs and rebuild tasks. The destaging improvement is depicted in the figure below.

However, there are other operations and availability improvements in 6.7 U3. Let's highlight the vSAN 6.7 U3 improvements one by one as published in Release Notes.

What's New?

Cloud Native Applications

Cloud Native Storage: Introducing vSphere integrated provisioning and management of Kubernetes persistent volumes (PVs) and a CSI (Container Storage Interface) based plugin. This integration enables unified management of modern and traditional applications in vSphere.

Intelligent Operations

Enhanced day 2 monitoring: vSAN’s capacity dashboard has been overhauled for greater simplicity, and the resync dashboard now includes improved completion estimates and granular filtering of active tasks

Data migration pre-check report: A new detailed report for greater insight and predictive analysis of host maintenance mode operations

Greater resiliency with capacity usage conditions: Key enhancements include reduced transient capacity usage for policy changes, and improved remediation during capacity-strained scenarios

Automated rebalancing: Fully automate all rebalancing operations with a new cluster-level configuration, including modifiable thresholds

Native support for Windows Server Failover Clusters: Deploy WSFC natively on vSAN VMDKs with SCSI-3 persistent reservations

Enhanced Performance & Availability

Performance enhancements: Deduplication & compression enabled workloads will have improved performance in terms of predictable I/O latencies and increased sequential I/O throughput

Availability enhancements: Optimizations to sequential I/O throughput and resync parallelization will result in faster rebuilds

For further details about vSAN 6.7 U3 read blog post vSAN 6.7 Update 3 – What’s New available at https://blogs.vmware.com/virtualblocks/2019/08/13/vsan67u3-whats-new/

vSAN Capacity planning - Understanding vSAN memory consumption in ESXi

It is very clear that VMware vSAN (VMware's software-defined storage) has the momentum in the field, as almost all my customers are planning and designing vSAN in their environments. Capacity planning is an important part of any logical design, so we have to do the same for vSAN. Capacity planning is nothing else than simple math, however, we need to know how the designed system works and what overheads we have to include in our capacity planning exercise. Over the years, a lot of VMware vSphere technical designers did get knowledge and practice how to do capacity planning for core vSphere because server virtualization is here for ages (13+ years). But we all are just starting (3+ years) with vSAN designs, therefore it will take some time to gain the practice and know-how what is important to calculate in terms of VMware hyper-converged infrastructure (VMware HCI = vSphere + vSAN).

One of the many important factors for HCI capacity planning is vSAN memory consumption from the ESXi host memory. There is very good VMware KB 2113954 explaining the math calculation and formulas behind the scene. However, we are tech geeks, so we do not want to do the math on the paper so here is the link to Google Sheets calculator I have prepared for vSAN (All-Flash) memory overhead calculation.

Here is the calculator embedded into this blog post, however, it is just in read-only mode. If you want to change parameters (yellow cells) you have to open Google Sheet available on this link.



Note: I did not finish the calculator for Hybrid configuration because I personally believe that 99% of vSAN deployments should be All-Flash. The reason for this assumption is the fact, that Flash capacity is only 2x or 3x more expensive than magnetic disks and the lower price of magnetic disks is not worth to low speed you can achieve by magnetic disks. In terms of capacity, advanced technics like erasure coding (RAID-5, RAID-6) and deduplication + compression can give you back capacity in All-Flash vSAN as these technics are available or make sense only on All-Flash vSAN. If you would like the same calculator for Hybrid vSAN, leave the comment below this blog post and I will try to find some spare time to prepare another sheet for Hybrid vSAN.

Let's document here some design scenarios with vSAN memory consumptions.

Scenario 1
ESXi host system memory: 192 GB
Number of disk groups: 1
Cache disk size in each disk group: 400 GB
Number of capacity disks in each disk group: 4
vSAN memory consumption per ESXi host is 17.78 GB.

Scenario 2
ESXi host system memory: 192 GB
Number of disk groups: 2
Cache disk size in each disk group: 400 GB
Number of capacity disks in each disk group: 2
vSAN memory consumption per ESXi host is 28 GB.

Scenario 3
ESXi host system memory: 256 GB
Number of disk groups: 2
Cache disk size in each disk group: 400 GB
Number of capacity disks in each disk group: 2
vSAN memory consumption per ESXi host is 28.64 GB.

Hope this is informative and it helps broader VMware community. 

Updating Firmware in vSAN Clusters from VUM

If you operate vSAN you know that correct firmware and drivers are super important for system stability as vSAN software heavily depends on IO controller and physical disks within the server.

Different server vendors have different system management. Some are more complex than other but typical vSphere admin is using vSphere Update Manager (VUM) so would not it be cool to do firmware management directly from VUM? Yes, of course, so how it could be done?

Well, there is a chapter in vSphere documentation covering "Updating Firmware in vSAN Clusters", however, there is no information what hardware is supported. I did some internal research and colleagues have pointed me to VMware KB 60382 "IO Controllers that vSAN supports firmware updating" where supported IO Controllers are listed.

At the time of writing this post, there are plenty of Dell IO Controllers, two Lenovo, two SuperMicro and one Fujitsu.

At the moment, you can update only IO Controllers but it may or may not change in the future.