vSphere 7 has been announced and will be GA and available to download into our labs very soon. Let's briefly summarize what's new in vSphere 7 and put some links to other resources.
vSphere with Kubernetes
Project Pacific evolved into Integrated Kubernetes and Tanzu. vSphere has been transformed in order to support both VMs and containers. Tanzu Kubernetes Grid Service is how customers can run fully compliant and conformant Kubernetes with vSphere. However, when complete conformance with the open-source project isn’t required, the vSphere Pod Service can provide optimized performance and improved security through VM-like isolation. Both of these options are available through VMware Cloud Foundation 4.
The important takeaway is that Kubernetes is now built into vSphere which allows developers to continue using the same industry-standard tools and interfaces they’ve been using to create modern applications. vSphere Admins also benefit because they can help manage the Kubernetes infrastructure using the same tools and skills they have developed around vSphere.
References:
DRS used to focus on the cluster state and the algorithm would recommend a vMotion when it would benefit the balance of the cluster as a whole. This meant that DRS used to achieve cluster balance by using a cluster-wide standard deviation model. The new DRS logic computes a VM DRS score on the hosts and moves the VM to a host that provides the highest VM DRS score. This means DRS cares less about the ESXi host utilization and prioritizes the VM “happiness”. The VM DRS score is also calculated every minute and this results in a much more granular optimization of resources.
References:
Improvements in live migrations of monster workloads. Monster VMs with a large memory & CPU footprint, like SAP HANA and Oracle database backends, had challenges being live-migrated using vMotion. The performance impact during the vMotion process and the potentially long stun-time during the switchover phase meant that customers were not comfortable using vMotion for these large workloads. With vSphere 7, we are bringing back that capability as we have greatly improved the vMotion logic.
How the improvement was achieved?
References:
There is a new framework called Assignable Hardware that was developed to extend support for vSphere features when customers utilize hardware accelerators. It introduces vSphere DRS (for initial placement of a VM in a cluster) and vSphere High Availability (HA) support for VM’s equipped with a passthrough PCIe device or a NVIDIA vGPU. Related to Assignable Hardware is the new Dynamic DirectPath I/O which is a new way of configuring passthrough to expose PCIe devices directly to a VM. The hardware address of a PCIe device is no longer directly mapped to the configuration (vmx) file of a virtual machine. Instead, it is now exposed as a PCIe device capability to the VM.
Together, Dynamic DirectPath I/O, NVIDIA vGPU, and Assignable Hardware are a powerful new combination unlocking some great new functionality. For example, let’s look at a VM that requires an NVIDIA V100 GPU. Assignable Hardware will now interact with DRS when that VM is powered on (initial placement) to find an ESXi host that has such a device available, claim that device, and register the VM to that host. If there is a host failure and vSphere HA kicks in, Assignable Hardware also allows for that VM to be restarted on a suitable host with the required hardware available.
References:
Bitfusion stays in vSphere 7 as a Tech Preview feature. It allows us to leverage hardware accelerators (GPUs) across an infrastructure (over network fabric) and integrate it with evolving technologies such as FPGAs and custom ASICs using the same infrastructure. This is actually the first implementation of the software-defined composable infrastructure within VMware SDDC stack, therefore it is a very promising and very needed technology for modern applications such as ML/AI workloads.
References:
Precision Time Protocol is helpful for financial and scientific applications requiring sub-millisecond accuracy. PTP requires VM Hardware 17 and it must be enabled on both an in-quest device and an ESXi service. Thus, you have to choose between NTP or PTP.
VM Template Management (Content Library)
VM template check-in and check-out operations with versioning feature. Content Library should also support of controlled replication into remote locations. With these vSphere 7 Content Library improvements, the Content Library is now a mature and very useful tool for VM template management.
References:
Desired state of ESXi hosts image (divers & firmware) and host configuration assigned to vSphere Clusters. It requires integration with hardware vendor system management like Dell OMIVV (OpenManage Integration for VMware vCenter) or HPE OneView for VMware vCenter.
References:
Update Planner is part of vLCM and it monitors current interoperability based on VMware HCL.
References:
Export / Import of VCSA (vCenter) configuration. This is good for effective management of a lot of vCenters but please, do NOT expect export/import of vCenter objects like Clusters, VM Folders, Resource Pools, Virtual Switches, etc... This is export / import of VCSA configurations.
References:
VCSA now supports multiple (up to 4) vNICs. The first vNIC (vNIC0) is for management, the second (vNIC1) is dedicated for vCenter Server HA and other vNICs can be used for other purposes like a backup or so.
vCenter and SSO Architecture
vCenter Server Appliance (VCSA) with embedded Platform Service Controler (PSC). External PSC is not supported and it leads into simple SSO topology.
Simplified Certificate Management
Much simpler SSL certificate management. Fewer certificates to manage. For example, vCenter has only two SSL certificates, a Machine SSL certificate, and Certification Authority Certificate. vSphere 7 introduced some vSphere Client UI improvements and also the REST API for certificate management for environments with more vCenters to manage. This is, of course, beneficial for environments implemented based on VMware Validated Designs (VVD) or VMware Cloud Foundation (VCF) environments which is the automated implementation of VVD.
Identity Federation
vCenter is not the key Identity Management System anymore. vSphere Client is using external authentication providers to optimize IDM integration in customer's environments. The first implementation supports only Microsoft Active Directory Federation Services (ADFS), however, VMware SSO still exists, therefore the customer can choose if he will use the brand new Identity Federation or keep existing AD/LDAP authentication through VMware SSO.
vSphere Trust Authority (vTA)
In vSphere 7, vCenter is not trusted authority anymore. vSphere 7 introduces vTA, which creates a hardware root of trust using a separate ESXi host cluster.
vSGX - Support of Intel Software Guard Extensions (SGX)
vSphere 7 introduces support of Intel Software Guard Extensions. I was blogging about SGX a few years ago in blog post Intel Software Guard Extensions (SGX) in VMware VM. Intel SGX allows applications to work with hardware to create a secure enclave that cannot be viewed by the guest OS or hypervisor. With SGX, applications can move sensitive logic and storage into this enclave. SGX is the Intel-only feature. AMD has SEV, which is a different approach.
References:
Hosts per single vCenter: 2,500
Powered-on VMs on single vCenter: 30,000
Hosts per SSO domain (vCenters in linked mode): 15,000
Powered-on VMs per SSO domain (vCenters in linked mode): 150,000
vCenter Server Latency - vCenter <-> vCenter: 150 ms
vCenter Server Latency - vCenter <-> ESXi: 150 ms
vCenter Server Latency - vSphere Client (web browser) <-> vCenter: 100 ms
The improvements between vSphere 6.7 and 7 are clearly visible in figure below.
For further configuration maximums, look at https://configmax.vmware.com/
Skyline Health for vSphere 7
Skyline Health for vSphere 7 is the unified health check tool for vSphere which works exactly as Skyline Health for vSAN available since vSphere 6.7 U3. It brings into infrastructure operations similar approach developers are doing in agile development methods - automated testing. You can think about it as a set of tests (health check tests) continually testing everything works as expected.
NVMe over Fabric
In vSphere 7, VMware added support for shared NVMe storage using NVMeoF. For external connectivity, NVMe over Fibre Channel and NVMe over RDMA (RoCE v2) are supported.
References:
Conclusion
vSphere 7 is another major vSphere Release. For those who work with VMware virtual infrastructures for ages (see old ESX 3i below), it is amazing where the VMware virtualization platform (vSphere 7, ESXi 7) evolved and what is possible nowadays.
Nowadays, there are totally different reasons to upgrade to the latest vSphere version in comparison to the old days of server consolidation, TCO reduction, and better manageability. Top reasons to upgrade to vSphere 7 are
vSphere with Kubernetes
Project Pacific evolved into Integrated Kubernetes and Tanzu. vSphere has been transformed in order to support both VMs and containers. Tanzu Kubernetes Grid Service is how customers can run fully compliant and conformant Kubernetes with vSphere. However, when complete conformance with the open-source project isn’t required, the vSphere Pod Service can provide optimized performance and improved security through VM-like isolation. Both of these options are available through VMware Cloud Foundation 4.
The important takeaway is that Kubernetes is now built into vSphere which allows developers to continue using the same industry-standard tools and interfaces they’ve been using to create modern applications. vSphere Admins also benefit because they can help manage the Kubernetes infrastructure using the same tools and skills they have developed around vSphere.
References:
- Introducing vSphere 7: Features & Technology for the Hybrid Cloud - https://blogs.vmware.com/vsphere/2020/03/vsphere-7-features.html
- INITIAL PLACEMENT OF A VSPHERE POD - https://frankdenneman.nl/2020/03/06/initial-placement-of-a-vsphere-native-pod/
- SCHEDULING VSPHERE PODS - https://frankdenneman.nl/2020/03/20/scheduling-vsphere-pods/
DRS used to focus on the cluster state and the algorithm would recommend a vMotion when it would benefit the balance of the cluster as a whole. This meant that DRS used to achieve cluster balance by using a cluster-wide standard deviation model. The new DRS logic computes a VM DRS score on the hosts and moves the VM to a host that provides the highest VM DRS score. This means DRS cares less about the ESXi host utilization and prioritizes the VM “happiness”. The VM DRS score is also calculated every minute and this results in a much more granular optimization of resources.
Another new feature is "DRS Scalable Shares". Scalable Shares solves a problem many have been facing over the last decade or so, which is that DRS does not take the number of VMs in the pool into account when it comes to allocating resources.
- Introducing vSphere 7: Features & Technology for the Hybrid Cloud - https://blogs.vmware.com/vsphere/2020/03/vsphere-7-features.html
- vSphere 7 and DRS Scalable Shares, how are they calculated? - http://www.yellow-bricks.com/2020/03/16/vsphere-7-and-drs-scalable-shares-how-are-they-calculated/
Improvements in live migrations of monster workloads. Monster VMs with a large memory & CPU footprint, like SAP HANA and Oracle database backends, had challenges being live-migrated using vMotion. The performance impact during the vMotion process and the potentially long stun-time during the switchover phase meant that customers were not comfortable using vMotion for these large workloads. With vSphere 7, we are bringing back that capability as we have greatly improved the vMotion logic.
How the improvement was achieved?
- Multi-threading
- A dedicated vCPU is used for page tracing which means that the VM and its applications can keep working while the vMotion processes are occurring. Prior to vSphere 7, page tracing occurred on all vCPUs within a VM, which could cause the VM and its workload to be resource-constrained by the migration itself.
- Introducing vSphere 7: Features & Technology for the Hybrid Cloud - https://blogs.vmware.com/vsphere/2020/03/vsphere-7-features.html
- vSphere 7 – vMotion Enhancements - https://blogs.vmware.com/vsphere/2020/03/vsphere-7-vmotion-enhancements.html
- Introduction to the vMotion process - https://www.youtube.com/watch?v=0Q_MPVeuWgc
- vMotion Memory Copy - Under the Hood - https://www.youtube.com/watch?v=KdI7-FJEEmk
There is a new framework called Assignable Hardware that was developed to extend support for vSphere features when customers utilize hardware accelerators. It introduces vSphere DRS (for initial placement of a VM in a cluster) and vSphere High Availability (HA) support for VM’s equipped with a passthrough PCIe device or a NVIDIA vGPU. Related to Assignable Hardware is the new Dynamic DirectPath I/O which is a new way of configuring passthrough to expose PCIe devices directly to a VM. The hardware address of a PCIe device is no longer directly mapped to the configuration (vmx) file of a virtual machine. Instead, it is now exposed as a PCIe device capability to the VM.
Together, Dynamic DirectPath I/O, NVIDIA vGPU, and Assignable Hardware are a powerful new combination unlocking some great new functionality. For example, let’s look at a VM that requires an NVIDIA V100 GPU. Assignable Hardware will now interact with DRS when that VM is powered on (initial placement) to find an ESXi host that has such a device available, claim that device, and register the VM to that host. If there is a host failure and vSphere HA kicks in, Assignable Hardware also allows for that VM to be restarted on a suitable host with the required hardware available.
References:
- Assignable Hardware in vSphere 7 - https://www.youtube.com/watch?v=AbOeM5Ojt2g
- More about Tesla GPUs: https://interactive.spiceworks.com/static/DellEMC-eBook-PowerEdgeNVIDIA/index.html
- DEEP LEARNING TECHNOLOGY STACK OVERVIEW FOR THE VADMIN – PART 1 - https://frankdenneman.nl/2020/03/12/deep-learning-technology-stack-overview-for-the-vadmin-part-1/
Bitfusion stays in vSphere 7 as a Tech Preview feature. It allows us to leverage hardware accelerators (GPUs) across an infrastructure (over network fabric) and integrate it with evolving technologies such as FPGAs and custom ASICs using the same infrastructure. This is actually the first implementation of the software-defined composable infrastructure within VMware SDDC stack, therefore it is a very promising and very needed technology for modern applications such as ML/AI workloads.
References:
- More about Tesla GPUs: https://interactive.spiceworks.com/static/DellEMC-eBook-PowerEdgeNVIDIA/index.html
- DEEP LEARNING TECHNOLOGY STACK OVERVIEW FOR THE VADMIN – PART 1 - https://frankdenneman.nl/2020/03/12/deep-learning-technology-stack-overview-for-the-vadmin-part-1/
Precision Time Protocol is helpful for financial and scientific applications requiring sub-millisecond accuracy. PTP requires VM Hardware 17 and it must be enabled on both an in-quest device and an ESXi service. Thus, you have to choose between NTP or PTP.
VM Template Management (Content Library)
VM template check-in and check-out operations with versioning feature. Content Library should also support of controlled replication into remote locations. With these vSphere 7 Content Library improvements, the Content Library is now a mature and very useful tool for VM template management.
References:
- What's New in vCenter Server 7 - https://www.youtube.com/watch?v=XKgrJXN6Q0U
- VM Template Check-in and Check-out and versioning - https://www.vladan.fr/vmware-vsphere-7-0-vm-template-check-in-and-check-out-and-versioning/
Desired state of ESXi hosts image (divers & firmware) and host configuration assigned to vSphere Clusters. It requires integration with hardware vendor system management like Dell OMIVV (OpenManage Integration for VMware vCenter) or HPE OneView for VMware vCenter.
References:
- vSphere Lifecycle Manager (vLCM) - https://www.youtube.com/watch?v=R4NGT12hvSM
- What's New in vCenter Server 7 - https://www.youtube.com/watch?v=XKgrJXN6Q0U
Update Planner is part of vLCM and it monitors current interoperability based on VMware HCL.
References:
- What's New in vCenter Server 7 - https://www.youtube.com/watch?v=XKgrJXN6Q0U (1:40)
Export / Import of VCSA (vCenter) configuration. This is good for effective management of a lot of vCenters but please, do NOT expect export/import of vCenter objects like Clusters, VM Folders, Resource Pools, Virtual Switches, etc... This is export / import of VCSA configurations.
References:
- What's New in vCenter Server 7 - https://www.youtube.com/watch?v=XKgrJXN6Q0U
- VMware vCenter Server 7.0 Profiles - https://www.vladan.fr/vmware-vcenter-server-7-0-profiles/
VCSA now supports multiple (up to 4) vNICs. The first vNIC (vNIC0) is for management, the second (vNIC1) is dedicated for vCenter Server HA and other vNICs can be used for other purposes like a backup or so.
vCenter and SSO Architecture
vCenter Server Appliance (VCSA) with embedded Platform Service Controler (PSC). External PSC is not supported and it leads into simple SSO topology.
Simplified Certificate Management
Much simpler SSL certificate management. Fewer certificates to manage. For example, vCenter has only two SSL certificates, a Machine SSL certificate, and Certification Authority Certificate. vSphere 7 introduced some vSphere Client UI improvements and also the REST API for certificate management for environments with more vCenters to manage. This is, of course, beneficial for environments implemented based on VMware Validated Designs (VVD) or VMware Cloud Foundation (VCF) environments which is the automated implementation of VVD.
Identity Federation
vCenter is not the key Identity Management System anymore. vSphere Client is using external authentication providers to optimize IDM integration in customer's environments. The first implementation supports only Microsoft Active Directory Federation Services (ADFS), however, VMware SSO still exists, therefore the customer can choose if he will use the brand new Identity Federation or keep existing AD/LDAP authentication through VMware SSO.
vSphere Trust Authority (vTA)
In vSphere 7, vCenter is not trusted authority anymore. vSphere 7 introduces vTA, which creates a hardware root of trust using a separate ESXi host cluster.
vSGX - Support of Intel Software Guard Extensions (SGX)
vSphere 7 introduces support of Intel Software Guard Extensions. I was blogging about SGX a few years ago in blog post Intel Software Guard Extensions (SGX) in VMware VM. Intel SGX allows applications to work with hardware to create a secure enclave that cannot be viewed by the guest OS or hypervisor. With SGX, applications can move sensitive logic and storage into this enclave. SGX is the Intel-only feature. AMD has SEV, which is a different approach.
References:
- Intel Software Guard Extensions (SGX) in VMware VM - https://www.vcdx200.com/2018/11/intel-software-guard-extensions-sgx-in.html
- vSGX & Secure Enclaves in vSphere 7 - https://youtu.be/5Y40GzAF2_s
Hosts per single vCenter: 2,500
Powered-on VMs on single vCenter: 30,000
Hosts per SSO domain (vCenters in linked mode): 15,000
Powered-on VMs per SSO domain (vCenters in linked mode): 150,000
vCenter Server Latency - vCenter <-> vCenter: 150 ms
vCenter Server Latency - vCenter <-> ESXi: 150 ms
vCenter Server Latency - vSphere Client (web browser) <-> vCenter: 100 ms
The improvements between vSphere 6.7 and 7 are clearly visible in figure below.
For further configuration maximums, look at https://configmax.vmware.com/
Skyline Health for vSphere 7
Skyline Health for vSphere 7 is the unified health check tool for vSphere which works exactly as Skyline Health for vSAN available since vSphere 6.7 U3. It brings into infrastructure operations similar approach developers are doing in agile development methods - automated testing. You can think about it as a set of tests (health check tests) continually testing everything works as expected.
NVMe over Fabric
In vSphere 7, VMware added support for shared NVMe storage using NVMeoF. For external connectivity, NVMe over Fibre Channel and NVMe over RDMA (RoCE v2) are supported.
References:
Conclusion
vSphere 7 is another major vSphere Release. For those who work with VMware virtual infrastructures for ages (see old ESX 3i below), it is amazing where the VMware virtualization platform (vSphere 7, ESXi 7) evolved and what is possible nowadays.
 |
| Old good ESXi from Virtual Infrastructure 3 from 2006-ish year :-) |
- Scalability: The fastest path to the Hybrid/Multi-Cloud and increase scalability through leveraging HCI (Hyper-Converged Infrastructure)
- Security: Infrastructure security, secure audits, and account management
- Performance: maximize performance and efficiency
- Manageability: Reduce complexity, simplify software patching and hardware upgrades, proactive support technology and services
- monster workloads such as SAP HANA
- traditional applications in virtual machines
- modern distributed applications (Cloud Native Applications, CNA) containerized and orchestrated by Kubernetes
