First of all, let's be absolutely clear. Disks with 4K sector size are not currently supported by VMware. See VMware KB- Support statement for 512e and 4K Native drives for VMware vSphere and VSAN (2091600)
UPDATE: vSphere 6.5 and VSAN 6.5 introduced 512e support so 4K native drives with 512 emulation (512e) are supported. In other words, 4K native drives without 512e are still not supported.
UPDATE 2018-04-18: vSphere 6.7 introdiuced support of 4K native drives.
IMPORTANT STATEMENTS FROM KB
Does current GA version of vSphere and VSAN support 4K Native drives?
No. 4K Native drives are not supported in current GA releases of vSphere and VSAN.
Does current GA version of vSphere and VSAN support 512e drives?
No. 512e drives are not supported with the current versions of vSphere and VSAN due to potential performance issues when using these drives.
Therefore only 512n (native) drives are supported on any ESXi (5.x, 6.x) at the moment.
It is usually not big deal on shared storage systems (aka disk arrays) because logical volumes are virtually emulated and sector size is usually 512 by default or configurable (512 or 4K).
I thought that it is the same with RAID controllers because virtual volumes are also emulated by RAID controller. However, I have just recently learned that it is not true. At least not for all RAID controllers. For example DELL PERC H730 (the best RAID controller DELL currently offers) doesn't allow you to choose sector size for virtual volume. Instead, sector size is passed from physical disks to operating system - ESXi hypervisor in our case.
Here is one real customer story with 4K native disks.
The customer was not able to create datastore on some disks. The error message was ...
Call "HostDatastoreSystem.QueryVmfsDatastoreCreateOptions" for object "ha-datastoresystem" on ESXi "esxi-test" failed.
The error message is depicted on screenshot below.
Server Model: PowerEdge R530 – System Revision I
Operating System: VMware ESXi 5.5.0 build-3343343
BIOS Version: 1.5.4
Lifecycle Controller Firmware: 2.21.21.21
RAID Controller: Perc H730Mini
RAID1 (SYSTEM)
Physical Disk 0:1:0 Online 0 278.88 GB Not Capable SAS HDD No
Physical Disk 0:1:1 Online 1 278.88 GB Not Capable SAS HDD No
DATASTORE – successfully created during ESXi installation – OK
RAID1 (DATA)
Physical Disk 0:1:2 Online 2 558.38 GB Not Capable SAS HDD No
Physical Disk 0:1:3 Online 3 558.38 GB Not Capable SAS HDD No
DATASTORE – Datastore cannot be created because 4K - FATAL ISSUE
RAID5 (DATA)
Physical Disk 0:1:4 Online 4 1862.50 GB Not Capable SAS HDD No
Physical Disk 0:1:5 Online 5 1862.50 GB Not Capable SAS HDD No
Physical Disk 0:1:6 Online 6 1862.50 GB Not Capable SAS HDD No
DATASTORE – datastore can be created but not officially supported by VMware because 512e – RISK
RAID Controller list of disks
T17: C0:PD Flags State Type Size S N F P Vendor Product Rev P C ID SAS Addr Port Phy DevH WU BFw BRev
T17: C0:------------------------------------------------------------------------------------------------------------------------------
T17: C0:0 f1400005 00020 00 22ecb25b 0 0 0 1 TOSHIBA AL13SXB30EN DK02 0 0 500003969802a076 03 04 0010 1 NA NA - 512b (512 Native)
T17: C0:1 f1400005 00020 00 22ecb25b 0 0 0 1 TOSHIBA AL13SXB30EN DK02 0 0 500003969802a062 00 00 000a 1 NA NA - 512b (512 Native)
T17: C0:2 f1400005 00020 00 8bba5f5 0 0 0 1 HGST HUC156060CS4204 EK11 0 0 5000cca059596e59 05 06 000e 1 NA NA - 4kn (4k Native)
T17: C0:3 f1400005 00020 00 8bba5f5 0 0 0 1 HGST HUC156060CS4204 EK11 0 0 5000cca0595aa1cd 02 02 000c 1 NA NA - 4kn (4k Native)
T17: C0:4 f1400005 00020 00 e8e088af 0 0 0 1 SEAGATE ST2000NX0273 NS28 0 0 5000c5008f3a8efd 04 05 000d 1 NA NA - 512e (512 Emulation)
T17: C0:5 f1400005 00020 00 e8e088af 0 0 0 1 SEAGATE ST2000NX0273 NS28 0 0 5000c5008f3ae545 01 01 000b 1 NA NA - 512e (512 Emulation)
T17: C0:6 f1400005 00020 00 e8e088af 0 0 0 1 SEAGATE ST2000NX0273 NS28 0 0 5000c5008f3b0661 06 07 000f 1 NA NA - 512e (512 Emulation)
T17: C0:20 01400005 00020 0d 0 0 0 0 0 DP BP13G+ 2.23 0 0 524180704c645200 00 08 0009 0 NA NA
T17: C0:100 00400005 00020 03 0 0 0 0 0 LSI SMP/SGPIO/SEP 4402 0 0 0 00 ff ffff 0 NA NA
CONCLUSION AND LESSONS LEARNED
It is obvious that VMware will support 4K disks sometimes in the future because industry is moving there but if you are planning to use directly attached disks choose disks with 512 sector. It is ESXi limitation at the moment. VMware VSAN is also impacted by this limitation because VSAN relies on ESXi.
Update 2016-01-21:
I have just received following question from one reader ...
"How can I list physical disks connected to internal PERC8?"Unfortunately I don't have access to any 13G Dell server but I did a research and there should be three available methods.
Method 1/ racadm
If you have DRAC (Dell Remote Access/management Card) you can leverage racadm.
Based on racadm documentation it should be possible
- Storage.PhysicalDisk.
BlockSizeInBytes (Read Only) Description This is readonly attribute. This property indicates the logical block size of the physical drive that this virtual disk belongs to. Legal Values Values: 512 or 4096
http://de.community.dell.com/techcenter/support-services/w/wiki/369.export-perc-raid-controller-log-with-dell-support-live-image-version-2-0-englisch
Method 3/ perccli
Dell has utility called perccli. You can check perccli documentation for all details but there is command for viewing physical drive details for the specified slot in the controller.
- Syntax is perccli /c0/e32/s4 show all
Downside of this method is that perccli binaries exist just for windows or linux so you cannot use it directly from ESXi and you have to boot for example Linux live CD.
Method 4/ megacli (not supported)
Third method is leveraging LSI megacli utility. Dell PERC is manufactured by LSI so it should work. LSI has megacli VIB for ESXi but it is not officially supported by VMware nor Dell.
See details at http://de.community.dell.com/
5 comments:
Method 5 - install dell open manage system administrator on esxi.
Hi, if I have configured RAID 0 in my PERC instead of pass-through? Since physical disk is under a Logical Unit, can I by pass the sector size or not? Why 4k disks under External Storages rejects about non 512-sectors problem?
Hi Wagner.
Unfortunately not. As far as I know, RAID controllers does not allow to change disk sectors for volumes composed from physical disks and exposed to OS.
BTW, I had the same idea because sector size can be usually changed on shared storage arrays but this is not the case for local RAID controllers. I did a research for Dell PERC and there was not such possibility.
External storages typically use 512B sectors and it can be usually changed per volume to 4K. External storages doing much better volume virtualization then local RAID controllers.
This is crazy, any idea as to when native 4k support will be available?
Sorry, but this is not publically available information. For such kind of information signed NDA is required.
Publically available information is here
FAQ: Support statement for 512e and 4K Native drives for VMware vSphere and vSAN (2091600)
https://kb.vmware.com/s/article/2091600
Post a Comment