Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1037959

SG3_UTILS: How to send SCSI commands to devices

http://sg.danny.cz/sg/sg3_utils.html
http://linux.die.net/man/8/sg3_utils

The sg3_utils package contains utilities that send SCSI commands to devices. As well as devices on transports traditionally associated with SCSI (e.g. Fibre Channel (FCP), Serial Attached SCSI (SAS) and the SCSI Parallel Interface(SPI)) many other devices use SCSI command sets.

How the Cluster service reserves a disk and brings a disk online

http://support.microsoft.com/kb/309186

This article (link above) describes how the Microsoft Cluster service reserves and brings online disks that are managed by cluster service and related drivers.

PuppetLabs | Razor: Next-Generation Provisioning

System administrators require the same agility and productivity from their hardware infrastructure that they get from the cloud. In response, Puppet Labs and EMC collaboratively developed Razor, a next-generation physical and virtual hardware provisioning solution. Razor provides you with unique capabilities for managing your hardware infrastructure, including:

• Auto-Discovered Real-Time Inventory Data
• Dynamic Image Selection
• Model-Based Provisioning
• Open APIs and Plug-in Architecture
• Metal-to-Cloud Application Lifecycle Management

Together, Razor and Puppet enable system administrators to automate every phase of the IT infrastructure lifecycle, from bare metal to fully deployed cloud applications.

More at ... https://puppetlabs.com/solutions/next-generation-provisioning/

Automated Storage Tiering - Sub-LUN tiering

Excellent comparisons between Automated Storage Tiering technologies of different vendors.

• http://searchstorage.techtarget.com/feature/Sub-LUN-tiering-Five-key-questions-to-consider
• http://www.computerweekly.com/feature/Automated-storage-tiering-product-comparison
• http://searchsolidstatestorage.techtarget.com/news/1378753/Vendors-take-different-approaches-to-automated-tiered-storage-software-for-solid-state-drives
• http://www.emc.com/collateral/software/white-papers/h8058-fast-vp-unified-storage-wp.pdf

I personally believe automated storage tiering (AST) is really important for dynamic virtualized datacenter and because AST differs among vendors I'm going to collect important information for design considerations.  I don't want to preferred or offend against any product. Each product has some advantages and disadvantages and we as infrastructure architects has to fully and deeply understand technology to be able prepare good design which is the most important factor for reliable and well performed infrastructure.

Good mid-range storage products on the market (my personal opinion):

• DELL Compellent
• Hitachi HUS
• EMC VNX

DELL Compellent
Tiers: SSD, SAS, NL-SAS (SATA)
AST Sub-LUN tiering block: 512kb, 2MB (default), 4MB
Tiering optimisation analysis period: [TBD]
Tiering optimisation relocation period: [TBD]
Tiering algorithm: [TBD]
QoS per LUN: no

Hitachi HUS (HUS 110, HUS 130, HUS 150)
Tiers: SSD, SAS, NL-SAS (SATA)
AST Sub-LUN tiering block: 32MB
Tiering optimisation analysis period: 30 minutes
Tiering optimisation relocation period: [TBD]
Tiering algorithm: [TBD]
QoS per LUN: no

EMC VNX 
Tiers: SSD, SAS, NL-SAS (SATA)
AST Sub-LUN tiering block: 1GB
Tiering optimisation analysis period: 60 minutes
Tiering optimisation relocation period: user defined
Tiering algorithm:

During user-defined relocation window, 1GB slice ae promoted according to both the rank ordering performed in the analysis stage and a tiering policy set by the user. During relocation, FAST VP relocates higher-priority slices to higher tiers; slices are relocated to lower tiers only if the space they occupy is required for a higher-priority slice. This way, FAST VP fully utilized the highest-performing spindles first. Lower-tier spindles are utilized as capacity demand grows. Relocation can be initiated manually or by a user configurable, automated scheduler. The relocation process targets to create 10% free capacity in the highest tiers in the pool. Free capacity in these tiers is used for new slice allocations of high priority LUNs between relocations.

QoS per LUN: yes


I've collected information from several public resources so if there is some wrong information please let me know directly or via comments.

Understand SCSI, SCSI command responses and sense codes

During troubleshooting VMware vSphere and storage related issues it is quite useful to understand SCSI command responses and sense codes.

Usually you can see in log something like "failed H:0x8 D:0x0 P:0x0 Possible sense data: 0xA 0xB 0xC"

H: means host codes
D: means device codes
P: means plugin codes
A: is Sense Key
B: is Additional Sense Code
C: is Additional Sense Code Qualifier

Some host codes:
0x2 Bus state busy

0x3 Timeout for other reason

0x5 Told to abort for some other reason

0x8 Bus reset

Some device codes:
00h  GOOD
02h  CHECK CONDITION
04h  CONDITION MET
08h  BUSY
18h  RESERVATION CONFLICT
28h  TASK SET FULL
30h  ACA ACTIVE
40h  TASK ABORTED

Some plugin codes:
00h  No error.
01h  An unspecified error occurred. Note: The I/O cmd should be tried.
02h  The device is a deactivated snapshot. Note: The I/O cmd failed because the device is a deactivated snapshot and so the LUN is read-only.
03h  SCSI-2 reservation was lost.
04h  The plug-in wants to requeue the I/O back. Note: The I/O will be retried.
05h  The test and set data in the ATS request returned false for equality.
06h  Allocating more thin provision space. Device server is in the process of allocating more space in the backing pool for a thin provisioned LUN.
07h  Thin provisioning soft-limit exceeded.
08h  Backing pool for thin provisioned LUN is out of space.

Some SCSI Sense Keys:
SCSI Sense Keys appear in the Sense Data available when a command returns with a CHECK CONDITION status. The sense key contains all the information necessary to understand why the command has failed.

Code Name
0h   NO SENSE
1h   RECOVERED ERROR
2h   NOT READY
3h   MEDIUM ERROR
4h   HARDWARE ERROR
5h   ILLEGAL REQUEST
6h   UNIT ATTENTION
7h   DATA PROTECT
8h   BLANK CHECK
9h   VENDOR SPECIFIC
Ah   COPY ABORTED
Bh   ABORTED COMMAND
Dh   VOLUME OVERFLOW
Eh   MISCOMPARE

There is VMware KB with further details here.

It is worth to read following documents
http://www.tldp.org/LDP/khg/HyperNews/get/devices/scsi.html (this is quite old document for programmers willing to write SCSI driver)
http://en.wikipedia.org/wiki/SCSI
http://en.wikipedia.org/wiki/SCSI_contingent_allegiance_condition
http://en.wikipedia.org/wiki/SCSI_Request_Sense_Command

What is SCSI reservation
http://mrwhatis.com/scsi-reservation.html

SCSI-3 Persistent Group Reservation
http://scsi3pr.blogspot.cz/

Using the VMware I/O Analyzer v1.5: A Guide to Testing Multiple Workloads

I encourage you to watch great video about good practice how to use VMware I/O Analyzer (VMware bundle of IOmeter).

There is mentioned very important step to get relevant results. The step is to increase the size of second disk in virtual machine (OVF appliance). Default size is 4GB which is not enough because it hits the cache of almost any storage array and results are unreal and misleading.

Video is here
bit.ly/118kWs1 
or here
http://www.youtube.com/watch?v=zHJr957kN1s&feature=youtu.be

Enjoy.