Understanding vSAN Architecture Components for better troubleshooting

VMware vSAN becomes more and more popular, thus more often used as primary storage in data centers and server rooms. Sometimes, as with any IT technology, is necessary to do the troubleshooting. Understanding of architecture and components interactions is essential for effective troubleshooting of vSAN. Over years, I have collected some vSAN architectural information into a slide deck I made available at https://www.slideshare.net/davidpasek/vsan-architecture-components

In the slide deck are the slides with the following sections ...

vSAN Terminology

• CMMDS - Cluster Monitoring, Membership, and Directory Service
• CLOMD - Cluster Level Object Manager Daemon
• OSFSD - Object Storage File System Daemon
• CLOM - Cluster Level Object Manager
• OSFS - Object Storage File System
• RDT - Reliable Datagram Transport
• VSANVP - Virtual SAN Vendor Provider
• SPBM - Storage Policy-Based Management
• UUID - Universally unique identifier
• SSD - Solid-State Drive
• MD - Magnetic disk
• VSA - Virtual Storage Appliance
• RVC - Ruby vSphere Console

Architecture components

• CMMDS
• Cluster Monitoring, Membership, and Directory Service
• CLOM
• Cluster Level Object Manager Daemon
• DOM
• Distributed Object Manager
• Each object in a vSAN cluster has a DOM owner and a DOM client
• LSOM
• Local Log Structured Object Manager
• LSOM works with local disks
• RDT
• Reliable Datagram Transport

Components interaction

Architecture & I/O Flow

Troubleshooting tools

• RVC
• vsan.observer
• vsan.disks_info
• vsan.disks_stats
• vsan.disk_object_info
• vsan.cmmds_find
• ESXCLI
• esxcli vsan debug disk list
• Objects tools
• /usr/lib/vmware/osfs/bin/objtool

How to use vSAN Observer

• SSH somewhere where you have RVC. It can be for example VCSA or HCIbench
• ssh root@[IP-ADDRESS-OF-VCSA]
• Run RVC command-line interface and connect to your vCenter where you have vSphere cluster with vSAN service enabled. RVC requires the password of the administrator in your vSphere domain. 
• rvc administrator@[IP-ADDRESS-OF-VCSA]
• Start vSAN Observer on your vSphere cluster with vSAN service enabled
• vsan.observer -r /localhost/[vDatacenter]/computers/[vSphere & vSAN Cluster]
• Go to vSAN Observer web interface
• vSAN Observer is available at https://[IP-ADDRESS-OF-VCSA]:8010

Slide deck includes little more info so download it from https://www.slideshare.net/davidpasek/vsan-architecture-components

If you have to troubleshoot vSAN, I highly recommend to follow the process documented at "Troubleshooting vSAN Performance".

Hope it helps the broader VMware community.

If you know some other detail or troubleshooting tool, please leave a comment below this post.