Sunday, October 02, 2016

Leveraging VMware LogInsight for VM hardware inventory

There is no doubt that VMware LogInsight is a must for any properly managed vSphere environment. I'm explaining LogInsight benefits to all my customers. The main use case for LogInsight is troubleshooting but there are infinite number of other use cases where LogInsight can help.

During last LogInsight presentation to one of my customers I have got an interesting question if LogInsight can be used for change management. Change management is very broad term but all VM hardware changes should be in log files therefore there is potential to prepare specific interactive analytics to track VM hardware changes.

You can very simply find reconfiguration tasks for particular VMs logged by vCenter or ESXi hosts.

In ESXi logs reconfiguration log entries looks like

 2016-10-01T21:01:27.126Z esx01.home.uw.cz Hostd: info hostd[31C40B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/d22648f3-b9848863/test_cmdb/test_cmdb.vmx opID=7c3fe7f0-976b-4afa-a182-6d9f074a6446-45173-ngc-25-78-fc98 user=vpxuser:UW.CZ\cdave] Send config update invoked  

And in vCenter tasks it looks like
 2016-10-01 21:01:27.434 vc01.home.uw.cz vcenter-server: Reconfigured test_cmdb on esx01.home.uw.cz in home  

The problem is that even we know there were some reconfigurations, there are no VM reconfiguration details like CPU, RAM, disks and NICs. This is the current state in vSphere 6.0 U2 and it would be great if logging of such details would be improved in future vSphere version but this is what it is at the moment.

However, lot of VM hardware details should be in vmware.log which is located in home directory of each VM. The problem is that these logs are placed on VMFS filesystem and not forwarded to syslog so LogInsight cannot be used for log analysis.

Fortunately, there is the blog of the best VMware virtualization blogger William Lam and more specifically his blog post "A Hidden vSphere 5.1 Gem – Forwarding Virtual Machine Logs (vmware.log) to Syslog Part 1". William very nicely explains how to redirect VM logs also to syslog.

You have to change following two advanced settings for each Virtual Machine you would like to redirect VM hardware logs to syslog server.
vmx.log.destination=syslog-and-disk 
vmx.log.syslogID=[YOUR-VM-NAME-OR-OTHER-ID]
And when VM hardware logs are available in LogInsight you can prepare custom analytics to visualize VM hardware changes over time and put it on to dashboard for future use. You can see my dashboard on screenshot below.

VMware LogInsight dashboard - VM Hardware Inventory 

I have exported my VM Hardware Inventory dashboard as LogInsight content pack and shared it here on github so you can download and import it in to your LogInsight and test it.

Content Pack source code from GitHub is embedded below ...
[ Download Content Pack from GitHub ]

And as always, any comments are more then welcome.

No comments: