Log Browser and ESXi RAM disk full

With the vSphere Web Client it is possible to view, search and export vCenter Server and ESXi log files using the Log Browser.

log_browser

When you use the log browser to retrieve the log files of an ESXi host, the log bundle is stored on the ESXi host in /tmp/scratch/downloads. The location /tmp is a symbolic link to the RAM disk created by ESXi to store temporary files.

You can view the log bundle and the size of it using the ‘ls ‘ command when logged in to the ESXi shell.

ssh_ls_tmp_location

Because the RAM disk is not very big it might be possible that the Log Browser cannot retrieve the log files of the ESXi host. If the RAM disk is full you will receive an error that no support bundle could be generated.

no_support_bundle

This error doesn’t show you why the log bundle couldn’t be created but the events tab of the ESXi host does.

esxi_events_ram_disk_full

You can view the RAM disk usage using the ‘vdf’ command when logged in to the ESXi shell.

ssh_vdf_ram_disk_full

It is possible to remove previously created log bundles using the log browser  but the failed log bundle will still be on the RAM disk. You can remove this log bundle using the ‘rm’ command when logged in to the ESXi shell.

ssh_failed_log_bundle

 

 

 

 

ESXi 6.x CBT issue

VMware has released a KB article regarding a new issue affecting ESXi 6.0 and 6.0 Update 1. This issue affects backups utilizing change block tracking (CBT).

When running incremental virtual machine backups, backup applications typically rely on the vSphere API call QueryDiskChangedAreas() to determine the changed sectors.

This issue occurs due to a problem with CBT in the disklib area, which results in the change tracking information of I/Os that occur during snapshot consolidation to be lost. The main backup payload data is never lost and it is always written to the backend device. However, the corresponding change tracking information entries which occur during the consolidation task are missed. Subsequent QueryDiskChangedAreas() calls do not include these missed blocks and, therefore a backup based on this CBT data is inconsistent.

All incremental backups which utilize CBT are potentially affected.

VMware has identified the root cause of this issue and will release a patch to fix this issue. A release date of this patch is not available. Until then there are some “workarounds”.

  • Downgrade the affected ESXi hosts to version 5.5, and downgrade the virtual Hardware Version from 11 to 10, if necessary.
  • Shutdown the virtual machine before doing an incremental backup.
  • Do a full virtual machine backup rather than an incremental backup

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2136854&sliceId=1&docTypeID=DT_KB_1_1&dialogID=840649925&stateId=0%200%20840663565

Update: VMware have released a patch to fix this issue, http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2137546&utm_content=bufferac1da&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

 

 

Bye Bye vSphere C# Client

Since the introduction of the Web Client new features to vSphere would be available only through the Web Client, with the most well known example of VSAN. The performance of the Web Client was not great at all which made it a big obstacle for using the Web Client instead of the old vSphere C# Client. Because of the availability of new features only in the Web Client, for me the choice to use the Web Client was forced but over time I learned to use the Web Client in a usable and relative fast way.

With the introduction of vSphere 6 the vSphere Web Client has been improved in terms of usability and performance. Working with the Web Client is now on par with the old vSphere C# Client.

The continuous improvement of the Web Client makes it really nice to work with and it only gets better. Improvements such as the VSAN Health Plugin and fully functional vSphere Update Manager integration makes the Web Client the tool for managing your vSphere environment.

vsan_health_plugin

update_manager

The only thing that was bothering me was that I still needed to use the vSphere C# Client when I wanted to manage a vSphere host directly. Luckily, a fling has been released which provides a HTML 5 web GUI directly on the host. The ESXi Embedded Host Client is a VIB that needs to be installed on the host. After installation the host can be managed through your browser. I like it very much, almost all tasks can be performed and the developers are always looking for feedback to improve it. Input is appreciated and if possible integrated in new releases of the Embbed Host Client. An example of this is the offline bundle used for Update Manager and Auto Deploy configurations.

https://labs.vmware.com/flings/esxi-embedded-host-client

embedded_host_client

I can only say one thing, and that is to use the vSphere Web Client and Embedded Host Client!

Disclaimer: I haven’t used the Web Client bundled with vCenter Server 5.5 Update 3 which according to the release notes should also be on par with the vSphere C# Client on performance.

http://pubs.vmware.com/Release_Notes/en/vsphere/55/vsphere-vcenter-server-55u3-release-notes.html