Inventory Service Cannot be Brought Up

One day, my vCenter Server suddenly lost search. It popped me “Unable to connect to web services to execute query. Verify that the ‘VMware VirtualCenter Management Webservices’ service is running on https://vCenter_Server_FQDN:10443” when I did object search on vSphere Client. Few hours later people starting complaint they got error on vSphere Web Client, it show “Client is not authenticated to VMware Inventory Service – https://Inventory_Service_FQDN:10443“.

[do_widget “Language Switcher” wrap=aside title=false]

The weird thing was the issue got fixed automatically after one or two days. It struggled me a while until I did more research. Initially reset inventory service database helped, but later we cannot bring the service up after we upgraded vCenter Server to 5.5 update 2. The service hung on starting status. I noticed the free space of the drive stores Inventory Service was very low. After discussed with VMware support, we thought the issue was caused by the low space.

vCenter Inventory Service didn’t fully bring up when first time the free space low, it leaded to VPX_PROPERTY_BULLETIN table in vCenter Server database was corrupted. Following restart got stuck due to that.

We eventually found a KB described same issue. Somehow it’s a internal KB in English but a public in Japanese and Chinese. (Thanks god I know Chinese! 🙂 ). Here I post another public KB about the table. Searching for virtual machines in the vSphere Client and vSphere Web Client return the error: Search returned no results

In my case, basically it need to stop inventory service, reset the inventory service database, truncate the table, then restart vCenter services.

If you don’t know how to reset inventory service please refer this KB.

Note:

  • This article only apply to vCenter Server 5.5.
  • When you do Inventory Service database reset, please don’t restart the service after reset the database since you need to do table truncate before startup the service.