Tag Archives: vCenter

Adobe Flash Player Out of Date on vSphere Web Client

You may see ‘Adobe Flash Player Out of Date’ on Chrome when you open vSphere Web Client. Click the text Chrome will update Flash Player automatically. But in some cases it doesn’t work due to maybe your Chrome is controlled by company policy or internet problem to Adobe.com. I found an article to show how to offline fix the issue. You can download Flash Player for Opera and Chromium-based browsers – PPAPI in official Adobe KB article.

You may also want to check out my other articles about Flash issue on browsers.

Flash menu appears when right click on vSphere Web Client in Chrome

Cannot open vSphere Web Client on IE11 on Windows 8.1

Advertisements

Flash menu appears when right click on vSphere Web Client in Chrome

There is a KB describes how to fix right click issue in IE on vSphere Web Client. But my problem was in Chrome. I searched a lot in internet but no lucky till today.

The problem was flash menu appears when I right click anything in vSphere Web Client in Chrome. I have two computers that both has Chrome installed but one has issue, other one works fine. I compared version of Chrome, noticed working one was 55.x, problematic one was 49.x. The issue gone after upgraded to 57.x.

After dig into that problem, looks like Google fixed the problem on version 54.0.2840 that there was a bug related to right click. Check out release notes here.

 

“No host data available” Reported in Hardware Status Tab

Just noticed a issue that nothing reported in ‘Hardware Status‘ tab of ESXi hosts in vSphere Web Client. KB 2112847 gives a solution but not works for me. The feature can be used to monitor hardware failures. I figured out a way to workaround it. You just need to login by Administrator account and click ‘Update‘ button under ‘Monitor‘ – ‘Hardware Status‘ for each ESXi host. You will get the status after few minutes.

Host Cannot Download Files From VMware vSphere Update Manager Patch Store

You may see following error when you scanning ESXi hosts by vCenter Update Manager.

Host cannot download files from VMware vSphere Update Manager patch store. Check the network connectivity and 
firewall setup, and check esxupdate logs for details.

You also see similar logs in /var/log/esxupdate.log.

[Errno -2] Name or service not known

The root cause could be following:

  1. ESXi host cannot resolve DNS name of vCenter Update Manager Server.
  2. One of the DNS servers incorrect if you set multiple DNS servers on ESXi host.

Migrate vCenter Server 5.5 Windows to 6.0 Virtual Appliance 

Virtual appliance is future of how VMware delivery their product to customers. It’s pain to migrate from vCenter Server Windows version to virtual appliance. The only way was build up new virtual appliance and move everything out of Windows vCenter Server. The challenge is you lost data if you have integrated vCenter Server with other VMware products, or using DVS.

VMware released vCenter Server Migration Tool after VMworld 2016. It gave me confidence to give it a try. I assume vCenter Server is embedded SSO. I did the migration 2 or 3 times, following is summary of my experience. The migration tool only support migrate vCenter Server 5.5 windows edition to vCenter Server 6.0 U2.

Prerequisites

  1. vCenter Server is more like core services today since lot of 3rd party software call vCenter API to interactive with VMs. You may have some products integrated with vCenter Server already. Please upgrade to vCenter 6 compatible version before migration.
  2. I suggest create a local account on source vCenter Server if your server is domain member. You can login back source vCenter by local account in case migration failed.
  3. vCenter Server Migration Tool applies temporary IP address on destination vCenter virtual appliance during migration. It’s used to communicate with source vCenter. Please register a temporary IP address for destination vCenter Server.
  4. A helper VM is required to run migration image. Please make sure you have a free Windows VM be ready to mount migration image.
  5. SQL database is exported to source vCenter Server if you want migrate performance and event data. So you need to make sure enough space on C: drive on source vCenter. The free space should be much bigger than vCenter database size.
  6. Of course you need a vCenter Server 6 license key since old key doesn’t support the version.
  7. Some cases show migration process stopped during export SQL database. That’s because memory of source vCenter is too small. Please make sure RAM of source vCenter should be equal or greater than destination vCenter Server.
  8. The other tricky is database table. You may see migration processes is completed but destination vCenter Server doesn’t come up, and no data actually imported. That’s because ‘checksum‘ column existing in table [dbo].[VMO_ResourceElementContent] in vCenter DB. You can run following SQL query to remove it before migration.
    alter table dbo.VMO_ResourceElementContent drop column checksum;

Procedure

The items above can be done anytime before the migration window. Following steps should be token during migration.

  1. You need to disable firewall and anti-virus software on old vCenter to avoid communication issue between Migration Assistant and new vCenter Server.
  2. To avoid any unstable, resource contention, or potential network connectivity lost issue, I suggest temporarily disable DRS and HA on source and destination cluster if they are virtual machine.
  3. Copy Migration Assistant from migration image to old vCenter.
  4. Take snapshot on old vCenter and backup database of old vCenter.
  5. Connect to console of source vCenter and run Migration Assistant.
  6. Mount vCenter Server 6 U2m image on helper VM. Launch vCenter migration. The migration process is straightforward. I wouldn’t introduce more here.

After Migration

Basically you need to revert all the temporary changes made before. Such as delete snapshot and DB backup, enable DRS and HA, and disable vNIC on source vCenter Server to avoid any human error.

Inventory Service无法启动

某日,vCenter Server突然无法搜索虚拟机了。在vSphere Client中搜索时会提示 Unable to connect to web services to execute query. Verify that the ‘VMware VirtualCenter Management Webservices’ service is running on https://vCenter_Server_FQDN:10443。没过几个小时用户就开始抱怨vSphere Web Client也出问题了,总是提示错误 Client is not authenticated to VMware Inventory Service – https://Inventory_Service_FQDN:10443

Continue reading

Inventory Service Cannot be Brought Up

One day, my vCenter Server suddenly lost search. It popped me “Unable to connect to web services to execute query. Verify that the ‘VMware VirtualCenter Management Webservices’ service is running on https://vCenter_Server_FQDN:10443” when I did object search on vSphere Client. Few hours later people starting complaint they got error on vSphere Web Client, it show “Client is not authenticated to VMware Inventory Service – https://Inventory_Service_FQDN:10443“.

Continue reading

Domain account locked out on vCenter Server

That’s a very small problem but it struggles you if you are enterprise datacenter administrator. As you may know the best practices to run application is by service account. But sometimes  you may testing applications by your own domain account and forget remove it.

Few days ago, my domain account locked out on domain controller. The audit report indicated it locked out by vCenter Server every 5 seconds. Then I logged in the vCenter Server, checked out Task SchedulerServicesTask Manager…etc. Nothing was running under my domain account. I stopped applications one by one on the vCenter Server and related plugin services. No help, I felt so frustrated!!!

Here is how I figured it out eventually.

  1. Download TCPView from Microsoft website.
  2. Run it on the vCenter Server.
  3. Sort by Local Address.
  4. See which foreign address is connecting the vCenter Server.

After the steps above I finally figured out that root cause was my VMware View LAB VM tried to authenticate on vCenter Server by my domain account and stored old password. I powered up the old VM few days ago.

这可能是一个很小的问题,但如果你是企业级数据中心管理员,这个问题可能会很困扰你。如你所知在日常使用中最好用Service Account来运行应用程序。但是有时候你可能和我一样需要用自己的域帐号做一些测试但之后又忘记删除了。

几天前,我的域帐号被域控制器锁定了。域报告显示我的帐号每5秒钟就会被vCenter服务器锁定一次。我在vCenter服务器上检查了任务管理器、服务、计划任务等等,并没有发现任何东西使用我的帐号。然后我将vCenter服务器上的所有服务、应用程序都停了,还是不行!

最终我找到了问题原因,以下是方法:

  1. 从微软网站下载TCPView
  2. 在vCenter服务器上运行。
  3. 选择以Local Address本地地址)排序。
  4. 查看连接到vCenter服务器的Foreign Address外部地址)。

最终原因是我几天前把一台旧的VM开机了,这台VM上是当时以我的域帐号安装的VMware View做测试用。

How to change password of vCenter Server service account

Many company use service account for vCenter Server database and services. To compliance with security policy, you may need to change password of vCenter Server at regular period. This is a way I used to change password:

1. Change the password of service account of vCenter Server and database in AD.
2. Change the password of Log-On As account of vCenter Server/Management Webservices in Services.
3. Run vpxd.exe -p command as administrator to change database password. ( It usually located on C:\Program Files\VMware\Infrastructure\VirtualCenter Server\ )
4. RDP vCenter Server by service account.
5. Open DSN of vCenter Server and click next button to save password to DSN.
6. Reboot vCenter Server.

Notes: You maybe able to logon vCenter Server if you just restart VC services, but you will face low performance to retrieve information of host, VM…etc.

Most company may has vCenter Update Manager together with vCenter Server, the password change of vCenter Update Manager service account is similar like vCenter Server.
1. Change the password of service account of vCenter Server and database in AD.
2. Change the password of Log-On As account of vCenter Update Manager in Services.
3. Run VMwareUpdateManagerUtility.exe as administrator. ( It’s usually on C:\Program Files (x86)\VMware\Infrastructure\Update Manager\ )
4. Input new database credential on Database Setting.
5. Re-register to vCenter Server by new credential.
6. Reboot vCenter Update Manager server.

You can also reference to http://kb.vmware.com/kb/1006482 and http://kb.vmware.com/kb/1034605.

vCenter Server Heartbeat 5.6 – Installation

I have to say you’ll not able to get what you anticipating if you follow VMware document. After referred few blogs and videos, I finally deployed the production in HA and DR mode both, it consumed a lot of time since I had to clone the VM from US to India over WAN. It’s pain, I’d like the share it to make sure you never fall in same situation.

If you don’t familiar with vCHB, please read vCenter Server Heartbeat 5.6 – Architecture.

Before install vCHB, you should know that:

  • Install vCenter Server and components on Primary Server, Secondary Server will be cloned.
  • vCenter Update Manager, vCenter Converter, ESXi Dump Collector, Syslog Collector are configured using Fully Qualified Domain Names (FQDN) rather than IP addresses.
  • Time Zone and time setting is correct.
  • Port 52267 and 57348 is enabled in firewall on both servers.
  • 2GB free memory available for vCenter Server Heartbeat.
  • Administrator right is required to install vCenter Server Heartbeat.
  • All vCenter Server components should functionally before install vCenter Server Heartbeat.
  • No * in SSO master password. ( I guess that’s a bug of 5.6U1, please refer to KB2034608 to reset master password )
  • vCenter Server FQDN is Primary Server computer name. ( It will be changed later )

Pre-configure before install vCHB:

  • Make sure Primary Server computer name is vCenter Server FQDN.
  • Change vCenter Server services to manually start up on Primary Server.
    VMware VirtualCenter Server
    VMware vSphere Profile-Drive Storage
    vCenter Inventory Service
    VMware VirtualCenter Management Webservices
  • Recovery system fingerprint encrypted file.
    Go to C:\Program Files\VMware\Infrastructure\SSOServer\utils
    Recovery footprint by following command:
    rsautil manage-secrets -a recover -m SSO Master Password
  • Power off Primary Server
  • Clone Primary Server to secondary site.
  • Disconnect vNICs on Secondary Server.
  • Power on both servers and set IP addresses.
    I use two vNICs on each server, one for Public Network, another for VMware Channel Network.
    Public Network contains two IP address, one for Management Network, another for Principle Network.
    Principle Network on both should be same if you deploy HA mode, otherwise they are different for DR mode.
  • Disable NETBIOS and DNS Register on each vNIC.
  • Leave domain and rename Secondary Server.
  • Reboot Secondary Server and connect vNICs.
  • Join Secondary Server back to domain and add proper AD groups to Administrator group.
    Note: You probably need to re-join domain twice to make sure AD synchronization correct, I got vCenter Server startup issue in initially deployment due to AD synchronization issue.
  • Create a share folder on reliable server that Primary and Secondary Server both can access.
  • Make sure configured IP addresses pingable from each server.
  • Bring up vCenter Server services on Primary Server.

Installation:

  • Select Install VMware vCenter Server Heartbeat to start installation.
  • Select Primary to install vCHB on Primary Server.
  • Accept agreement.
  • Apply license key.
  • Select LAN or WAN according to your architecture.
  • Select Secondary Server is Virtual option. ( I only tested that option )
  • Confirm installation path.
  • Select vNIC for VMware Channel network.
  • Enter VMware Channel IP addresses of Primary and Secondary Server.
    For HA mode, you could use non-routable or routable IP address.
    For DR mode, you must use routable IP addresses to make sure VMware Channel network can communicate each other over WAN.
  • Select vNIC for Public Network.
  • Enter IP addresses of Principal Network for both server.
    For HA mode, IP address should be same on both server.
    For DR mode, IP addresses should be different, you have to enter manually.
    Select the options accordingly.
  • If you select Different IP addresses in step above, you will need to enter a DNS update account of Windows. ( Refer to KB1008605 if you use BIND9 DNS instead of Windows DNS service )
  • Then configure Management Network. This network is used for RDP.
  • Rename computer name of both server. It looks like only rename Primary Server, no change for Secondary Server, but you don’t have to worry about that since we already renamed Secondary Server in early step.
  • Set client port, I used default.
  • Select components you want to protect and enter vCenter Login, this Login must have Administrator right on vCenter Server.
    Also input SSO master password, please note the SSO master password may different with SSO administrator password, please make sure you enter correct password.
  • Enter the share path you created earlier, this folder will store cluster configuration information for Secondary Server installation.
  • vCHB start checking system.
  • You will lost RDP connectivity for 10 seconds during installation due to Package Filter installation.

  • Once the installation complete, you can start on Secondary Server, just make sure you select Secondary.
    All other steps is similar like Primary Server.

After Installation:

  • Startup vCHB services on Secondary Server.
  • Open vCenter Server Heartbeat Management Console.
  • Add each node by Management Network.
  • Wait a while, you will see similar screen like following screenshot.

vCenter Server Heartbeat 5.6 – Architecture

I start to use VMware workstation since 2002 or earlier, my bad memory can’t recall it. That’s 1st generation of virtualization. If you look at today’s virtual world, we are on the way to “Matrix”! J Enterprise is virtualizing more and more server lead to vCenter Server becomes to a critical role. We have to prepare for any contingency. vCenter Server Heartbeat (vCHB) is a nice candidate for protecting vCenter Server. It provides your infrastructure ability to prevent downtime/outage of vCenter Server. To gearing up for implementation in production environment, I did some testing on my LAB, the product is nice, but the document is not ideal. I’d like to share my experience, this blog also referred to my project document, please let me know if you have any idea can help me make my document ideally. Thanks in advance.

vCHB is a cluster service like Microsoft Cluster Service or any other 3rd part cluster software. The benefit of this product is you don’t have to create the cluster on RDM and your ESXi maintenance operation would become much easier. You could deploy vCHB in HA or DR mode, I’ll focus on HA mode at this moment since I haven’t tested DR mode yet.

Server

My original LAB infrastructure contains one vCenter Server with remote SQL database server, data transmits over LAN. So my vCHB topology is one SQL database (I already have MSCS to protects SQL database server), two vCenter servers (Primary Server and Secondary Server).

vCHB uses Active-Passive for HA mode, Active Role runs protected applications, Passive Role receives changed data.

Primary Server – Original vCenter Server which I want to protect, it runs all vCenter components except outage happening.

Secondary Server – Another server of the pair, it’s Passive Role. Generally it receives change of Primary Server and takes over Active Role when outage happens.

In my LAB Active Role is Primary Server, and Passive Role is Secondary Server in most of time.

Networking

vCHB have two networks: Public Network and VMware Channel Network. You could use single NIC to run all networks or multiple NICs to separate them.

VMware Channel Network – vCHB monitors alive of each via VMware Channel Network and syncs changed data, it’s very important network.

Public Network – It contains two sub-networks: Principle Network and Management Network. Principle Network for vCHB cluster, Management Network for day-to-day operation.

Confuse? To simple it, I understand the networks like that:

VMware Channel Network – Can be private IP address or any IP address outside of the subnet of Public Network. It used for heartbeat and data transmitting.

Public Network Principle Network is IP address of Cluster DNS name, Management Network is IP address for RDP, they are in same routable subnet, but better in different prefix of IP address, please refer to KB 2004926.

Storage

No special storage requirement, but 2GB free space should be there where you want to install vCHB to. We also need a reliable share folder to store cluster data, I prefer to create share folder on a server other than vCHB servers since vCHB networks usually interrupt for few seconds during vCenter failover.

Okay, I’ll share how to install vCHB in next blog, this architecture for your reference: