vSphere Web Client stuck on the loading screen

a blank sticky note stuck on a lilac surface

It’s been a while since my last post. I got an exciting issue a few days ago. I was trying to log in to a vCenter Server in Chrome. I can see the login screen and enter the credential. However, I was not able to get into the main page. vSphere Web Client was stuck on the “loading.”

The reason is the Chrome version was 50. It’s incompatible with the vSphere Web Client version.

Here are the vSphere Client requirements for your convenience.

This operation is restricted by the administrator – ‘vpxd.stats.maxQueryMetrics’

You may see vpxd.stats.maxQueryMetrics relevant error when retrieving performance data of ESXi hosts, virtual machines, or other entities on vCenter Server by PowerCLI. The full error message:

This operation is restricted by the administrator – ‘vpxd.stats.maxQueryMetrics’. Contact your system administrator

VMware published a KB about how to fix that. Please notice that the mentioned vCenter advanced parameter config.vpxd.stats.maxQueryMetrics is case-sensitive.

202203 Update:

The parameter is removed in vCenter Advanced Settings after upgrading to vCenter Server 7.x. You have to reconfigure it.

vCenter is not sending email alerts

It’s useful to get email notification when there is an alarm raised in vCenter Server. To configure the function. You need a valid SMTP server. And configure the SMTP server on vCenter Server. If there is a problem that vCenter Server is not sending email alerts. We need to isolate is it a vCenter Server side issue, or an SMTP server problem. Therefore how to identify problem source is important.

VMware moved vCenter Server to a customized Linux based platform since 5.5. It’s easy for administration, but limited ability in troubleshooting. You may notice that the telnet command is not available when troubleshooting the SMTP issue.

The following is how to troubleshoot such an issue on the vCenter Server virtual appliance.

Firstly, you need to ping the IP address and FQDN of the SMTP server. Make sure the basic connectivity between the vCenter Server and SMTP server is working.

Second, try to reach out to SMTP service on the vCenter Server to make sure the SMTP protocol can be established. Since the telnet command is unavailable on the vCenter Server virtual appliance. We use curl command as a workaround:

curl -v telnet://SMTP SERVER:SMTP PORT

For example:

curl -v telnet://zhengwu.org:25

You should see the keyword “Connected to zhengwu.org….” if the connection is successful. That means the basic connectivity between the vCenter Server and the SMTP server is working.

Then you can try sending an email manually on the vCenter Server shell. The following are the commands. Please note there is a . at the end of the command. It indicates the end of the mail.

MAIL FROM:sender@zhengwu.org
RCPT TO:Reception@zhengwu.org
DATA
Subject: This is a testing mail
This is testing content
.

If SMTP is working fine. You should see following message:

250 2.0.0 xxxxxx Message accepted for delivery

If all of these testing are passed. Then it maybe indicate it’s a SMTP server issue.

Following are some articles for your reference:

Emails are not sent from vCenter Server Appliance 6.5 or 6.7 after an Alert/Event is triggered.

Diagnosing why VMware vCenter Server is not sending email alerts

How to Find Out Source of Domain Accounts Locking on vCenter Server

I wrote an article talk about how to find out which services lockout domain accounts on vCenter Server. It only applies to the scenarios that domain accounts very frequently lockout. Like every 1 second. If it’s minutes, it will be hard to find out as it’s manually processing.

The other way to identify source is to use vSphere Web Client. The trick was told by VMware BCS team.

Log in to vSphere Web Client. Go to the main page and the Events node. Search for “authen” you may see some error events. The real source is red text.

User CONTOSOtest-user@192.168.1.1 .....

 

在VMware Workstation上部署vCenter Server VCSA

网上有很多关于如何在VMware Workstation上部署vCenter Server VCSA的文章,但根据这些文章在部署过程中总是会遇到各种各样。以下是几点我总结出来的要点,仅供参考。

我假设你的实验环境里没有DNS或者域服务器,只是简单的使用VMware Workstation的DHCP服务,虚拟机的网卡选择的是“host-only”。以下步骤仅用于做一些快速测试时使用。

  1. vCenter Server安装好后第一次启动的时候会检测FQDN。如果你没有DNS服务器,FQDN检测会失败。所以在安装vCenter Server时要确保“Host Network Identity”输入的是IP地址。
  2. OVA文件导入后虚拟机会立刻自动启动,有时候虚拟机的网卡可能会是断开状态的。要确保网卡是连接状态。
  3. 第一次启动耗时大约15至20分钟,在没有完全启动完毕前虚拟机的控制台界面是不现实IP地址的。另外一个vCenter Server准备就绪的表现是IP地址ping得通了。
  4. vCenter Server第一次启动后,需要打开 https://vcenter_ip:5480 继续完成vCenter Server的配置。
  5. Administrator@vsphere.local 的密码就是你在OVA导入界面里输入的密码。

2018 5月28日更新:

在以上步骤的第四步中,你可能无法登陆root,提示验证失败。这是由于root锁死造成的,需要按照以下步骤解锁:

  1. 重启vCenter Server虚拟机。
  2. 在Photon启动界面按“e”键。
  3. 在第二行结尾加入”rw init=/bin/bash“。具体参考这里
  4. 当你看到#提示符时,运行命令”passwd“更改root密码。
  5. 运行命令”pam_tally2 –user root“检查root密码输入错了多少次。
  6. 如果输入错误次数大于1,运行命令 “pam_tally2 –user root –reset” 解锁root账号。
  7. 重启虚拟机,现在应该可以登录了。

2018 5月31日更新:

在以上步骤的第四步中,登陆后你应该会看到vCenter Server安装向导。如果你的vCenter Server只想用IP地址,请确保“System name”项填写的是IP地址。

Deploye vCenter Server Virtual Appliance on VMware Workstation

There are a lot of articles introduce how to deploy vCenter Server virtual appliance on VMware Workstation. I tried but somehow it’s failed. Following are some notes for your reference if you want to deploy vCenter Server virtual appliance on VMware Workstation real quick.

I assume you don’t have DNS or domain servers. Native DHCP services of VMware Workstation is used. You just want to use vCenter Server for some quick testings purpose, and “host-only” NIC you want to select.

  1. vCenter Server installer validates FQDN when it’s first boot up. The process fails if FQDN doesn’t work. So please make sure “Host Network Identity” is IP address of the VM when you set the OVA options.
  2. The VM is immediately booted up after importing the OVA file. But VM NIC is “disconnected” status sometimes. You have to enable the NIC in VM properties real quick.
  3. You have to wait for about 15 – 20 minutes after first boot. Console screen doesn’t show IP address before it’s fully ready. The indicator of readiness is the IP address of the VM is responding to ping.
  4. Login https://vcenter_ip:5480 to continue vCenter Server installation after the first boot is ready.
  5. The password of Administrator@vsphere.local is same as you set during importing the OVA.

Updates 28th May 2018:

Root authentication on step 4 above maybe failed. It’s caused by root account locking. Please follow the procedures below:

  1. Reboot vCenter VM.
  2. Press “e” when you see the Photon booting screen.
  3. Add “rw init=/bin/bash” to the end of the 2nd line. Refer here for detail.
  4. Run “passwd” to change root password when you see # prompt.
  5. Run “pam_tally2 –user root” to check how many failures root hits.
  6. Run “pam_tally2 –user root –reset” to unlock root if you see more than 1 in step 5.
  7. Reboot. You should be able to login root now.

Updates 31st May 2018:

You should see the installation wizard in step 4. Please make sure “System name” field is IP address if you only want to use IP for vCenter Server.

Updates 5th Sep 2018:

You may see the following error during installation.

Could not connect to VMware Directory Service via LDAP

It indicates vCenter Server FQDN doesn’t work. If you’re a home lab, you may want to add the DNS entries in the hosts file.

The older version of cis-upgrade-runner cannot be removed when upgrade vCenter Server 6.0

When you upgrade or patch vCenter Server 6.0 for Windows, you may see following symptoms:

“The older version of cis-upgrade-runner cannot be removed. Contact your technical support group.”

Or error code 1063:

“Installation of component VMware CIS upgrade runner failed with error code ‘1063’”

That means the vCenter Server installer cannot find MSI files of existing vCenter Server services. It could be following reasons:

  • You delete MSI files in “Temp” folder of the profile you used to install vCenter Server.
  • The account you used to login and install vCenter Server was roaming profile. The profile’s “Temp” folder was automatically deleted when you reboot/logoff the server.

vCenter Server 6.0 for Windows is consist of lot of standalone package. The upgrading process usually uninstall old packages, and then install newer packages. So the failure doesn’t impact to database or inventory data. You can re-initiate the upgrading again.

But you cannot manually uninstall old package since upgrading process brings down vCenter services first then uninstall old packages. If you already uninstalled old packages, the upgrading process will be stuck on bring down vCenter Services stage since some processes may already be removed. For example “vmware-python” it maps to “VMware vCenter Configuration Service”. If you manually uninstalled it before launch upgrading. It removes the service. Upgrading is not able to check status of the service.

Easiest way to get ride of this problem is

  1. Open Registry Editor (regedit) and go to the path: HKEY_LOCAL_MACHINESOFTWAREClassesInstallerProducts”
    You would see lot of keys are there.
  2. Search keyword “vmware-“. These keys store package info of vCenter Server.
  3. Expand one of the keys. Go to “SourceList”.
  4. The value of “LastUsedSource” is path of MSI file of old vCenter Server installer.
    For example my value is “m;1;X:vcenter-serverpackages”.
  5. Make sure your server has the path mentioned in previous step (My case it’s X:vcenter-serverpackages) and old MSI files are available in the path. If it’s a CD-ROM letter, you just need mount old vCenter Server image to the drive.
  6. Copy new vCenter Server image to a local folder, uncompresse and launch installer locally.
  7. Now the upgrading process can read original packages on the mentioned path in step 4. It will automatically remove old packages by the old MSI files.

There are two other workarounds. One is modify the value of “LastUsedSource” to reflect a new location of packages. But you still need the old MSI files be there. Another way is delete the key after you find it in step 2. (I never tested this way but it should work as it let vCenter Server installer thinks the server is brand new so installer can override the existing folders)

I also wrote another article for upgrading error on vCenter Server 5.5 for your reference:

CustomAction VM_InstallJRE returned actual error code 1624

 

Adobe Flash Player Out of Date on vSphere Web Client

You may see ‘Adobe Flash Player Out of Date’ on Chrome when you open vSphere Web Client. Click the text Chrome will update Flash Player automatically. But in some cases it doesn’t work due to maybe your Chrome is controlled by company policy or internet problem to Adobe.com. I found an article to show how to offline fix the issue. You can download Flash Player for Opera and Chromium-based browsers – PPAPI in official Adobe KB article.

You may also want to check out my other articles about Flash issue on browsers.

Flash menu appears when right click on vSphere Web Client in Chrome

Cannot open vSphere Web Client on IE11 on Windows 8.1

Flash menu appears when right click on vSphere Web Client in Chrome

There is a KB describes how to fix right click issue in IE on vSphere Web Client. But my problem was in Chrome. I searched a lot in internet but no lucky till today.

The problem was flash menu appears when I right click anything in vSphere Web Client in Chrome. I have two computers that both has Chrome installed but one has issue, other one works fine. I compared version of Chrome, noticed working one was 55.x, problematic one was 49.x. The issue gone after upgraded to 57.x.

After dig into that problem, looks like Google fixed the problem on version 54.0.2840 that there was a bug related to right click. Check out release notes here.

 

“No host data available” Reported in Hardware Status Tab

Just noticed a issue that nothing reported in ‘Hardware Status‘ tab of ESXi hosts in vSphere Web Client. KB 2112847 gives a solution but not works for me. The feature can be used to monitor hardware failures. I figured out a way to workaround it. You just need to login by Administrator account and click ‘Update‘ button under ‘Monitor‘ – ‘Hardware Status‘ for each ESXi host. You will get the status after few minutes.