Private IP Address Routes to L3 Subnet on Dual vNIC VM

It’s not easy for me to describe the issue in one line on the title. Let me give some background here. I have 2 set of VMs. Set 1 has VM A & VM B. Set 2 has VM C & VM D. Each VM has a vNIC configured with a private IP address. VM A and VM C also have another vNIC configured with an L3 (Routable) IP address. Each set’s private IP addresses are the same. To make sure no confusion I implemented a vRouter VM for each set. The vRouter is same as VM A or VM C, it has two vNICs. One is connected to L3 network, another is connected to the private network. This way can keep the private network traffic not going outside of the set. So the both set no disturb each other when I set same private IP addresses.

Diagram

Following are IP addresses I set for each VM:

  • VM A: 192.168.0.11
  • VM B: 192.168.0.12
  • VM C: 192.168.0.11
  • VM D: 192.168.0.12

The problem is I still can get ping responding on VM A to 192.168.0.12 when I turn off VM B. I expected to see the L2 traffic goes to it own vRouter and finds VM B is offline. But tracert command shows me the traffic goes from VM A’s L3 network to vRouter of the 2nd set, and then get the answer from VM D. Looks like the L2 ping package is broadcasting on L3 network.

The issue was fixed by enabling a feature on L3 network. It called “Enforce Subnet Check for IP Learning“. Cisco changed the name to “Limit IP Learning To Subnet“. It’s a VLAN level setting. It will not allow broadcasting the private Ip traffic on an L3 network. It forces private IP traffic to go to L2 network only.

Emulex OneConnect OCe10102 on ESXi 6.0

Please refer to following post for basic troubleshooting of Emulex OneConnect.

How to Install Proper Drivers for 3rd Party Network Adapter on ESXi 5.x

I have a box uses Emulex OneConnect OCe10102 network adapters. The adapter is quite old and Emulex brand card doesn’t support ESXi 6.0. I upgraded the server to ESXi 6.0 and the Emulex adapters lost.

In the initial troubleshooting, I noticed that the adapters are still visible in BIOS. So it should be some driver level issues. I checked VMware Compatibility Guide. The model OCe10102 doesn’t support by ESXi 6.0.

If you run the following command you will still be able to see the adapters in PCI list on ESXi.

[code language="perl"]
esxcli hardware pci list
[/code]

So it indicates the adapters are not visible in ESXi since the newer Emulex driver doesn’t contain the model of the adapter in ESXi 6.0 native driver.

Then I uninstalled the native Emulex driver for ESXi 6.0 by the following command and rebooted the ESXi host.

[code language="perl"]
esxcli software vib remove -n elxnet
[/code]

The adapters still not visible after rebooting since no any drivers for Emulex adapters. Then I downloaded the Emulex drivers for ESXi 5.5 on VMware website and uploaded the “offline” package in the zip file to /tmp directory of the host. Then installed the driver by the following command:

[code language="perl"]
esxcli software vib install -d "/tmp/xxxxx.zip"
[/code]

The adapters appeared after rebooting the host.

The older version of cis-upgrade-runner cannot be removed when upgrade vCenter Server 6.0

When you upgrade or patch vCenter Server 6.0 for Windows, you may see following symptoms:

“The older version of cis-upgrade-runner cannot be removed. Contact your technical support group.”

Or error code 1063:

“Installation of component VMware CIS upgrade runner failed with error code ‘1063’”

That means the vCenter Server installer cannot find MSI files of existing vCenter Server services. It could be following reasons:

  • You delete MSI files in “Temp” folder of the profile you used to install vCenter Server.
  • The account you used to login and install vCenter Server was roaming profile. The profile’s “Temp” folder was automatically deleted when you reboot/logoff the server.

vCenter Server 6.0 for Windows is consist of lot of standalone package. The upgrading process usually uninstall old packages, and then install newer packages. So the failure doesn’t impact to database or inventory data. You can re-initiate the upgrading again.

But you cannot manually uninstall old package since upgrading process brings down vCenter services first then uninstall old packages. If you already uninstalled old packages, the upgrading process will be stuck on bring down vCenter Services stage since some processes may already be removed. For example “vmware-python” it maps to “VMware vCenter Configuration Service”. If you manually uninstalled it before launch upgrading. It removes the service. Upgrading is not able to check status of the service.

Easiest way to get ride of this problem is

  1. Open Registry Editor (regedit) and go to the path: HKEY_LOCAL_MACHINESOFTWAREClassesInstallerProducts”
    You would see lot of keys are there.
  2. Search keyword “vmware-“. These keys store package info of vCenter Server.
  3. Expand one of the keys. Go to “SourceList”.
  4. The value of “LastUsedSource” is path of MSI file of old vCenter Server installer.
    For example my value is “m;1;X:vcenter-serverpackages”.
  5. Make sure your server has the path mentioned in previous step (My case it’s X:vcenter-serverpackages) and old MSI files are available in the path. If it’s a CD-ROM letter, you just need mount old vCenter Server image to the drive.
  6. Copy new vCenter Server image to a local folder, uncompresse and launch installer locally.
  7. Now the upgrading process can read original packages on the mentioned path in step 4. It will automatically remove old packages by the old MSI files.

There are two other workarounds. One is modify the value of “LastUsedSource” to reflect a new location of packages. But you still need the old MSI files be there. Another way is delete the key after you find it in step 2. (I never tested this way but it should work as it let vCenter Server installer thinks the server is brand new so installer can override the existing folders)

I also wrote another article for upgrading error on vCenter Server 5.5 for your reference:

CustomAction VM_InstallJRE returned actual error code 1624

 

VMworld 2016

本年度VMworld重回拉斯维加斯,相比旧金山各方面都要好很多,参会这几天有不少乐趣。技术层面,本年度vmworld主要聚焦于用户端统一管理、公有云、以及vmware自家的“超融合软件”。
第一天的主题演讲回顾了近十几年来传统IT、私有云、公有云的增长趋势,这里有个很有意思的地方。我一直以为IT的转变方式应该是从传统到私有,最终到公有的。但是从演讲看,私有云和公有云占有率同时保持增长,公有云甚至一直比私有云要高一些。但直到今年云的占有率依旧不敌传统IT架构,这还是在国际大公司接近80%虚拟化的现状下。演讲中还提出了一个观点——数字化商业。今天,企业都在追求创新、高效的商业,IT已经不像过去仅仅支持企业内部的信息系统,而是引导企业走向何方,IT更加紧密的和商业结合在一起,很难想象一家没有技术驱动的企业可以高效的与客户沟通和分析数据、快速的对市场做出反应、自动化的生产经营。第二天的主题演讲介绍了一些用户层面的新解决方案,由于本人不关注该层面,没有太多了解。

Continue reading “VMworld 2016”

My blog was award to vExpert 2014

Few days ago, VMware sent an email notification to me that my blog was award to vExpert 2014. I would thanks Sean’s reference and suggestion. I’m very glad to hear that news! It encourage me continue contribute to virtualization community, share my experience with people. Here is the link lists my name and Twitter account. http://blogs.vmware.com/vmtn/2014/10/vexpert-2014-q3-announcement.html.

几天前,VMware通知我获得了本年度vExpert的殊荣。我非常高兴我可以获得这项荣誉。非常感谢Sean的推荐和给我的建议,这项荣誉是我对虚拟化社区做出贡献的鼓励,我会继续将我的经验和大家分享!你可以点击这里查看到我的名字和我的Twitter帐号。

How to configure nested Hyper-V VM on VMware Workstation

First, I would like to recommend DELL M4800 for small home lab, I spent lot of time to looking for a solution for my lab, I need some hardware low noise, low weight, and it is better portable. I checked out HP mini server, Apple MAC mini, Mac book pro…etc. They are nice products to show best design of IT industry, but no one perfect. Finally I choose DELL M4800. The reason is RAM can be upgraded to 32GB. 3 SSD hard disks is supported, you can install one native SSD, one in CD-ROM slot, another msata SSD in WAP slot. SSD disk is must have for IT LAB, it can provide you more than 10K IOPS without significant performance degrade.

Okay, back to topic…if you want to testing Windows Server 2012 R2 Hyper-V on lab, you probably prefer install it on a VM of VMware Workstation. You have to follow up proper steps to make sure the Hyper-V functionally.

After you create Hyper-V VM:

  1. Keep VM power off status.
  2. Go to Settings of VM.
  3. Highlight Processors.
  4. Select Virtualize Intel VT-x/EPT or AMD-V/RVI option.
  5. Go to Options tab.
  6. Highlight General.
  7. Select Microsoft Windows.
  8. Select Hyper-V on drop-list.
  9. Power on.