Wu's Blog

My IT Adventure

Tag: virtualization

  • Fixing “Keyset Does Not Exist” When Adding a Node to a Windows Server Failover Cluster

    Recently, I ran into a strange issue while trying to add newly built Windows Server 2022 hosts to an existing Microsoft Failover Cluster. Every attempt to add a node failed with the following error message:

    The server ‘host.zhengwu.org’ could not be added to the cluster.
    An error occurred while adding node ‘host.zhengwu.org’ to cluster ‘cluster.zhengwu.org’.

    Keyset does not exist

    At first glance, this error didn’t provide much direction. However, after some investigation, I discovered that the problem was caused by a mismatch in the node status between Microsoft Failover Cluster Manager and System Center Virtual Machine Manager (SCVMM).

    Root Cause: Node Status Out of Sync Between SCVMM and Failover Cluster

    One of the existing cluster nodes was still marked as Maintenance Mode in SCVMM, even though Failover Cluster Manager showed it as Online.

    This mismatch caused several issues:

    • The node appeared online in Failover Cluster Manager
    • Virtual machines were running on it normally
    • But SCVMM showed no VMs on the host
    • And SCVMM treated the host as if it were not fully operational

    Because SCVMM maintains cluster metadata and permissions, the inconsistent state led to the “Keyset does not exist” error when adding a new node.

    The following Windows Event log is also observed in Microsoft-Windows-FailoverClustering-Client/Diagnostic:

    AddClusterNodeEx (4570): Exception – error -2146893802 while adding node xxxx: Failed to Get Cluster Secret via CPrep Server on node xxxx

    Solution: Rejoin the Problematic Cluster Node to the Cluster

    To resolve the issue, I simply shut down the problematic cluster node.

    1. Go to the Failover Cluster Manager.
    2. Go to Nodes.
    3. Select the problematic node.
    4. Click More Actions in the right-side panel.
    5. Select Evict.
    6. I suggest reboot the node after evicting.
    7. Add the node back to the cluster after rebooting.

  • Convert Virtual Machine of VirtualBox to ESXi

    My coworker want to build a virtual machine on ESXi, but vendor only support virtual appliance of Oracle VirtualBox format. VMware has a KB article to show how to “Importing Virtual Machine from Oracle VirtualBox to VMware Fusion, Workstation, or Player (2053864)“. It’s working fine. But it’s not applicable for VMware ESXi.

    If you follow the guide to export .ova file and import to ESXi. It will show error below on ESXi 6.0 or later:

    Issues detected with selected template. Details….No supported hardware versions among….

    After couple of hours’ deep dive. I figured out a way to convert VirtualBox to ESXi. You need Oracle VirtualBox, VMware Workstation or VMware Player and VMware ESXi host.

    1. Select the virtual machine -> Go to main menu -> File -> Export appliance.
    2. Choose the virtual machine.
    3. Make sure Format is “Open virtualization format 1.0“.
    4. Export to a .ova file.
    5. Open the .ova file in VMware Workstation or VMware Player.
    6. The import of the VM maybe failed with following error. Just click Retry button it will work.
      The import failed because xxxxx did not pass OVF specification conformance or virtual hardware compliance checks.
    7. Select the virtual machine and go to main menu -> File -> Export to OVF.
    8. VMware Workstation or VMware Player generates .ovf, .mf and .vmdk files.
    9. Edit .ovf file and find the line with keyword “VirtualSystemType“.
    10. Change the value “vmx-XX” to the version lower or equal to your ESXi version.
    11. Edit .mf file and remove SHA256 value of .ovf file in first line.
      SHA256(XXXXX.ovf)= xxxxxxxxxxxxxxxxx
    12. Now it’s ready to import to VMware ESXi host.

    Conclusion

    This procedure is not involve any code or command. There are also couple of other ways to convert VirtualBox to ESXi by ovftool command line. I tried several ways but didn’t work. Maybe I did something wrong.

    In step 10, I changed VM version in .ovf file directly. I think you can also leverage VMware Workstation or VMware Player to downgrade the virtual machine’s version in GUI. It should work as long as the version is lower than your ESXi supported VM version.

  • Cannot Open KVM Virtual Machine Manager on CentOS 7

    I got following error message when I try to run KVM Virtual Machine Manager: virt-manager on SSH.

    Gtk-WARNING **: cannot open display:

    There are several things need to be checked:

    • Make sure “X11Forwarding” is set to “yes” in /etc/ssh/sshd_config on the machine you run virt-manager. 
      cat /etc/ssh/sshd_config | grep "^X11"
    • If you are using Windows to connecting SSH. The X11 need to be forwarded to an “X Window server” on top of Windows. I use xming.
    • If you connect SSH by Putty on Windows. Please configure X11 forwarding.
      • Go to “Connection” -> “SSH” -> “X11“.
      • Check “Enable X11 forwarding“.
      • Assign xming.exe path in “X authority file for local display“.
    • If you are using terminal on Mac OS. You need to install Xquartz. It configures terminal automatically.

    Now you are ready to use “virt-manager“.