Connect-NsxtServer shows “Unable to connect to the remote server”

When you run Connect-NsxtServer in the PowerCLI, it may show “Unable to connect to the remote server“.

Because the error message is a little bit confusing with other login issues. It’s not easy to troubleshoot. The actual reason is the NSX-T uses a self-signed certificate, and the PowerCLI cannot accept the certificate automatically.

The fix is super easy. You need to set the PowerCLI to ignore the invalid certificate with the following command:

Set-PowerCLIConfiguration -Scope User -InvalidCertificateAction:Ignore -Confirm:$false

Move Terraform Providers to Other Folders

Create a new control file with the name .terraformrc or terraform.rc in your profile folder.

Add the following lines:

plugin_cache_dir   = "$HOME/.terraform.d/plugin-cache"

Create the folder .terraform.d/plugin-cache in your profile folder.

The providers will be downloaded to the cache folder when you run terraform init.


If you don’t want to create the control file in the profile folder. Alternative is to create an environment variable.

export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache"

Setup Terraform and Ansible for Windows provisionon CentOS

black server racks on a room

Provisioning Windows machines with Terraform is easy. Configuring Windows machines with Ansible is also not complex. However, it’s a little bit challenging to combine them. The following steps are some ideas about handling a Windows machine from provisioning to post configuration without modifying the winrm configuration on the guest operating system.

  1. Install required repos for yum.
yum -y install https://repo.ius.io/ius-release-el7.rpm
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
yum -y install epel-release
yum -y install yum-utils
yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
  1. Install Terraform.
sudo yum -y install terraform
  1. Install Ansible.
sudo yum -y install ansible
  1. Install Kerberos.
yum -y install gcc python-devel krb5-devel krb5-libs krb5-workstation
  1. Install pip.
sudo yum -y install python-pip

# You probably need the following packages if you are using VPN
pip install pysocks
  1. Install pywinrm[kerberos].
pip install pywinrm[kerberos]
  1. Configure /etc/krb5.conf.
    The following are the required lines. Please make sure to change the domain name to yours. And it’s case-sensitive.
[libdefaults]
 dns_lookup_realm = true
 dns_lookup_kdc = true
 forward = true
 forwardable = true
 default_realm = ZHENGWU.ORG


[realms]
 ZHENGWU.ORG = {
  kdc = DC.ZHENGWU.ORG
  admin_server = DC.ZHENGWU.ORG
 }

[domain_realm]
 .zhengwu.org = ZHENGWU.ORG
 zhengwu.org = ZHENGWU.ORG
  1. Create an Ansible inventory file.

[win] #Group name
dc.zhengwu.org #This is the target server list
 

[win:vars]
ansible_connection=winrm 

ansible_user=administrator #It's better a domain admin account.
ansible_password=P@ssw0rd #Change this password
ansible_port=5985
ansible_winrm_transport=kerberos
ansible_winrm_server_cert_validation=ignore
  1. Run Ansible win_ping test.
ansible <group in inventory file> -m win_ping -i <inventory file>

How to Manage Windows Servers With Ansible on CentOS 8

Ansible is a popular automation tool for infrastructure configuration. It runs on the Linux system. CentOS is an ideal distribution to run Ansible for lab purposes. It is similar to the Red Hat Linux but free. And the latest major release is CentOS 8. It contains Python 3 by default. So the Ansible configuration is different from CentOS 7. I will focus on the configuration in the lab environment. The goal is to create a simple environment to manage Windows servers with Ansible.

Ansible Installation on CentOS 8

I used CentOS 8 mini installation. It has no extra software installed. The procedure below maybe a bit different from your environment if you installed other roles on the OS.

Ansible is a standalone application that not rely on databases. There are two files it mainly needed in a quick lab environment: Playbook and host files. You can install multiple Ansible servers. They can run independently to control the same group of Windows servers.

I would suggest you take a snapshot before moving forward if your Ansible will running on a virtual machine.

  1. Enable Extra Packages for Enterprise Linux for yum.
yum install epel-release
  1. Install Ansible
yum install ansible
  1. (Optional) Install pip for Python 3. This step is for Red Hat 8.
yum install python3-pip
  1. Install pywinrm. The pywinrm will be used to communicate to Windows servers via winrm.
pip3 install pywinrm
  1. Install dependencies for pywinrm to use Kerberos in order to authenticate to Active Directory.
yum install gcc python3-devel krb5-devel krb5-libs krb5-workstation
pip3 install pywinrm[kerberos]

Ansible installation is completed. The procedure is elementary level but I spent some time figuring it out. Especially the Kerberos and pywinrm parts. 🙂

Please go to pywinrm GitHub if you want to dig into it.

Ansible Configuration on CentOS 8

Configure Ansible

As I mentioned in the previous section. There are two main files: Playbook and host. A Playbook is a file consist of multiple tasks that will run on the target Windows servers. It’s not covered by this article. The host file stores variables, and target server FQDNs or IP addresses. Ansible gets the target servers’ information in the host file when you run a playbook.

The host file location is /etc/ansible/hosts. There are two sections in the file for lab purpose.

  1. Server group. You can have multiple groups. Group name is in [ ]. You can give FQDN or IP addresses of the target Windows servers. I recommend using FQDNs if your targets are domain member servers. My example uses server win2019test1.zhengwu.org.
[windows]
win2019test1.zhengwu.org
  1. Variables of the target server group. Since this is for lab purpose. I’ll just list required variables in the /etc/ansible/hosts file. You need to use standalone variable files and avoid to input password if it’s for production. Following is a sample of the variable set for the windows group.
    • Variables are linked to a group by the variable name in the first line: [group name:vars].
    • The domain name should be uppercase in ansible_user. The reason is krb5 requires the uppercase domain name in the configuration file. We should match the name here. The domain name is not required if you use a local account.
    • ansible_winrm_server_cert_validation is optional. It only useful when ansible_winrm_scheme is ‘https‘.
    • ansible_port is ‘5985‘ when ansible_winrm_scheme is ‘http‘. Or ‘5986‘ when ansible_winrm_scheme is ‘https‘.
    • ansible_winrm_transport is ‘kerberos‘ in this example since the target Windows servers are domain members. It can be ‘ntlm‘ if you want to authenticate by local account. There are 5 authentication methods on Windows. Kerberos and NTLM are enabled by default. Please refer to Windows Remote Management for detail.
[windows:vars]
ansible_user='administrator@ZHENGWU.ORG'
ansible_password='123321'
ansible_connection='winrm'
ansible_winrm_scheme='http'
ansible_port='5985'
ansible_winrm_transport='kerberos'
ansible_winrm_server_cert_validation='ignore'

Configure Kerberos

Apart from Ansible configuration. We should configure Kerberos for domain authentication if the target Windows servers are joined a domain. My lab servers are joined domain ‘zhengwu.org‘. We have installed Kerberos components in the Ansible Installation on CentOS 8 section. So we just need to configure it. Edit Kerberos configuration file: /etc/krb5.conf.

  1. Change the default domain name. Make sure to remove # to uncomment the line. The domain name should be uppercase.
default_realm = ZHENGWU.ORG
  1. Uncomment all lines in the realms section. Please note domain name should be uppercase. The parameters kdc and admin_server are the same for the lab environment. The following is an example:

[realms]
ZHENGWU.ORG = {
     kdc = DC.ZHENGWU.ORG
     admin_server = DC.ZHENGWU.ORG
 }

Please refer to MIT Kerberos Documentation for the explanation.

Now Kerberos is configured. We have configured domain credentials in Ansible configuration file, specified Kerberos as the authentication method, and configured Kerberos for Active Directory. We just need to run the Windows ping module in Ansible to test the connection to target Windows servers. You should complete section Manage Windows servers with Ansible if the testing is failed.

ansible windows -m win_ping

You should see following output if authentication is successfully.

win2019test1.zhengwu.org | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Kerberos troubleshooting

You may see authentication problem when validating target Windows server connection by Ansible win_ping module. Here is simple steps to troubleshooting Kerberos authentication

  1. Try authenticate to target Windows servers by domain account on Ansible server. It can be any domain account.
kinit administrator@ZHENGWU.ORG
  1. List cached authentication data. You should see something similar below.
Ticket cache: KCM:0
Default principal: administrator@ZHENGWU.ORG
Valid starting       Expires              Service principal
06/26/2020 03:56:12  06/26/2020 13:56:12  krbtgt/ZHENGWU.ORG@ZHENGWU.ORG
        renew until 07/03/2020 03:56:09

Manage Windows servers with Ansible

The target Windows servers should be configured to accept the winrm connection. Ansible provides a PowerShell script to configure target Windows servers automatically. The script should not be used in a production environment according to Ansible stated in their document.

The configuration is super easy for production. Open a command prompt under the administrator permission and then run following command

winrm quickconfig

Conclusion

Manage Windows servers with Ansible is not so hard as long as the authentication is configured correctly. Ansible is not the only tool for automation. I’m a super fan of PowerShell. I have posted some articles for automation you may want to check. PowerShell and Ansible both are automation tools.

I think manage Windows server with Ansible is like outsourcing PowerShell scripting works to communities. You give inputs to the tasks then Ansible modules will execute pre-defined PowerShell scripts and feedbacks output. Ansible reduces the development time of Windows automation but it still has some disadvantages. Such as you have to run multiple tasks to enable Remote Desktop on target Windows servers which is just a single task in PowerShell DSC. So I think automation of infrastructure is a combination of tools like Swiss Army Muti-Tools, each one has an advantage. We have to use them together to achieve the final goal of automation.

Access Deny When Run PowerShell Scripts

You may get access deny when modify particular section of Windows Server. Such as some registry keys or system directories.

The reason is Windows Server protects sensitive part of operating system. This is similar like running command without root permission on Linux. You have to run as administrator to work around access deny problem.

I faced this issue when run guest command on Embotics Commander workflow. Looks like there is no official document talk about this issue. The workaround is disable UAC on Windows Server. Following are some helpful references.

Please refer to Disabling User Account Control (UAC) on Windows Server to understanding impact of disable UAC.

There are plenty of articles on internet talk about how to disable UAC.

There are two steps:

  • Disable UAC notification in Control Panel.
  • Change value of key EnableLUA from 1 to 0 in registry path HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionpoliciessystem.

You may need to reboot the server, or wait few minutes.

Validation is you should see a reminder message when run command.

Get SSD Hard Disk Information by PowerShell to HPE Servers

Summary

HPE published an advisor for SSD issue recently. The issue impacts most popular Proliant servers in the world. The remediation is upgrading firmware. Unfortunately HPE doesn’t have a product can easy report hard disk model for Gen9 and earlier models. I have tried HPE OneView and OneView Global Dashboard. However, we can get SSD hard disk information by PowerShell through API.

Solution

Following procedure helps you get SSD hard disk information in large environment.

  1. Make sure you have same credential available on iLO of Proliant servers. It can be local or domain credentials.
  2. Prepare a Windows 2016 or Windows 10 computer with latest patch and internet.
  3. Install HPEiLOCmdlet by following PowerShell command
Install-Module -Name HPEiLOCmdlets
  1. Connecting to HPE iLO.
$Conn = Connect-HPEiLO -IP xxx -User xxx -Password xxx -DisableCertificateAuthentication
  1. Retrieving HPE Smart Array Storage Controller information.
$HardDisks = Get-HPEiLOSmartArrayStorageController -Connection $Conn
  1. Run following command to get physical disk information.
HardDisks.Controllers.PhysicalDrives

Conclusion

PowerShell API is much flexible to get any information of hardware. The solution above is core part. Of course you can leverage ForEach-Object to do some automation report to export to CSV file. PowerShell is not only method, you can also get SSD hard disk information by other API.

Machine Learning Basic – Calculate Euclidean Distance by PowerShell

The core of Machine Learning is to find out rules in a set of data. One basic operation of Machine Learning is “Cluster”. Or simply call it “classify data”. For example, there are 1000 records of toy sales data. It will be useful for proactive new incoming customer’s behavior if we can classify the data to multiple groups (Such as “buy stuffed toys” group and “buy electronic toys” group).

So we need to leverage partition methods to classify the sales data. There are multiple ways to do so. One simple method call “K-Means“. It calculates the distance between each data point and centroids ( Center point of a group ). And then assign data points to the closest centroids. Wikipedia has a detail description of the method.

Hence, as you can see, the key to “K-Means” is to calculate distance. There are several ways of calculation. “Euclidean Distance” is one way. Please refer to Wikipedia for deep dive. Long to short, you need to distribute data to a 2D axis. Each data point has x and y value. “Euclidean Distance” between two data points is:

The formulation is simple, but you have to calculate the distance between each data point to every centroids. Following is a super simple PowerShell code to help calculate Euclidean Distance of a 3 clustered data.

$K1 = (3.67, 9)
$K2 = (7, 4.33)
$K3 = (1.5, 3.5)

$input = Import-Csv "c:tempinput.txt"

$i = 1
foreach ($seed in $input){
    $K1Result= [math]::Sqrt([math]::pow(($K1[0]-$seed.x),2)+[math]::pow(($K1[1]-$seed.y),2))
    $K2Result= [math]::Sqrt([math]::pow(($K2[0]-$seed.x),2)+[math]::pow(($K2[1]-$seed.y),2))
    $K3Result= [math]::Sqrt([math]::pow(($K3[0]-$seed.x),2)+[math]::pow(($K3[1]-$seed.y),2))
    Write-Host "K1 to  A$i distance is $K1Result"
    Write-Host "K2 to  A$i distance is $K2Result"
    Write-Host "K3 to  A$i distance is $K3Result"
    $i++
}
PowerShell

This script assumes you want to partition records to a set of 3 clusters (K1, K2, K3). $K1, $K2, and $K3 are centroids of each cluster (group). You can adjust it according to your purpose.

The script loads records in “input.txt” file then calculates Euclidean Distance of each record. Each record in “input.txt” only has x and y value. Following is a sample of “input.txt“. You can copy it for testing.

x,y
2, 10
2, 5
8, 4
5, 8
7, 5
6, 4
1, 2
4, 9

Following is the result:

K1 to A1 distance is 1.9465096968677
K2 to A1 distance is 7.55968914704831
K3 to A1 distance is 6.51920240520265
K1 to A2 distance is 4.33461647669087
K2 to A2 distance is 5.04469027790607
K3 to A2 distance is 1.58113883008419
K1 to A3 distance is 6.61429512495474
K2 to A3 distance is 1.05304320899002
K3 to A3 distance is 6.51920240520265
K1 to A4 distance is 1.66400120192264
K2 to A4 distance is 4.17958131874474
K3 to A4 distance is 5.70087712549569
K1 to A5 distance is 5.20469979921993
K2 to A5 distance is 0.67
K3 to A5 distance is 5.70087712549569
K1 to A6 distance is 5.5162396612185
K2 to A6 distance is 1.05304320899002
K3 to A6 distance is 4.52769256906871
K1 to A7 distance is 7.49192231673554
K2 to A7 distance is 6.43652856748108
K3 to A7 distance is 1.58113883008419
K1 to A8 distance is 0.33
K2 to A8 distance is 5.55057654663009
K3 to A8 distance is 6.04152298679729

“The terminal process terminated with exit code: 1” in Visual Studio Code when open PowerShell file

When you open a PowerShell file in Visual Studio Code, you may see following error:

The terminal process terminated with exit code: 1

The issue usually occurred on new provisioned system or enterprise environment with restricted security policy. The reason and solution are same like my other post: “Timed out waiting for the PowerShell extension to start” in Visual Studio Code.

Authentication failed when clone git repository on Windows for Bitbucket

I wrote a post talk about how to install Git and integrate with Visual Studio Code for Bitbucket server. Today, I got following message when I cloned a new repository. The reason was incorrect password.

fatal: Authentication failed for ‘https://bb.zhengwu.org/vmware.git/’

Time needed: 10 minutes

Following is express solution for authentication failed for git repository clone.

  1. Open “Credential Manager” on Windows

    a. Click Start button
    b. Type “Credential Manager” and open it
    c. Click “Windows Credentials“.

  2. Change password for Git repository

    a. Click your Git repository in the list
    b. Click “Edit” to change credential.

Disable Default Untitled Tab in Visual Studio

A new tab with name “Untitled-1” is opened by default when you run Visual Studio if you closed all files last time run Visual Studio. It’s not big deal but annoying. Following is how to disable the default untitled tab in Visual Studio

  1. Click “Manage” button on lower left corner of Visual Studio.
  2. Search keyword “Untitled“.
  3. You will find a option call “Workbench: Startup Editor“.
  4. It’s “newUntitiledFile” by default.
  5. Change it to “none“.

Now. “Untitled-1” tab goes away when you launch Visual Studio next time.