Machine Learning Basic – Calculate Euclidean Distance by PowerShell

The core of Machine Learning is to find out rules in a set of data. One basic operation of Machine Learning is “Cluster”. Or simply call it “classify data”. For example, there are 1000 records of toy sales data. It will be useful for proactive new incoming customer’s behavior if we can classify the data to multiple groups (Such as “buy stuffed toys” group and “buy electronic toys” group).

So we need to leverage partition methods to classify the sales data. There are multiple ways to do so. One simple method call “K-Means“. It calculates the distance between each data point and centroids ( Center point of a group ). And then assign data points to the closest centroids. Wikipedia has a detail description of the method.

Hence, as you can see, the key to “K-Means” is to calculate distance. There are several ways of calculation. “Euclidean Distance” is one way. Please refer to Wikipedia for deep dive. Long to short, you need to distribute data to a 2D axis. Each data point has x and y value. “Euclidean Distance” between two data points is:

The formulation is simple, but you have to calculate the distance between each data point to every centroids. Following is a super simple PowerShell code to help calculate Euclidean Distance of a 3 clustered data.

$K1 = (3.67, 9)
$K2 = (7, 4.33)
$K3 = (1.5, 3.5)

$input = Import-Csv "c:\temp\input.txt"

$i = 1
foreach ($seed in $input){
    $K1Result= [math]::Sqrt([math]::pow(($K1[0]-$seed.x),2)+[math]::pow(($K1[1]-$seed.y),2))
    $K2Result= [math]::Sqrt([math]::pow(($K2[0]-$seed.x),2)+[math]::pow(($K2[1]-$seed.y),2))
    $K3Result= [math]::Sqrt([math]::pow(($K3[0]-$seed.x),2)+[math]::pow(($K3[1]-$seed.y),2))
    Write-Host "K1 to  A$i distance is $K1Result"
    Write-Host "K2 to  A$i distance is $K2Result"
    Write-Host "K3 to  A$i distance is $K3Result"
    $i++
}
PowerShell

This script assumes you want to partition records to a set of 3 clusters (K1, K2, K3). $K1, $K2, and $K3 are centroids of each cluster (group). You can adjust it according to your purpose.

The script loads records in “input.txt” file then calculates Euclidean Distance of each record. Each record in “input.txt” only has x and y value. Following is a sample of “input.txt“. You can copy it for testing.

x,y
2, 10
2, 5
8, 4
5, 8
7, 5
6, 4
1, 2
4, 9

Following is the result:

K1 to A1 distance is 1.9465096968677
K2 to A1 distance is 7.55968914704831
K3 to A1 distance is 6.51920240520265
K1 to A2 distance is 4.33461647669087
K2 to A2 distance is 5.04469027790607
K3 to A2 distance is 1.58113883008419
K1 to A3 distance is 6.61429512495474
K2 to A3 distance is 1.05304320899002
K3 to A3 distance is 6.51920240520265
K1 to A4 distance is 1.66400120192264
K2 to A4 distance is 4.17958131874474
K3 to A4 distance is 5.70087712549569
K1 to A5 distance is 5.20469979921993
K2 to A5 distance is 0.67
K3 to A5 distance is 5.70087712549569
K1 to A6 distance is 5.5162396612185
K2 to A6 distance is 1.05304320899002
K3 to A6 distance is 4.52769256906871
K1 to A7 distance is 7.49192231673554
K2 to A7 distance is 6.43652856748108
K3 to A7 distance is 1.58113883008419
K1 to A8 distance is 0.33
K2 to A8 distance is 5.55057654663009
K3 to A8 distance is 6.04152298679729

“The terminal process terminated with exit code: 1” in Visual Studio Code when open PowerShell file

When you open a PowerShell file in Visual Studio Code, you may see following error:

The terminal process terminated with exit code: 1

The issue usually occurred on new provisioned system or enterprise environment with restricted security policy. The reason and solution are same like my other post: “Timed out waiting for the PowerShell extension to start” in Visual Studio Code.

Authentication failed when clone git repository on Windows for Bitbucket

I wrote a post talk about how to install Git and integrate with Visual Studio Code for Bitbucket server. Today, I got following message when I cloned a new repository. The reason was incorrect password.

fatal: Authentication failed for ‘https://bb.zhengwu.org/vmware.git/’

Time needed: 10 minutes.

Following is express solution for authentication failed for git repository clone.

  1. Open “Credential Manager” on Windows

    a. Click Start button
    b. Type “Credential Manager” and open it
    c. Click “Windows Credentials“.

  2. Change password for Git repository

    a. Click your Git repository in the list
    b. Click “Edit” to change credential.

Disable Default Untitled Tab in Visual Studio

A new tab with name “Untitled-1” is opened by default when you run Visual Studio if you closed all files last time run Visual Studio. It’s not big deal but annoying. Following is how to disable the default untitled tab in Visual Studio

  1. Click “Manage” button on lower left corner of Visual Studio.
  2. Search keyword “Untitled“.
  3. You will find a option call “Workbench: Startup Editor“.
  4. It’s “newUntitiledFile” by default.
  5. Change it to “none“.

Now. “Untitled-1” tab goes away when you launch Visual Studio next time.

Validating Connection Result of Connect-VIServer

Validating connection result of Connect-VIServer is trick since VMware doesn’t use the standard way of throwing errors.

There are different error messages for different scenarios. We can leverage “Try Catch Finally” of PowerShell to testing connection result and output customized errors.

The essential format of testing vCenter connection is below. PowerShell cannot catch error message if you don’t add “-ErrorAction Stop” when connect vCenter.

try{
    connect-viserver vcenter-xxxx -ErrorAction Stop
}
catch [VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin]{
    Write-Host "Permission issue"
}
catch [VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViServerConnectionException]{
    Write-Host "Cannot connect to vCenter Server"
}
catch
    {Write-Host "Other issue"}
finally{
    write-host "the end."
}

How to know error types of the validating connection result?

You probably have question how did I know which error types are for what? I used a very simple way that just give it a try. 🙂

# Simulating connection error
Connect-VIServer vCenter.zhengwu.org -User "xxx" -Password "yyy"

# Format output of $Error. $Error stores output of errors in PowerShell.
$error[0] | fl * -force

# You will see similar lines below
writeErrorStream      : True
PSMessageDetails      :
Exception             : VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin: 7/29/2019 9:27:30 AM   Connect-VIServer                Cannot complete login due to an
                        incorrect user name or password.         ---> VMware.Vim.VimException: Cannot complete login due to an incorrect user name or password. --->
                        System.ServiceModel.FaultException`1[VimApi_67.InvalidLogin]: Cannot complete login due to an incorrect user name or password.

VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin is the error type we want to find.

Conclusion

This is very simple way to validating connection result of Connect-VIServer. Looks like the result errors coming from different assemblies. For example, connection error is VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViServerConnectionException. It’s hard to find out all possible errors. I’m looking for a official document but no lucky. Please let me know if you know where I can reference.

Show CDP Neighbor of Cisco UCS Uplinks

There are two ways to know which network switch ports the network uplinks of Cisco UCS Fabric Interconnects are connected to.

By CLI

  • SSH to the Cisco UCS Manager.
  • Connect to FI-A.
# connect nxos a
  • Show neighbor of network uplinks.
# show cdp neighbor interface ethernet <port num>

By PowerShell

  • Make sure Cisco PowerTool (For UCS Manager) is installed.
  • Enabling the Information Policy via UCSM GUI.
    • Go to “Equipment” -> “Policies” tab -> “Global Policies” tab -> “Info Policy” area.
    • Change to “Enabled“. (No impact to running blades)
  • Open a PowerShell window.
  • Connect to the UCS Manager.
# Connect-Ucs <UCS FQDN>
  • Show CDP neighbor details.
# Get-UcsNetworkLanNeighborEntry

Side notes

Following command can shows network switch name, network switch ports and FI ports

# Get-UcsNetworkLanNeighborEntry | Select deviceid,remoteinterface,localinterface

If you prefer to enable the “Info Policy” by PowerShell, run following command

# Get-UcsTopInfoPolicy | Set-UcsTopInfoPolicy -State enabled -Force

How To Find Non-tagged ESXi Hosts

There are plenty of scripts to find tagged ESXi hosts. But what if you want to find out all ESXi hosts not be tagged? Following is a simple script:

Compare-Object ((Get-VMHost | Get-TagAssignment).Entity | select -uniq) (Get-VMHost)

The output is similar like following:

InputObject      SideIndicator
-----------      -------------
esx1         =>  
esx2         =>  
esx3         =>

The => indicates ESXi hosts in InputObject are not tagged.

If return is nothing, it means all ESXi hosts are tagged.

Please refer to “Using the Compare-Object Cmdlet” for detail.

Automatic vSphere Capacity Report in PPT

Reporting is important to management. To be a IT Pro, you may need to run regular reports for management. Some reports may be generated time consume. vRealize Operations Manager is an alternative to create customized reports. It’s a powerful product to organize data and create PDF or CSV files on scheduled intervals. I recommend have a look if you have planned to implement performance, capacity and alarm system for virtual environment.

What if budget is constrained? Is there a way to create such kind of reports? The answer is “Yes”. I worked out an automatic workflow to create the reports. I will not provide step-by-step guide in this post since it’s advanced integration of multiple products, everyone may have different way to do that. You can even create everything by script if you have strong programming skill. I’m not, I only look for the easiest way to achieve the goal.

Here is a scenario for  example: I want to run a monthly report for vSphere CPU and memory count and present to management by PowerPoint. I want to show management the historical trend of CPU and memory data. The traditional way is collect data in vCenter, organize and create charts in PowerPoint slides. So the whole workflow is: vCenter -> PowerPoint

If you want to automate the whole process you need to introduce few things more: PowerCLI, CSV and Excel. You need to develop a PowerCLI script to grab CPU and memory data on vCenter Server, then export the data to a CSV table by PowerShell command export-csv. Then import the table to an Excel file by Office feature Query Data. It loads the CSV table dynamically, you can even specific what data can be queried by filter.

Once the table is present in Excel, you need to create a chart accordingly. It’s trick when you paste the chart to PowerPoint Slide. You need to use Paste Special to paste the chart as Microsoft Excel Chart Object. The pasted chart can be updated automatically when you open the PowerPoint file.

The last step is created a scheduled task to run the PowerCLI script. Make sure you read my blog Extremely slow when run PowerShell script by scheduled tasks before create the task.

You can also configure the Excel file to automatically update table by CSV file.

How to Integrate PowerCLI 6.5 with PowerShell and PowerShell ISE

I wrote an article to introducing how to integrate PowerCLI with PowerShell and PowerShell ISE. VMware just released PowerCLI 6.5 R1, it includes lot  of new features and modules. And somehow my way doesn’t work. Following is new way to integrate PowerCLI 6.5 with PowerShell and PowerShell ISE in Windows 10.

PowerShell and PowerShell ISE both have it own $profile. So we need to do two times.

Before we start

Please make sure your PowerShell execution policy is not restricted. You can get the setting by run  following command:

Get-ExecutionPolicy

PowerShell Integration

  1. Open PowerShell window. Run following command to confirm profile file is not existing.
    Test-Path $profile

    If return is ‘False’, go to step 2.
    If return is ‘True’, Backup the file and go to step 3.

  2. Run following command if the  profile file  doesn’t existing.
    New-Item -Path $profile -type file -force | Out-Null
    
    Test-Path $profile

    The return above should be ‘True’. Profile file is created.

  3. Run following command to include VMware PowerCLI modules in  PowerShell.
    Add-Content -Path $profile -value "# Load Windows PowerShell cmdlets for managing vSphere `r`n. 'C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1'"

    The blue text above maybe different in your environment base upon where your PowerCLI is installed.

PowerShell ISE Integration

PowerShell ISE process is same to PowerShell, only different is all the operation should be completed in PowerShell ISE window.

Reboot is not required in my environment. But anyhow please reboot if you see any issue.

The processes above integrate PowerCLI 6.5 with PowerCLI and PowerCLI ISE for  current user only. If you want to integrate for all users on the machine, you need to refer to this article.