Machine Learning Basic – Calculate Euclidean Distance by PowerShell

The core of Machine Learning is to find out rules in a set of data. One basic operation of Machine Learning is “Cluster”. Or simply call it “classify data”. For example, there are 1000 records of toy sales data. It will be useful for proactive new incoming customer’s behavior if we can classify the data to multiple groups (Such as “buy stuffed toys” group and “buy electronic toys” group).

So we need to leverage partition methods to classify the sales data. There are multiple ways to do so. One simple method call “K-Means“. It calculates the distance between each data point and centroids ( Center point of a group ). And then assign data points to the closest centroids. Wikipedia has a detail description of the method.

Hence, as you can see, the key to “K-Means” is to calculate distance. There are several ways of calculation. “Euclidean Distance” is one way. Please refer to Wikipedia for deep dive. Long to short, you need to distribute data to a 2D axis. Each data point has x and y value. “Euclidean Distance” between two data points is:

The formulation is simple, but you have to calculate the distance between each data point to every centroids. Following is a super simple PowerShell code to help calculate Euclidean Distance of a 3 clustered data.

$K1 = (3.67, 9)
$K2 = (7, 4.33)
$K3 = (1.5, 3.5)

$input = Import-Csv "c:\temp\input.txt"

$i = 1
foreach ($seed in $input){
    $K1Result= [math]::Sqrt([math]::pow(($K1[0]-$seed.x),2)+[math]::pow(($K1[1]-$seed.y),2))
    $K2Result= [math]::Sqrt([math]::pow(($K2[0]-$seed.x),2)+[math]::pow(($K2[1]-$seed.y),2))
    $K3Result= [math]::Sqrt([math]::pow(($K3[0]-$seed.x),2)+[math]::pow(($K3[1]-$seed.y),2))
    Write-Host "K1 to  A$i distance is $K1Result"
    Write-Host "K2 to  A$i distance is $K2Result"
    Write-Host "K3 to  A$i distance is $K3Result"
    $i++
}
PowerShell

This script assumes you want to partition records to a set of 3 clusters (K1, K2, K3). $K1, $K2, and $K3 are centroids of each cluster (group). You can adjust it according to your purpose.

The script loads records in “input.txt” file then calculates Euclidean Distance of each record. Each record in “input.txt” only has x and y value. Following is a sample of “input.txt“. You can copy it for testing.

x,y
2, 10
2, 5
8, 4
5, 8
7, 5
6, 4
1, 2
4, 9

Following is the result:

K1 to A1 distance is 1.9465096968677
K2 to A1 distance is 7.55968914704831
K3 to A1 distance is 6.51920240520265
K1 to A2 distance is 4.33461647669087
K2 to A2 distance is 5.04469027790607
K3 to A2 distance is 1.58113883008419
K1 to A3 distance is 6.61429512495474
K2 to A3 distance is 1.05304320899002
K3 to A3 distance is 6.51920240520265
K1 to A4 distance is 1.66400120192264
K2 to A4 distance is 4.17958131874474
K3 to A4 distance is 5.70087712549569
K1 to A5 distance is 5.20469979921993
K2 to A5 distance is 0.67
K3 to A5 distance is 5.70087712549569
K1 to A6 distance is 5.5162396612185
K2 to A6 distance is 1.05304320899002
K3 to A6 distance is 4.52769256906871
K1 to A7 distance is 7.49192231673554
K2 to A7 distance is 6.43652856748108
K3 to A7 distance is 1.58113883008419
K1 to A8 distance is 0.33
K2 to A8 distance is 5.55057654663009
K3 to A8 distance is 6.04152298679729

Agentless Management with PowerShell for DELL iDRAC

DELL provides agentless management of iDRAC by PowerShell. It’s more convenience for enterprise level infrastructure compare with HPE Scripting Tools or Cisco PowerTools. You don’t have to install any extra software to retrieve hardware data from iDRAC

DELL provides an article “Agentless Management with PowerShell 3.0 – CIM Cmdlets and iDRAC/Lifecycle Controller” with a sample code which is very detail. The scripts leverage public repos on internet: schemas.dell.com.

But what if your infrastructure is offline or internet is blocked?

After few hours dig into it. I found a workaround works in my environment.

  1. Go to DCIM library profile to find profile of what your iDRAC. (I use DCIM System Info Profile for example)
  2. The document tells what’s the ” Implementation Namespace” and “Central Class”.
  3. Follow sample code in DELL document but when you connect CIM session use following format:
Get-CimInstance -CimSession $session -ClassName <class name> -Namespace <Name space name>

In my sample, the command is:

Get-CimInstance -CimSession $session -ClassName DCIM_SystemView -Namespace "root/dcim"

I have some posts talk about PowerShell. Here is the article to get space utilization of Windows Servers.

Bitbucket Server Integration With Visual Studio Code on Windows

There is an official Bitbucket extension for Visual Studio Code if you use Bitbucket Cloud. Bitbucket has not yet released official extension for Bitbucket Server users. Following is how to configure Visual Studio Code to use Bitbucket Server.

Before the procedure you need to collect the following information:

  • Your user name on Bitbucket Server.
  • Your email on Bitbucket Server.
  • Decide the local path to store code. (It’s c:\temp\git\ in this guide)
  • Your account has permission to modify repositories on Bitbucket Server.
  • Assume project name is “ExampleProject” on Bitbucket Server.
  • Assume repository name in the project is “ExampleRepository“.
  • Get the URL of the target repository. (It’s https://userid@bb.zhengwu.org/scm/exampleproject/examplerepository.git in this guide)
  • Make sure you have the latest Visual Studio Code installed.

Time needed: 30 minutes.

  1. Download and install Git.

    Download windows installer in Github.
    Installation is simple. Only thing is to make sure to choose “Visual Studio Code” when it asks editor integration.

  2. Configure user name and email in Git.

    Name and email should match your account information on Bitbucket server.
    Run the commands in Windows command prompt.
    $ git config --global user.name "Name"
    $ git config --global user.email "sample@zhengwu.org"

  3. Clone repository to local.

    Open a command prompt. Go to a folder you want to clone the remote “ExampleRepositiry” to. Run command:
    git clone https://userid@bb.zhengwu.org/scm/exampleproject/examplerepository.git
    You will see a sub-folder with the name “ExampleRepository” is created in local.
    A hidden folder “.git” is created in the sub-folder. It’s used to tracking changes.

  4. Open workspace in Visual Studio Code

    Bitbucket Server configuration is completed in local.
    Now open Visual Studio Code -> File -> Add folder to workspace -> Open “ExampleRepository” folder.
    Visual Studio Code is integrated with Git in-the-box. It detects the repository automatically.
    It shows “Unrack” if you create a new file in the folder.

Conclusion

This is an expressway to integrate Bitbucket Server with Visual Studio Code. I’m still new on Git. Following are some useful information.

https://www.atlassian.com/git/tutorials/install-git?section=windows

https://www.atlassian.com/git?utm_campaign=learn-git&utm_medium=in-app-help&utm_source=stash

Show CDP Neighbor of Cisco UCS Uplinks

There are two ways to know which network switch ports the network uplinks of Cisco UCS Fabric Interconnects are connected to.

By CLI

  • SSH to the Cisco UCS Manager.
  • Connect to FI-A.
# connect nxos a
  • Show neighbor of network uplinks.
# show cdp neighbor interface ethernet <port num>

By PowerShell

  • Make sure Cisco PowerTool (For UCS Manager) is installed.
  • Enabling the Information Policy via UCSM GUI.
    • Go to “Equipment” -> “Policies” tab -> “Global Policies” tab -> “Info Policy” area.
    • Change to “Enabled“. (No impact to running blades)
  • Open a PowerShell window.
  • Connect to the UCS Manager.
# Connect-Ucs <UCS FQDN>
  • Show CDP neighbor details.
# Get-UcsNetworkLanNeighborEntry

Side notes

Following command can shows network switch name, network switch ports and FI ports

# Get-UcsNetworkLanNeighborEntry | Select deviceid,remoteinterface,localinterface

If you prefer to enable the “Info Policy” by PowerShell, run following command

# Get-UcsTopInfoPolicy | Set-UcsTopInfoPolicy -State enabled -Force

How To Find Non-tagged ESXi Hosts

There are plenty of scripts to find tagged ESXi hosts. But what if you want to find out all ESXi hosts not be tagged? Following is a simple script:

Compare-Object ((Get-VMHost | Get-TagAssignment).Entity | select -uniq) (Get-VMHost)

The output is similar like following:

InputObject      SideIndicator
-----------      -------------
esx1         =>  
esx2         =>  
esx3         =>

The => indicates ESXi hosts in InputObject are not tagged.

If return is nothing, it means all ESXi hosts are tagged.

Please refer to “Using the Compare-Object Cmdlet” for detail.

Automatic vSphere Capacity Report in PPT

Reporting is important to management. To be a IT Pro, you may need to run regular reports for management. Some reports may be generated time consume. vRealize Operations Manager is an alternative to create customized reports. It’s a powerful product to organize data and create PDF or CSV files on scheduled intervals. I recommend have a look if you have planned to implement performance, capacity and alarm system for virtual environment.

What if budget is constrained? Is there a way to create such kind of reports? The answer is “Yes”. I worked out an automatic workflow to create the reports. I will not provide step-by-step guide in this post since it’s advanced integration of multiple products, everyone may have different way to do that. You can even create everything by script if you have strong programming skill. I’m not, I only look for the easiest way to achieve the goal.

Here is a scenario for  example: I want to run a monthly report for vSphere CPU and memory count and present to management by PowerPoint. I want to show management the historical trend of CPU and memory data. The traditional way is collect data in vCenter, organize and create charts in PowerPoint slides. So the whole workflow is: vCenter -> PowerPoint

If you want to automate the whole process you need to introduce few things more: PowerCLI, CSV and Excel. You need to develop a PowerCLI script to grab CPU and memory data on vCenter Server, then export the data to a CSV table by PowerShell command export-csv. Then import the table to an Excel file by Office feature Query Data. It loads the CSV table dynamically, you can even specific what data can be queried by filter.

Once the table is present in Excel, you need to create a chart accordingly. It’s trick when you paste the chart to PowerPoint Slide. You need to use Paste Special to paste the chart as Microsoft Excel Chart Object. The pasted chart can be updated automatically when you open the PowerPoint file.

The last step is created a scheduled task to run the PowerCLI script. Make sure you read my blog Extremely slow when run PowerShell script by scheduled tasks before create the task.

You can also configure the Excel file to automatically update table by CSV file.

How to Integrate PowerCLI 6.5 with PowerShell and PowerShell ISE

I wrote an article to introducing how to integrate PowerCLI with PowerShell and PowerShell ISE. VMware just released PowerCLI 6.5 R1, it includes lot  of new features and modules. And somehow my way doesn’t work. Following is new way to integrate PowerCLI 6.5 with PowerShell and PowerShell ISE in Windows 10.

PowerShell and PowerShell ISE both have it own $profile. So we need to do two times.

Before we start

Please make sure your PowerShell execution policy is not restricted. You can get the setting by run  following command:

Get-ExecutionPolicy

PowerShell Integration

  1. Open PowerShell window. Run following command to confirm profile file is not existing.
    Test-Path $profile

    If return is ‘False’, go to step 2.
    If return is ‘True’, Backup the file and go to step 3.

  2. Run following command if the  profile file  doesn’t existing.
    New-Item -Path $profile -type file -force | Out-Null
    
    Test-Path $profile

    The return above should be ‘True’. Profile file is created.

  3. Run following command to include VMware PowerCLI modules in  PowerShell.
    Add-Content -Path $profile -value "# Load Windows PowerShell cmdlets for managing vSphere `r`n. 'C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1'"

    The blue text above maybe different in your environment base upon where your PowerCLI is installed.

PowerShell ISE Integration

PowerShell ISE process is same to PowerShell, only different is all the operation should be completed in PowerShell ISE window.

Reboot is not required in my environment. But anyhow please reboot if you see any issue.

The processes above integrate PowerCLI 6.5 with PowerCLI and PowerCLI ISE for  current user only. If you want to integrate for all users on the machine, you need to refer to this article.