Terraform Cloud – How to List All Users

woman writing on whiteboard

Terraform Cloud has a rich API. However, the API documentation does not mention how to list all users. We can leverage the organization membership API and the PowerShell command Invoke-RestMethod to get a user list.

  1. Create an organization token in Terraform Cloud.
  2. Create the token variable $Token in PowerShell.
$Token = "abcd"
PowerShell
  1. Create the API parameters variable in PowerShell.
    You need to replace ZHENGWU with your own organization name. And I used 100 at the end of the URI to retrieve the first 100 users. It can be any number.
$params = @{
  Uri = "https://app.terraform.io/api/v2/organizations/ZHENGWU/organization-memberships?page%5Bsize%5D=100"
  Authentication = "Bearer"
  Token = $Token
  ContentType = "application/vnd.api+json"
}
PowerShell
  1. Retrieve the API return and list the user’s email address.
$Test = Invoke-RestMethod @params
$Test.data.attributes.email
PowerShell

Connect-NsxtServer shows “Unable to connect to the remote server”

When you run Connect-NsxtServer in the PowerCLI, it may show “Unable to connect to the remote server“.

Because the error message is a little bit confusing with other login issues. It’s not easy to troubleshoot. The actual reason is the NSX-T uses a self-signed certificate, and the PowerCLI cannot accept the certificate automatically.

The fix is super easy. You need to set the PowerCLI to ignore the invalid certificate with the following command:

Set-PowerCLIConfiguration -Scope User -InvalidCertificateAction:Ignore -Confirm:$false

Move Terraform Providers to Other Folders

Create a new control file with the name .terraformrc or terraform.rc in your profile folder.

Add the following lines:

plugin_cache_dir   = "$HOME/.terraform.d/plugin-cache"

Create the folder .terraform.d/plugin-cache in your profile folder.

The providers will be downloaded to the cache folder when you run terraform init.


If you don’t want to create the control file in the profile folder. Alternative is to create an environment variable.

export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache"

Setup Terraform and Ansible for Windows provisionon CentOS

black server racks on a room

Provisioning Windows machines with Terraform is easy. Configuring Windows machines with Ansible is also not complex. However, it’s a little bit challenging to combine them. The following steps are some ideas about handling a Windows machine from provisioning to post configuration without modifying the winrm configuration on the guest operating system.

  1. Install required repos for yum.
yum -y install https://repo.ius.io/ius-release-el7.rpm
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install https://packages.endpointdev.com/rhel/7/os/x86_64/endpoint-repo.x86_64.rpm
yum -y install epel-release
yum -y install yum-utils
yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
  1. Install Terraform.
sudo yum -y install terraform
  1. Install Ansible.
sudo yum -y install ansible
  1. Install Kerberos.
yum -y install gcc python-devel krb5-devel krb5-libs krb5-workstation
  1. Install pip.
sudo yum -y install python-pip

# You probably need the following packages if you are using VPN
pip install pysocks
  1. Install pywinrm[kerberos].
pip install pywinrm[kerberos]
  1. Configure /etc/krb5.conf.
    The following are the required lines. Please make sure to change the domain name to yours. And it’s case-sensitive.
[libdefaults]
 dns_lookup_realm = true
 dns_lookup_kdc = true
 forward = true
 forwardable = true
 default_realm = ZHENGWU.ORG


[realms]
 ZHENGWU.ORG = {
  kdc = DC.ZHENGWU.ORG
  admin_server = DC.ZHENGWU.ORG
 }

[domain_realm]
 .zhengwu.org = ZHENGWU.ORG
 zhengwu.org = ZHENGWU.ORG
  1. Create an Ansible inventory file.

[win] #Group name
dc.zhengwu.org #This is the target server list
 

[win:vars]
ansible_connection=winrm 

ansible_user=administrator #It's better a domain admin account.
ansible_password=P@ssw0rd #Change this password
ansible_port=5985
ansible_winrm_transport=kerberos
ansible_winrm_server_cert_validation=ignore
  1. Run Ansible win_ping test.
ansible <group in inventory file> -m win_ping -i <inventory file>

Quick Note: Modify PowerShell Execution Policy in Registry

I think to maintain a solid security policy is mandatory in an enterprise network. The information leak not only a threat to IT but also impact the firm’s profits and stock performance. Firms usually leverage Active Directory Group Policy to control security settings on Windows. The PowerShell execution policy is the one most company ITs would restrict.

IT Pros may need to do some testing in scripting. I have posted how to change PowerShell execution policy by PowerShell command.

Following is the step of how to get rid of the restriction. This is just for temporary use. You should revert it immediately after testing!!!

  1. Open Registry Editor.
regedit.msc
  1. Go to HKLMSOFTWAREMicrosoftPowerShell1ShellIdsMicrosoft.PowerShell.
  2. Change the registry key ExecutionPolicy value to RemoteSigned.
  3. Open a new PowerShell prompt and run your script.

If you don’t want to change the registry key. You can paste the scripts into Windows PowerShell ISE and run all of the lines. Please note this option may throw error if you have relative paths in the script due to the temp script running under the same directory of Windows PowerShell ISE application.

How to run selection in Windows PowerShell ISE
How to run selection in Windows PowerShell ISE

Please refer to Microsoft document if you want to learn more about PowerShell execution policy.

How to Manage Windows Servers With Ansible on CentOS 8

Ansible is a popular automation tool for infrastructure configuration. It runs on the Linux system. CentOS is an ideal distribution to run Ansible for lab purposes. It is similar to the Red Hat Linux but free. And the latest major release is CentOS 8. It contains Python 3 by default. So the Ansible configuration is different from CentOS 7. I will focus on the configuration in the lab environment. The goal is to create a simple environment to manage Windows servers with Ansible.

Ansible Installation on CentOS 8

I used CentOS 8 mini installation. It has no extra software installed. The procedure below maybe a bit different from your environment if you installed other roles on the OS.

Ansible is a standalone application that not rely on databases. There are two files it mainly needed in a quick lab environment: Playbook and host files. You can install multiple Ansible servers. They can run independently to control the same group of Windows servers.

I would suggest you take a snapshot before moving forward if your Ansible will running on a virtual machine.

  1. Enable Extra Packages for Enterprise Linux for yum.
yum install epel-release
  1. Install Ansible
yum install ansible
  1. (Optional) Install pip for Python 3. This step is for Red Hat 8.
yum install python3-pip
  1. Install pywinrm. The pywinrm will be used to communicate to Windows servers via winrm.
pip3 install pywinrm
  1. Install dependencies for pywinrm to use Kerberos in order to authenticate to Active Directory.
yum install gcc python3-devel krb5-devel krb5-libs krb5-workstation
pip3 install pywinrm[kerberos]

Ansible installation is completed. The procedure is elementary level but I spent some time figuring it out. Especially the Kerberos and pywinrm parts. 🙂

Please go to pywinrm GitHub if you want to dig into it.

Ansible Configuration on CentOS 8

Configure Ansible

As I mentioned in the previous section. There are two main files: Playbook and host. A Playbook is a file consist of multiple tasks that will run on the target Windows servers. It’s not covered by this article. The host file stores variables, and target server FQDNs or IP addresses. Ansible gets the target servers’ information in the host file when you run a playbook.

The host file location is /etc/ansible/hosts. There are two sections in the file for lab purpose.

  1. Server group. You can have multiple groups. Group name is in [ ]. You can give FQDN or IP addresses of the target Windows servers. I recommend using FQDNs if your targets are domain member servers. My example uses server win2019test1.zhengwu.org.
[windows]
win2019test1.zhengwu.org
  1. Variables of the target server group. Since this is for lab purpose. I’ll just list required variables in the /etc/ansible/hosts file. You need to use standalone variable files and avoid to input password if it’s for production. Following is a sample of the variable set for the windows group.
    • Variables are linked to a group by the variable name in the first line: [group name:vars].
    • The domain name should be uppercase in ansible_user. The reason is krb5 requires the uppercase domain name in the configuration file. We should match the name here. The domain name is not required if you use a local account.
    • ansible_winrm_server_cert_validation is optional. It only useful when ansible_winrm_scheme is ‘https‘.
    • ansible_port is ‘5985‘ when ansible_winrm_scheme is ‘http‘. Or ‘5986‘ when ansible_winrm_scheme is ‘https‘.
    • ansible_winrm_transport is ‘kerberos‘ in this example since the target Windows servers are domain members. It can be ‘ntlm‘ if you want to authenticate by local account. There are 5 authentication methods on Windows. Kerberos and NTLM are enabled by default. Please refer to Windows Remote Management for detail.
[windows:vars]
ansible_user='administrator@ZHENGWU.ORG'
ansible_password='123321'
ansible_connection='winrm'
ansible_winrm_scheme='http'
ansible_port='5985'
ansible_winrm_transport='kerberos'
ansible_winrm_server_cert_validation='ignore'

Configure Kerberos

Apart from Ansible configuration. We should configure Kerberos for domain authentication if the target Windows servers are joined a domain. My lab servers are joined domain ‘zhengwu.org‘. We have installed Kerberos components in the Ansible Installation on CentOS 8 section. So we just need to configure it. Edit Kerberos configuration file: /etc/krb5.conf.

  1. Change the default domain name. Make sure to remove # to uncomment the line. The domain name should be uppercase.
default_realm = ZHENGWU.ORG
  1. Uncomment all lines in the realms section. Please note domain name should be uppercase. The parameters kdc and admin_server are the same for the lab environment. The following is an example:

[realms]
ZHENGWU.ORG = {
     kdc = DC.ZHENGWU.ORG
     admin_server = DC.ZHENGWU.ORG
 }

Please refer to MIT Kerberos Documentation for the explanation.

Now Kerberos is configured. We have configured domain credentials in Ansible configuration file, specified Kerberos as the authentication method, and configured Kerberos for Active Directory. We just need to run the Windows ping module in Ansible to test the connection to target Windows servers. You should complete section Manage Windows servers with Ansible if the testing is failed.

ansible windows -m win_ping

You should see following output if authentication is successfully.

win2019test1.zhengwu.org | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Kerberos troubleshooting

You may see authentication problem when validating target Windows server connection by Ansible win_ping module. Here is simple steps to troubleshooting Kerberos authentication

  1. Try authenticate to target Windows servers by domain account on Ansible server. It can be any domain account.
kinit administrator@ZHENGWU.ORG
  1. List cached authentication data. You should see something similar below.
Ticket cache: KCM:0
Default principal: administrator@ZHENGWU.ORG
Valid starting       Expires              Service principal
06/26/2020 03:56:12  06/26/2020 13:56:12  krbtgt/ZHENGWU.ORG@ZHENGWU.ORG
        renew until 07/03/2020 03:56:09

Manage Windows servers with Ansible

The target Windows servers should be configured to accept the winrm connection. Ansible provides a PowerShell script to configure target Windows servers automatically. The script should not be used in a production environment according to Ansible stated in their document.

The configuration is super easy for production. Open a command prompt under the administrator permission and then run following command

winrm quickconfig

Conclusion

Manage Windows servers with Ansible is not so hard as long as the authentication is configured correctly. Ansible is not the only tool for automation. I’m a super fan of PowerShell. I have posted some articles for automation you may want to check. PowerShell and Ansible both are automation tools.

I think manage Windows server with Ansible is like outsourcing PowerShell scripting works to communities. You give inputs to the tasks then Ansible modules will execute pre-defined PowerShell scripts and feedbacks output. Ansible reduces the development time of Windows automation but it still has some disadvantages. Such as you have to run multiple tasks to enable Remote Desktop on target Windows servers which is just a single task in PowerShell DSC. So I think automation of infrastructure is a combination of tools like Swiss Army Muti-Tools, each one has an advantage. We have to use them together to achieve the final goal of automation.

[Quick Note] Failed to install pywinrm on CentOS 8

You may see error message “Running setup.py install for pykerberos … error” when install pywinrm on CentOS 8. The additional errors are “unable to execute 'gcc': No such file or directory” and “command 'gcc' failed with exit status 1“.

The reason because gcc dependence is missing on the machine. You need to run following command to install gcc then try install pywinrm again.

# yum install gcc

Access Deny When Run PowerShell Scripts

You may get access deny when modify particular section of Windows Server. Such as some registry keys or system directories.

The reason is Windows Server protects sensitive part of operating system. This is similar like running command without root permission on Linux. You have to run as administrator to work around access deny problem.

I faced this issue when run guest command on Embotics Commander workflow. Looks like there is no official document talk about this issue. The workaround is disable UAC on Windows Server. Following are some helpful references.

Please refer to Disabling User Account Control (UAC) on Windows Server to understanding impact of disable UAC.

There are plenty of articles on internet talk about how to disable UAC.

There are two steps:

  • Disable UAC notification in Control Panel.
  • Change value of key EnableLUA from 1 to 0 in registry path HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionpoliciessystem.

You may need to reboot the server, or wait few minutes.

Validation is you should see a reminder message when run command.

Disable Default Untitled Tab in Visual Studio

A new tab with name “Untitled-1” is opened by default when you run Visual Studio if you closed all files last time run Visual Studio. It’s not big deal but annoying. Following is how to disable the default untitled tab in Visual Studio

  1. Click “Manage” button on lower left corner of Visual Studio.
  2. Search keyword “Untitled“.
  3. You will find a option call “Workbench: Startup Editor“.
  4. It’s “newUntitiledFile” by default.
  5. Change it to “none“.

Now. “Untitled-1” tab goes away when you launch Visual Studio next time.

Validating Connection Result of Connect-VIServer

Validating connection result of Connect-VIServer is trick since VMware doesn’t use the standard way of throwing errors.

There are different error messages for different scenarios. We can leverage “Try Catch Finally” of PowerShell to testing connection result and output customized errors.

The essential format of testing vCenter connection is below. PowerShell cannot catch error message if you don’t add “-ErrorAction Stop” when connect vCenter.

try{
    connect-viserver vcenter-xxxx -ErrorAction Stop
}
catch [VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin]{
    Write-Host "Permission issue"
}
catch [VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViServerConnectionException]{
    Write-Host "Cannot connect to vCenter Server"
}
catch
    {Write-Host "Other issue"}
finally{
    write-host "the end."
}

How to know error types of the validating connection result?

You probably have question how did I know which error types are for what? I used a very simple way that just give it a try. 🙂

# Simulating connection error
Connect-VIServer vCenter.zhengwu.org -User "xxx" -Password "yyy"

# Format output of $Error. $Error stores output of errors in PowerShell.
$error[0] | fl * -force

# You will see similar lines below
writeErrorStream      : True
PSMessageDetails      :
Exception             : VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin: 7/29/2019 9:27:30 AM   Connect-VIServer                Cannot complete login due to an
                        incorrect user name or password.         ---> VMware.Vim.VimException: Cannot complete login due to an incorrect user name or password. --->
                        System.ServiceModel.FaultException`1[VimApi_67.InvalidLogin]: Cannot complete login due to an incorrect user name or password.

VMware.VimAutomation.ViCore.Types.V1.ErrorHandling.InvalidLogin is the error type we want to find.

Conclusion

This is very simple way to validating connection result of Connect-VIServer. Looks like the result errors coming from different assemblies. For example, connection error is VMware.VimAutomation.Sdk.Types.V1.ErrorHandling.VimException.ViServerConnectionException. It’s hard to find out all possible errors. I’m looking for a official document but no lucky. Please let me know if you know where I can reference.