Setting NTP on ESXi Hosts with PowerCLi

Setting NTP on ESXi hosts is a quick procedure using PowerCLi. Don’t forget to edit the script for your environment. Once you’re connected to vCenter you can run this script:


silhouette with brain and gears turning
Knowledge is power

Comments are always welcomed. Let me know if this has been helpful.

More from my site

Scratch File Location for VMHosts PowerCLi Script

Schannel Set-Crypto PowerCLI Script

Backup VDS PowerCLI Script

Cross SSO vMotion Between vCenters

Setting DNS for Windows Servers Using PowerShell

Be social and share:
LINKEDIN

SEsparse Snapshot Data Inconsistencies

SEsparse snapshot data inconsistencies have been reported with databases. This is one of last week’s more interesting knowledge base articles. This is not something to take lightly. The knowledge base article is titled ‘Virtual Machines running on an SEsparse snapshot may report guest data inconsistencies (59216)’ . The good news is that it has been resolved and there is also a workaround. Please click on the link to see the entire kb article that includes the workaround.

Symptoms

  • Applications such as databases may report block-level data inconsistency.
  • Guest operating systems may report file system metadata inconsistencies
  • The VM fails to boot when it is running from an SEsparse snapshot.

SEsparse is a snapshot format introduced in vSphere 5.5 for large disks, and is the preferred format for all snapshots in vSphere 6.5 and above with VMFS-6.

Cause

VMware has identified an issue in SEsparse VM snapshots that can cause data inconsistencies.

This issue occurs when a VM is running on an SEsparse snapshot and experiences a burst of non-contiguous write IO in a very short period of time.

Impact / Risks

Impacted Configurations:

  • VMFS-5 or NFS Datastores: VMs with virtual disks >2TB and snapshots. On VMFS-5 and NFS, the SEsparse format is used for virtual disks that are 2 TB or larger
  • VMFS-6 Datastores: VMs with snapshots. SEsparse is the default format for all snapshots on VMFS-6 datastores.

Impacted vSphere releases:

  • vSphere 6.5 and above with VMFS-6 and any VM with snapshots.
  • vSphere 5.5 and above when VMs with virtual disks >2TB have snapshots.

Note: VMFS-5 or NFS datastores only use SEsparse with snapshots when virtual disks exceed 2 TB in size or SEsparse is explicitly configured.

silhouette with brain and gears turning
Knowledge is power

Resolution

The issue is resolved the following releases:

Be social and share:
LINKEDIN

Beyond SQL on VMware Best Practices

Going beyond SQL in VMware Best Practices has involved settings and configurations that are never mentioned in the SQL Server On VMware Best Practices Guide. Working in a healthcare environment means working with a large EMR (Electronic Medical Record) provider like EPIC. In tuning one of their many large sql databases they had some recommendations. Some of these are throwbacks to my old days of tweaking performance out of a desktop with settings like TCP parameter KeepAliveTime. How many of you go beyond the recommendations in the SQL Best Practice Guide?

floating book in quaint library
Knowledge shared becomes wisdom attained (Photo by Jaredd Craig on Unsplash)

I’ve listed here some of the settings that have been suggested for our larger more accessed and just plain busy sql servers:

ESXi Settings

Adjust the Round Robin IOPS limit from the default 1000 to 1 on each database LUN. Refer to VMware KBA 2069356 for more information on setting this parameter. (We already utilize Round Robin but each lun was set to the default)

Why would you want to make this change?
“The default of 1000 input/output operations per second (IOPS) sends 1000 I/O down each path before switching. If the load is such that a portion of the 1000 IOPS can saturate the bandwidth of the path, the remaining I/O must wait even if the storage array could service the requests. The IOPS or bytes limit can be adjusted downward allowing the path to be switched at a more frequent rate. The adjustment allows the bandwidth of additional paths to be used while the other path is currently saturated. “

How to make this change:

In ESXi 5.x/6.x:
for i in esxcfg-scsidevs -c |awk '{print $1}' | grep naa.xxxx; do esxcli storage nmp psp roundrobin deviceconfig set –type=iops –iops=1 –device=$i; done

Where, .xxxx matches the first few characters of your naa IDs.
 
To verify if the changes are applied, run this command:

esxcli storage nmp device list
 
You see output similar to:
 
Path Selection Policy: VMW_PSP_RR
Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0;lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba33:C1:T4:L0, vmhba33:C0:T4:L0

Registry Settings

Configure Windows TCP Parameters in the Registry

The default setting for the Windows TCP parameter KeepAliveTime is two hours. This setting controls how often TCP sends a keep-alive packet to verify that an idle connection is still intact. Reducing it from two hours to five minutes helps windows detect and clean up stale network connections faster.


How to make this change:

Use regedit to create the DWORD KeepAliveTime (if it does not currently exist) at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters\ . Then modify the value to 300000 (time in milliseconds).

The default setting for the Windows TCP parameter TCPTimedWaitDelay is four minutes. This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. By reducing the value of this entry, TCP/IP can release closed connections faster and provide more resources for new connections.

How to make this change:

Using regedit to create the REG_DWORD TCPTimedWaitDelay (if it does not currently exist) at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters\ . Then modify the value to 30 .

What kind of ESXi settings, Windows registry settings, config file changes, etc. do you implement in your environment that goes beyond the SQL Server On VMware Best Practices Guide ? As always, I look forward to your comments and sharing of knowledge.

Be social and share:
LINKEDIN

Last Week’s Most Interesting VMware KBs

Last Week’s VMware KBs of Interest for week ending May 27th, 2018

These are the KBs I found of the most interest to me in my environment.  They might be of interest to you as well.

vCenter Server Appliance Management Interface might not display the vCenter Server 6.7.0a patch  

If you were one of the psychos that likes to upgrade to the bleeding edge as soon as it goes GA then this one may have bitten you. There is currently no resolution but there is a workaround. 

 

While migrating from a Windows vCenter Server to a 6.5 vCenter Appliance, one or more services fail to start

If you utilize hosts files instead of a valid forward and reverse dns entry then this one my rear its ugly head during your migration.  There is a workaround with two parts.  You really should use the first part of the workaround and skip the second part which continues the use of hosts files.  This is why you are in this mess in the first place.  Please check it out.

Upgrading VM Virtual Hardware through Update Manager 6.x fails on Linux Virtual Machines

This is a known issue and there is no resolution currently.  Please see KB for a list of the symptoms to see if you are truly affected.

Hopefully these are helpful and hopefully reading these will prevent some issues before they happen. Please read the full list by clicking the picture below or the source link.

Get in there!

 

VMware Support News, Alerts, and Announcements

Source: VMware Support Insider

Be social and share:

LINKEDIN

Backup VDS PowerCLI Script

I started doing all my VDS (Virtual Distributed Switch) backups manually which isn’t too hard. But you can do better with a Backup VDS script.  Even if you don’t realize it now, you will want to automate as much as possible.  Here is how I automated VDS backups. Starting with a powercli script from VCDX56 I made sure that I understood how it worked and modified it to fit my environment. 

Backup VDS PowerCLI Script Modification

You can edit your Backup VDS PowerCLI script to fit your needs. My modification started with a few variables which are automatically incorporated into the naming of the backup files.  To keep several versions of the backup files I added the date into the file naming as well. This really helps keep the vds backup files organized by name and date.  You will need a vCenter connection in the script.  If you want to utilize mine called Get-vCenter (It is very handy in a multi-vCenter environment) please feel free.

Backup VDS Code

For complete automation create a static array for all your vCenters and loop them. Then you nest this script in another loop.  Utilize Task Scheduler on a Windows computer to call the script on a schedule.  Then you have automated backups of all your VDS switches.

Caution: Like all code you download from the internet, please understand and modify the code accordingly to prevent unforeseen production problems.  Also known as career-altering events. 

 For this script the main command is Export-VDSwitch. I like to familiarize myself with any PowerCLI commands I am running in my environment.  The quickest way to get help and examples is to run from a Powershell prompt the command ‘help verb-noun -showwindow’.  In this case it would be help Export-VDSwitch -showwindow.

Be social and share:

LINKEDIN

vami_config_net To Change VCSA Hostname

After reading the title of this article I’m sure you’re saying, ‘You can’t change VCSA hostname.  You have to redeploy.’  That is what I was told and all the documentation I have read says you have to redeploy.  Well it is not true.  With that said here is your warning about mucking with the VCSA configuration.  Don’t do it!  Unless you are working in your test environment and you came across a ‘workaround’ that you wanted to try.  I still wouldn’t recommend using this ‘workaround’ in your production environment without extensive testing and upon recommendation from VMware Support.

Why did I need to change the hostname in my test lab?  I applied 6.5 U1 to the VCSA in my test lab.  I then checked the VCSA and saw that the hostname was changed to ‘localhost’ and AD authentication was broken.  It also broke ssl certificates.  I was getting ready to redeploy the VCSA when I came across this ‘workaround’ and I gave it a try.  It worked great.

Change VCSA 6.5 U1 Hostname with vami_config_net

The ‘workaround’ is  really a built in utility called vami_config_net. The full name of this utility is configure-network command-line utility. Here is what the utility looks like in use with relevant configuration names blacked out.  

Shows vami_config_net command
vami_config_net

shows Main Menu for vami_config_net
vami_config_net Main Menu

show Current Configuration in vami_config_net
vami_config_net Current Configuration

shows Hostname vami_config_net
vami_config_net Hostname

Option # 3 will prompt with ‘New hostname’ and show the current hostname.  I made this change in my test lab and it did not require a reboot to start working.  However, I made this change after the VCSA had lost domain trust.  I had to take the VCSA completely out of the domain, delete the computer object, and join it to the domain again.  The VCSA is working wonderfully once more.

If I had tried to change the ip address it would have failed without changing the dns to reflect this before making the change.  In this very limited use case vami_config_net worked because I changed the hostname back to the original name. I would not have faith in using this utility to just to change from one hostname to a completely different hostname until I have tested further.

More info on vami_config_net

Allocate a static IP address to the VMware vCenter Server Appliance (VCSA)

How to “fix” VCSA IP settings from command line.

Automating VCSA Network Configurations For Greenfield Deployments

Let me know if this article helps you.  Please share your experiences with vami_config_net.  I look forward to hearing from you!

shows silhouette with working gears for a brain

 

Be social and share:

LINKEDIN

Changing vCenter Default Domain

One of the less annoying things I encounter on a daily basis is the wrong default domain on my vCenter appliance. Changing the vCenter default domain is necessary in my environment because the empty-root domain is default. Our main domain where all of our user accounts reside is a sub-domain of the empty-root domain. That means that you can’t just login with your normal credentials without using the domain\username or username@domain.com formats. This isn’t a large problem but anything that speeds up my day is always appreciated.

It turns out that this is a known problem for users in a child domain where the vCenter has been upgraded from version 5.5.0 to 5.5.0b or later.  In my case the users can login still if they put the domain prefix as part of their login.  I just don’t want to have to worry about that especially for those in our enterprise that can’t figure out how to login by using a domain prefix.

Resolution

To change the behavior of the identity source, the default domain can be changed on the Single Sign-On (SSO) server from the domain that was created during the upgrade.

Windows-based Single Sign-On (SSO)

Connect to the machine that is running the SSO instance.
Create the defaultdomain.ldif file containing this information using a plain text editor:

dn: cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local
changetype: modify
replace: vmwSTSDefaultIdentityProvider
vmwSTSDefaultIdentityProvider: example.com

Note: Replace example.com with the desired default domain from your environment. Contents of .ldif file should be terminated with “-” .

As an Administrator, click Start > Run, type cmd and then click OK.
Run C:\>ldifde command to confirm that the ldifde tool is available. This list returns a list of available commands.
If the tool is not present, install it by running this command:

C:\>ServerManagerCmd -i RSAT-ADDS-Tools

For Windows 2012 run this powershell command:

Install-WindowsFeature RSAT-ADDS

Run this command to update the default domain:

C:\>ldifde -i -f filepath\defaultdomain.ldif -s localhost -t 11711 -a “cn=Administrator,cn=Users,dc=vsphere,dc=local” *

 

When prompted, enter the Administrator@vsphere.local Single Sign-On (SSO) password.
The command should complete successfully.

VMware vCenter Server Appliance with local Single Sign-On (SSO)

Connect to the machine that is running the SSO instance.
Create the defaultdomain.ldif file containing this information using a plain text editor:

dn: cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local
changetype: modify
replace: vmwSTSDefaultIdentityProvider
vmwSTSDefaultIdentityProvider: example.com

Note: Replace example.com with the desired default domain from your environment. Contents of .ldif file should be terminated with “-” .

Open a console to the vCenter Server Appliance.
Run this command to update the default domain:

/opt/likewise/bin/ldapmodify -f filepath/defaultdomain.ldif -h localhost -p 11711 -D “cn=Administrator,cn=Users,dc=vsphere,dc=local” -W

 

Enter the Administrator@vsphere.local SSO password.
The command should complete successfully.

 

Here is the link to kb2070433  if you would like to read the full article for yourself. It is a trivial change to fix a trivial problem but I am glad to say it works like a charm.

Be social and share:

LINKEDIN

Killing SSLv3 Tip

This week brought another unusual problem. We have a multi-domain environment that includes 2 different active directory forests with a trust. Like most of the world we have disabled SSLv3 on desktops as well as servers to prevent SSLv3 connections but this was only done completely in our main active directory domain. Everything has been working fine until this week.   Over the weekend a new change was introduced to the environment in the form of a new sha2 certificate for domain controllers in the other active directory domain. Once this change was implemented user accounts from the other domain would no longer authenticate for our Horizon View vCenter.

Settings were checked and the LDAPs identity source was identical on both our vCenters in our main domain but one did not work. Certificate stores were checked and they both had the relevant certificates.  After digging further there was one difference between the 2 vcenter servers concerning SSL.

Look under vCenter Server Settings.

SSLv3-SS1

There is a setting located under Advanced Settings called SSL.Version.

SSLv3-SS2

Choose TLSv1 to completely stop vCenter from trying to communicate over SSLv3.

SSLv3-SS3

 

 

 

Be social and share:

LINKEDIN