vCenter 5.1 SSO upgrade to update 1

While upgrading from vCenter 5.1 to vCenter 5.1 Update 1 everything went fine (at least the installer). But when trying to logon in vCenter after half an hour I noticed it was only possible to login with a local account and not with a “domain” account.

While searching trough the SSO logs I saw some strange things like: Connection timed out: connect

Troubleshooting VMware Single Sign-On configuration and installation issues in a Windows server (2033880)

When logging in to the Webclient and watching the SSO settings, all the domains where tested successfully. So there is a connection, but I guess something during the upgrade or a change in the domains caused it to fail.

I used the command below to do a rediscover of the domains, 2 new domain resources where added. Unfortunately it still not worked properly. But this didn’t change the default domains.
Now I removed all the domains listed in the SSO and did another rediscover.

C:\Program Files\VMware\Infrastructure\SSOServer\utils>ssocli.cmd configure-riat -a discover-is -u admin -p masterPassword

Now I noticed that the log files changed and a lot of other information came trough the logs.
I normally use Baretail to follow tails in Windows log files.

When I saw a lot of “Success” logins in the logfiles I had a good feeling it was working again.


Login works fine now !

After 24 hours, it seemed to be failing again, now I removed everything again, waited a few minutes to be sure the DB has time to cleanup. Then I re-added the domains, according to the log files everything should be working again. But when I try to login I now received an error message that I don’t have any authorization. I noticed when I logged in locally the permissions are missing. So I needed to re-add them to the folders etc.

So be warned that permissions can be removed when waiting to long !

Logging in to vSphere Client 5.1 fails with the error: The server took too long to respond (2038918)
Updating the vCenter Single Sign On server database configuration (2045528)

Snapshots and a twoGbMaxExtentSparse VMDK

While converting some machines from KVM to ESX, we suddenly noticed that a scheduled VEAAM backup job, killed the VM and prevented it starting. We got messages like this. Somehow we have a lot of other Linux machines which work flawlessly including backups.

Power On virtual machine
File [Datastore]
was not found
View details...

Consolidate virtual machine disk files
File [] was not found

The strange thing I noticed when I logged in with SSH I saw  a lot of -S001,vmdk disks. Mmm…does me reming to another post I made earlier:  See last part of this post, I didn’t write about the details but I noticed that the imported machines where twoGbMaxExtentSparse format.

With the storage vMotion or vmkfstools inflate it is converted to a normal single VMDK.

Mmm….should this cause the snapshot to fail and let the VM think it’s disk is lost ?

What I did is remove the disks from the VM, re-added them to the VM and directly after that svMotioned them to another datastore so the disk will be inflated. After that the machine starts flawlessly. Now I will need to do a little research to see if the snapshot operation caused the VM to crash.

It’s also possible to use the vmkfstools -i to inflate the disk of course.

Extra resources:

Recreating a missing virtual machine disk (VMDK) descriptor file (1002511)
Recreating a missing virtual disk (VMDK) descriptor file for disks split into 2GB files (1026266)
Cannot power on a virtual machine because the virtual disk cannot be opened (1004232)




ESXTop Batchmode Quick Reference


Thanks to Duncan’s Blogpost, a quick summary to start a quick capture for analysis later.

To start ESXtop in batch mode:
esxtop -b  -d 5 -n 120| gzip -9c > /tmp/stats.csv.gz

If you created a configuration file (/tmp/logging), use the command below
esxtop -b -c /tmp/logging -d 5 -n 120| gzip -9c > /tmp/stats.csv.gz


usage: esxtop [-h] [-v] [-b] [-l] [-s] [-a] [-c config file] [-R vm-support-dir-path]
[-d delay] [-n iterations]
[-export-entity entity-file] [-import-entity entity-file]
-h prints this help menu.
-v prints version.
-b enables batch mode.
-l locks the esxtop objects to those available in the first snapshot.
-s enables secure mode.
-a show all statistics.
-c sets the esxtop configuration file, which by default is .esxtop50rc
-R enables replay mode.
-d sets the delay between updates in seconds.
-n runs esxtop for only n iterations.
—–Experimental Features————-
-export-entity writes the entity ids into a file, which can be modified
to select interesting entities.
-import-entity reads the file of selected entities. If this opion
is used, esxtop only shows the data for the selected entities.


Usage: gzip [-cfd] [FILE]…

Compress FILEs (or stdin)

-d      Decompress
-c      Write to stdout
-f      Force