CPU affinity or not!

For our Citrix environment there is an existing design to use 24 vCPU on a 16 core Hyperthreaded ESX host which presents 32 logical CPU’s. This should work properly in my opinion.
But as our project team comes across some performance issues, they searched around a bit and came with the following proposal.
6 VM’s – 4 vCPU’s each and set affinity to 0-3,4-7 etc.

I already asked them to do some proper research and testing about this but they lacked a bit on this, so I tested this myself. As I found on different sites, they almost all mention : DON’T USE IT! Unless you really know what to do. As far as I can see, they found some best practice guides and decided to follow them blindly!

With a little test I could prove them they need to put at least 1 extra CPU in the affinity rule for some hypervisor stuff. Or just don’t use it.

See the below results when I turned it on and off, you immediately see an increase on  Ready times for the machine I turned it on for, and a decrease on the machine I added another affinity CPU.

VM1 used no affinity before and was default, around 09:30 I put the CPU affinity to 0-3; as you can see an immediately increase in ready times. Removed the CPU affinity at 14:00 the same day, as you can see the graph drops instant.

VM2 was set to affinity 16-19, I added another CPU to the affinity as seen in some best practice resources. After submitting the changes around 09:25 you can see the ready times drop to a lower level.

There is a little peak at 10:05 because there were a lot of VM’s started on that particular ESXi host, but you can see the clear difference after that time, as soon as I had enough logging I turned everything back to the original settings. You can see this at 13:55-14:00. Also noticeable is the little higher average after the settings are returned as before, but as I mentioned a little earlier this could be explained because there was more load on the ESXi host where the VM’s were residing.

Graph 1 (Long day)

Day_CPU_Ready
Graph 2 (Today)

Day_CPU_Ready2

http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.0.pdf
http://frankdenneman.nl/2011/01/11/beating-a-dead-horse-using-cpu-affinity/
http://www.yellow-bricks.com/2009/04/28/cpu-affinity/

ESXi5.1 U1 resolves CIM issues

A while ago when we rolled out ESXi5.1 on our environment I noticed a lot of strange CIM errors coming trough in our Kiwi Syslog server. I tried to investigate where this was coming from, but couldn’t find a proper solution.

CIM

Finally when I was reading the release notes of ESXi5.1U1 I noticed there were some things concerning CIM and API issues.
http://www.vmware.com/support/vsphere5/doc/vsphere-esxi-51u1-release-notes.html#resolvedissuescimapi

We already came accross the issue with the full ram disk trough the SNMP trap settings. After installing U1 on some test ESXi hosts I noticed the strange CIM errors dissappear. After a week, I didn’t see any coming back while they normally occured in a 4 hours time frame.

So if you might have similar problems, try U1 as it solves some of these issues.

Resetting the inventory service database

When a colleague cleaned some log files which filled up the C: drive; we noticed that this caused the Inventory Service to die. Some of the vCenter user where also complaining that the “Search” function didn’t work anymore.

The location of the log files was: “C:\Program Files\VMware\Infrastructure\Inventory Service\data”

This caused the “inventory service” to get killed and remain unstartable. Because we had no backup of the database we needed to reset it. See the link below for the proper VMware article. In short here this is my summary.

Caution

This procedure can cause data loss. Perform this procedure with VMware Technical Support.

I took the guess and did it without VMware support because it wasn’t working anyway 🙂

  1. Stop the Inventory Service
  2. Remove folder: “C:\Program Files\VMware\Infrastructure\Inventory Service\data”
  3. Go to “c:\Program Files\VMware\Infrastructure\Inventory Service\scripts”
  4. “Createdb.bat” with no arguments
  5. Start “Inventory Service”
  6. Reregister the “inventory service”
  7. “cd C:\Program Files\VMware\Infrastructure\VirtualCenter Server\isregtool”
    “register-is.bat <vcenterFQDN>:443/sdk <vcenterFQDN>:10443 <vcenterFQDN>:7444/lookupservice/sdk”
  8. Restart the “Inventory Service”
com.vmware.vim.vcauthorization.impl.provider.InMemoryAuthCache] Processing unresolved entities 06F31E7F-0667-4628-A5E2-FAB1AE6B9D61
com.vmware.vim.query.server.provider.AbstractAtomPullProviderBase] Provider 06F31E7F-0667-4628-A5E2-FAB1AE6B9D61 new generation: 2102329
com.vmware.vim.vcauthorization.impl.provider.InMemoryAuthCache] Done processing feed update for 06F31E7F-0667-4628-A5E2-FAB1AE6B9D61
com.vmware.vim.vcauthorization.impl.provider.InMemoryAuthCache] Processing unresolved entities 06F31E7F-0667-4628-A5E2-FAB1AE6B9D61
com.vmware.vim.query.server.provider.AbstractAtomPullProviderBase] Provider 06F31E7F-0667-4628-A5E2-FAB1AE6B9D61 new generation: 2102330
com.vmware.vim.dataservices.federation.FederationReconfigurator] Checking/updating federation configuration
com.vmware.vim.dataservices.federation.FederationReconfigurator] No peers reachable - skipping reconfiguration

The search function will now work again 🙂

net stop vimQueryService
del C:\Program Files\VMware\Infrastructure\Inventory Service\data
C:\Program Files\VMware\Infrastructure\Inventory Service\scripts\createDB.bat
net start vimQueryService
cd C:\Program Files\VMware\Infrastructure\VirtualCenter Server\isregtool
register-is.bat vcenter.fqdn:443/sdk https://vcenter.fqdn:10443 <a href="https://wsv10372.office01.internalcorp.net:7444/lookupservice/sdk">https://vcenter.fqdn:7444/lookupservice/sdk</a>
net stop vimQueryService
net start vimQueryService

http:\\kb.vmware.com\selfservice\microsites\search.do?language=en_US&cmd=displayKC&externalId=2017370

Incorrect “Degraded status” on HP Array Controller

I tried and worked flawlessly, so for all of you with this error….go and donwload !

Fixes:

Upgrade Requirement:
Recommended – HP recommends users update to this version at their earliest convenience.


The following fixes were added to this release:

  • Smart Array Controller incorrectly reports Degraded status: Fixed issue where HP Insight Management WBEM Providers were incorrectly reporting a ‘Degraded’ status for the Smart Array Controllers. This caused VMware vSphere Management Console under Health Status category to display a ‘Warning’ Status (Yellow Exclamation) for the HP Smart Array Controllers.  HP System Insight Manager and Insight Control for vCenter and other users of the HP Insight Management WBEM Providers would also report the Smart Array Controller as ‘Degraded’.
  • Smart Array physical drive incorrectly reports OK status: Fixed issue where HP Insight Management WBEM Providers were not correctly reporting a not-OK status for a removed drive connected to a Smart Array Controller. This caused HP System Insight Manager, VMware vSphere Management Console, Insight Control for vCenter and other users of the HP Insight Management WBEM Providers to report the physical drive as OK after it was removed.
  • Server reports incorrect processor Model: Fixed issue where HP Insight Management WBEM Providers would report an incorrect Processor Model for ProLiant servers and blades. For example: “Intel(R) Family: Intel(R) Xeon(TM) 2.5GHz (x86 Family 179 Model 125 Stepping 7)”” is reported instead of the correct  “Intel(R) Family: Intel(R) Xeon(TM) 2.5GHz (x86 Family 179 Model 45 Stepping 7

>Software can be downloaded here<

 

How do snapshots work?

How do snapshots work ?

Sometimes I get some questions how snapshots actually work, luckily there are some good articles on the VMware site.
A good post about this is described on the VMware site in the link below:

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1015180

2 nice videos about consolidating snapshots

ESXi3.x and 4.x

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007849

ESXi5.x

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2003638

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2003638

1 2 3 4