HP-AMS older then DL380Gen8 Hardware

HP-AMS keeps restarting


A few weeks ago we started to deploy HP Custom Image for ESXi 5.1.0 Update 2 on all our ESXi hosts. Everything seemed to work without problems until a colleague recently discovered in the logfiles that the HP-AMS provider keeps restarting every 5 minutes and gives an error message that it can’t start because it only works on.

We also noticed the problem only occured on ESXi hosts which are not HP DL380Gen8. So DL585 G5,G6,G7 gave these errors. Makes sense, the error also notices that it runs on Gen8 and older!


Luckily I found a VMWare article KB2085618 which described our problem:

Too bad the only solution is to remove the Agentless Management agent…by hand on the command line on 50+ ESXi hosts.

Damn! I was too lame to do this by hand so build a little powerCLI script. It’s not completed yet or error free. It was just a quick and dirty solution for fast results. So it’s not yet completed, but would like to share it already as it is faster then enabling SSH everywhere, connecting to ESXi hosts, insert commands, reboot etc.



– Connect to your vCenter

– Put host in maintenance mode

– Load the module of function

– Plink installed and edited the script to use the right Plink directory

Download Plink here:



Function Enable-TSM {
Param (
[parameter(valuefrompipeline = $true, mandatory = $true,
HelpMessage = "Enter an ESX(i) entity")]

process {
switch ($VMHost.gettype().name) {
"String" {
if ($Local) {$VMHost = Get-VMHost -Name $VMHost | Enable-TSM -Local}
else {$VMHost = Get-VMHost -Name $VMHost | Enable-TSM}
"VMHostImpl" {
if ($Local) {
$VMHost | Get-VMHostService | Where {$_.Key -eq "TSM"} | %{
if ($_.running -eq $false) {
$_ | Start-VMHostService -Confirm:$false | Out-Null
Write-Host "$($_.Label) on $VMHost started"
else {Write-Warning "$($_.Label) on $VMHost already started"}
else {
$VMHost | Get-VMHostService | Where {$_.Key -eq "TSM-SSH"} | %{
if ($_.running -eq $false) {
$_ | Start-VMHostService -Confirm:$false | Out-Null
Write-Host "$($_.Label) on $VMHost started"
else {Write-Warning "$($_.Label) on $VMHost already started"}
default {throw "No valid type for parameter -VMHost specified"}

Function Disable-TSM {
Param (
[parameter(valuefrompipeline = $true, mandatory = $true,
HelpMessage = "Enter an ESX(i) entity")]

process {
switch ($VMHost.gettype().name) {
"String" {
if ($Local) {$VMHost = Get-VMHost -Name $VMHost | Disable-TSM -Local}
else {$VMHost = Get-VMHost -Name $VMHost | Disable-TSM}
"VMHostImpl" {
if ($Local) {
$VMHost | Get-VMHostService | Where {$_.Key -eq "TSM"} | %{
if ($_.running -eq $true) {
$_ | Stop-VMHostService -Confirm:$false | Out-Null
Write-Host "$($_.Label) on $VMHost stopped"
else {Write-Warning "$($_.Label) on $VMHost already stopped"}
else {
$VMHost | Get-VMHostService | Where {$_.Key -eq "TSM-SSH"} | %{
if ($_.running -eq $true) {
$_ | Stop-VMHostService -Confirm:$false | Out-Null
Write-Host "$($_.Label) on $VMHost stopped"
else {Write-Warning "$($_.Label) on $VMHost already stopped"}
default {throw "No valid type for parameter -VMHost specified"}

These functions were still in my profile so I put them on the site but were not created by me, these are only needed to enable/disable SSH.

function Get-HP{
#Help file


try {
$Hosts = Get-VMHost $VMhostName
if ($Status -eq $true){
#######Check for HP-AMS Provider Status #######
foreach ($VMHost in $Hosts){
$ESXCLI = Get-EsxCli -VMHost $VMHost
$HP = $ESXCLI.software.vib.list() | Where { $_.Name -like "hp-ams"} | Select @{N="VMHost";E={$ESXCLI.VMHost}}, Name, Version
if ($HP.name -eq "hp-ams"){
if($Hosts.Model -match "Gen8"){
Write-Host -fore Green "HP-AMS Provider found on" $HP.VMhost $hosts.model "Version:" $HP.version
else {
Write-Host -fore Red "Please remove HP-AMS Provider found on" $HP.VMhost $hosts.model "Version:" $HP.version
Write-Host -ForegroundColor Red "No HP-AMS Provider found on $HP.VMhost $hosts.model"
elseif ($Remove -eq $true){
#######Remove option#######
# Maintenance mode check

Write-Host "Checking Maintenance mode"
if ((Get-VMHost $hosts | select ConnectionState).Connectionstate -ne "Maintenance")
{throw "Put host in maintenance mode please"}
Write-Host -ForegroundColor Green "Maintenance mode OK"
#2 Enable SSH
Enable-TSM $Hosts
if ((Get-VMHostService -VMHost $Hosts|?{$_.key -eq "TSM-SSH"}).running -eq "True")
{Write-Host -ForegroundColor Green "SSH running succesfull"}
{Write-Host -ForegroundColor Red "SSH failed starting"}

#3     HP Service stoppen middels Plink actie
# Creating alias for plink and test path
if (-not (test-path "D:\Putty\plink.EXE")) {throw "D:\Putty\plink.EXE needed"}
set-alias plink "D:\Putty\plink.EXE"
$Str1 = 'echo Y | plink -pw Password -l root '
$Stop = ' /etc/init.d/hp-ams.sh stop'
$Server = $hosts.name
$command= $str1+$server+$Stop
$output = Invoke-Expression -Command $command

#4Verwijderen HP service
Write-Host "Starting removal"
$Str2 = 'plink -pw Password -l root '
$Remover = ' esxcli software vib remove -n hp-ams'
$command= $str2+$Server+$Remover
$output1 = Invoke-Expression -Command $command
if ($output1 -like "*successfully*"){
Write-Host -ForegroundColor green "Removal completed succesfully"
if ($output1 -like "*reboot*")
Write-Host -ForegroundColor Yellow "Reboot required and starting now"
Restart-VMhost -VMHost $Hosts -Confirm:$false|Out-Null
Write-Host -ForegroundColor Yellow "Restart started"
write-host "Possible dry-run?"
Else {
if ($output1 -like "*NoMatchError*"){
Write-Host "Nothing to do already removed probably restart required";Disable-TSM $Hosts

else {
Write-Host "No switch parameter found, use -remove or -status";Disable-TSM $Hosts
catch {throw}


-status : Checks the status of the host, is the agent installed and which model is the host.

-remove : Checks if host is in maintenance mode, stops the HP-AMS service, uninstalls the HP-AMS service and restarts the VMhost


Example for a DL585G5

get-hp -VMhostName esx1.net –Status
Please remove HP-AMS Provider found on esx1.net ProLiant DL585 G5 Version: 500.10.0.0-18.434156

Example for a DL380 Gen8

get-hp -VMhostName esx2.net -Status
HP-AMS Provider found on esx2.net ProLiant DL380p Gen8 Version: 500.10.0.0-18.434156

#Remove exection (need to paste)

Add or subtract date/time in Excel and Powershell


Manipulate the date

Let’s say we have a field with a date, for example I take an easy date like : 1-1-2011 I put it in field A1
Now how to add/subtract this value in another field.

Depending on your language :

Enter the following formula to create a new date property field (of course another field)


1-1-2014 1-1-2014

So now we have 2 fields with the same date. Now let’s add something to the original date.

For this example I add +5 to the Year, Month and Day.


This results in :

1-1-2014 6-6-2019

The same trick can be used to only edit one property of course, to subtract we use -5


1-1-2014 27-7-2008

Manipulate the time

As you can see it’s pretty easy the same can be done with the time property





Let’s create a date/time variable

$date = get-date

The $date variable contains a lot of properties

PS C:\> $date|select *
DisplayHint : DateTime
DateTime    : maandag 3 november 2014 14:02:14
Date        : 3-11-2014 0:00:00
Day         : 3
DayOfWeek   : Monday
DayOfYear   : 307
Hour        : 14
Kind        : Local
Millisecond : 447
Minute      : 2
Month       : 11
Second      : 14
Ticks       : 635506201344476042
TimeOfDay   : 14:02:14.4476042
Year        : 2014

Now we can use the “add Methodes to manipulate the $date variable”

PS C:\> $date|gm
TypeName: System.DateTime

Name                 MemberType     Definition
----                 ----------     ----------
Add                  Method         datetime Add(timespan value)
AddDays              Method         datetime AddDays(double value)
AddHours             Method         datetime AddHours(double value)
AddMilliseconds      Method         datetime AddMilliseconds(double value)
AddMinutes           Method         datetime AddMinutes(double value)
AddMonths            Method         datetime AddMonths(int months)
AddSeconds           Method         datetime AddSeconds(double value)
AddTicks             Method         datetime AddTicks(long value)
AddYears             Method         datetime AddYears(int value)


As you can see there are a lot things that can be changed.

Manipulate the date

With the example of the Excel commands I just add +5 to the Year,Month,Day

woensdag 8 april 2020 14:02:14

Or to subtract use

vrijdag 29 mei 2009 14:02:14

Manipulate the time

This can be done the sameway like described above

maandag 3 november 2014 19:07:19

Or to subtract use

maandag 3 november 2014 8:57:09

Hopefully someone finds it useful.

ESXi CPU Status Demand/Usage/Ready?

ESXi CPU Status Demand/Usage/Ready?

Demand: Amount of CPU the virtual machine is demanding / trying to use
Usage: Amount of CPU the virtual machine is actually currently being allowed to use
Ready: Amount of time the virtual machine is ready to run but unable to because vSphere could not find physical resources to run the virtual machine on.

Virtual machines can be in any one of four high-level CPU States:
Wait: This can occur when the virtual machine’s guest OS is idle (Waiting for Work), or the virtual machine could be waiting on vSphere tasks. Some examples of vSphere tasks that a vCPU may be waiting on are either waiting for I/O to complete (Blocked) or waiting for ESX level swapping to complete (SWPWT). These non-idle vSphere system waits are called VMWAIT.
Ready (RDY): A vCPU is in the Ready state when the virtual machine is ready to run but unable to run because the vSphere scheduler is unable to find physical host CPU resources to run the virtual machine on. One potential reason for elevated Ready time is that the virtual machine is constrained by a user-set CPU limit or resource pool limit, reported as max limited (MLMTD).
CoStop(CSTP): Time the vCPUs of a multi-way virtual machine spent waiting to be co-started. This gives an indication of the co-scheduling overhead incurred by the virtual machine.
Run: Time the virtual machine was running on a physical processor

ESXi Setting syslog and firewall settings with PowerCLI

Syslog and firewall configuration with PowerCLI

Setting PowerCLI

Due the arrival of some SIEM solution I needed to reconfigure ESXi hosts to not only point to our Kiwi Syslog server, but also to the appliance. So a good job for some PowerCLI

I had some trouble using the set-VMHostSysLogServer as it didn’t seem to work as expected. It worked on 2 hosts which hadn’t any syslog configured, but somehow I couldn’t set all to $Null or to the new value, very strange. But I don’t give up and found the Set-VMHostAdvancedConfiguration cmdlet to set the syslog values on another way.

get-vmhost| Set-VMHostAdvancedConfiguration -NameValue @{'Syslog.global.logHost'='syslog'} -confirm:$false

While testing I noted the message:

This cmdlet is deprecated. Use New-AdvancedSetting, Set-AdvancedSetting, or Remove-AdvancedSetting instead.

Mmm let’s have a look here:

get-vmhost|select -first 1|get-advancedsetting -Name syslog* |select name,value|Ft -a

Name                                        Value
—-                                           —–
Syslog.Remote.Port                   514
Syslog.Remote.Hostname          syslog
Syslog.Local.DatastorePath        [] /vmfs/volumes/4dd2476c-etc.

Let’s try to set it

Get-AdvancedSetting -Entity (get-vmhost|select -first 1) -Name Syslog.Remote.Hostname|Set-AdvancedSetting -Value syslog -confirm:$false

You also can set multiple values like:

Get-AdvancedSetting -Entity (get-vmhost|select -first 1) -Name Syslog.Remote.Hostname|Set-AdvancedSetting -Value syslog1,syslog2 -confirm:$false

After setting the proper syslog setting it was necessary to open the syslog firewall ports on ESXi. To do this on all hosts, it can easily be done with the onelinerbelow using the Get-VMHostFirewallException cmdlet

Get-VMHostFirewallException -VMHost (get-vmhost) -Name syslog|Set-VMHostFirewallException -Enabled:$True -Confirm:$false

MiFi Vodafone R212 and Bliep

MiFi Vodafone R212 and Bliep

Vodafone R212-Z

I purchased a Vodafone R212-Z to build a WiFi hotspot.

The device can be ordered referring to the link below, when you order, remove the subscriptions or contracts so only the device is left. The device costs €65,29 (ex BTW) and €79,00 with BTW included.You can connect up to 10 WiFi devices.



The Provider

To make an internet connection I ordered  a Bliep* Sim. The Vodafone R212 uses a normal SIM slot, so be sure to order a standard SIM. With this provider it is possible by turning data on/off per day for the costs of €0,50 or €1 for one day.

€0,50 Rate  = 384 Kbps download and 64 Kbps upload
€1,00 Rate  = 3,6 Mbps download and 1 Mbps upload (max of 2 GB each month)

If you reached the maximum rate, it will turn down to the cheaper speed of 384/64 Kbps

After one workday I received both items in my mail and tried to start with them. First problem was to activate the SIM card, it should be done by putting it in a telephone and next up dial number 1350 to activate the SIM card. Damn I don’t have a phone any more to activate it. Searched the internet if it was available online or through a SMS but too bad couldn’t find anything that worked. Luckily in our office our telecom apartment had some old phones around which could be used to activate it.


After the card was activated I put it in my Vodafone R212-Z and booted it. Nice SIM card found and a standard hotspot is created. The login details are available on the “in the box” instructions. But can also be found on the back of your device or by double clicking the “Power” button.


The next step is to log in to the management interface, connect first to the WiFi hotspot with the credentials provided, next take a device and go to here you can enter a  SIM code to unlock the SIM card. Do this if needed.

Login as admin by filling in the standard password “admin”. Of course it is wise to change this password, but feel free to change the SSID and password as well. Remember that (even if you changed it) you can double tap the devices power button to see the SSID and password to connect with.

Go to account settings and change the APN to “internet.arpa” and number “0”.

Normally other providers use prefixes like *99# or *99***1, but somehow this doesn’t.


How did I found this? Because I couldn’t find it on the internet/forums etc.

Well on the support site is a link to an Apple APN configuration file.


Opening this file with an editor it will show a sort of XML file, when digging trough I found these keys



APN  = internet.arpa
Proxyport = 0

After filling this in, all works flawlessly.

Hope I could help someone with these settings.

There are also a few sites with the most common APN settings like :



*Update 5-10-2014

After 2 days I noticed that the connection was dropped sometimes and didn’t re-establish by itself, a collegaue told me sort-like issues can be solved by installing the newer firmware on the device.

Firmware Update 4.5 can be found here ; https://www.vodafone.nl/_assets/downloads/software/R212-Z.exe

After running the installation I get an error message that the device cannot be found, damn, maybe I have to install the drivers then.

So next I downloaded the “Always Best Connected Dashboard” waited for the installation to finish and ran the firmware after the Dashboard stuff is installed.

Right, now the device is found and will be updatet….there you go..

Will update later what my experiences are with the new firmware

*Update 15-10-2014

After installing the latest firmware, I had to set the settings again. I now tried using intranet.arpa and number *99#
And connected again….yehaaa! It works now, also I don’t have any troubles manually connecting each time the device was powered off. It’s now just a matter of pushing the button, waiting for signal and go !

Powershell logging function with date and time stamp

Logging Function

While I made this function a few times on demand with different purposes, I gathered a nice clean script somewhere (not sure where I found it) to start logging to a file.
I added the date/time stamp together with a write-host command to get the logging also on screen. This is a script I know always copy in my scripts to get a fast and simple logging function


#Variable to set the logfile location
$Logfile = "D:\LogFile.txt"

Function LogWrite{
Simple function to write to a appending logfile with date and time stamp

.PARAMETER  $LogString
Input parameter which holds the data which goes to the $logfile

.PARAMETER  $Logfile
Parameter which contains the location of the logfile the information is written to

LogWrite "Starting with $StrTitle files"
LogWrite "Found $StrTitle files with an amount of $TotalSize MB."

Version:        1.0
Creation Date:  26-08-2014
Purpose/Change: PRD
Param ([string]$logstring)
Add-content $Logfile -value (((Get-Date).ToString()) + " " + $logstring)
Write-Host (((Get-Date).ToString()) + " " + $logstring)

Exection is as simple as :

logwrite "This line is being sent to screen and text file while a date & time stamp is added"

Powershell shows:

PowerCLI C:\> logwrite "This line is being sent to screen and text file while a date & time stamp is added"
30-8-2014 10:05:52 This line is being sent to screen and text file while a date & time stamp is added

And the text file:

30-8-2014 10:05:52 This line is being sent to screen and text file while a date & time stamp is added

Hopefully it can help someone just copy/paste and adjust where needed:)

Locked files ESXi5.1

Locked files 101

Locked files ?

Since  a few months we see more and more “consolidation needed messages” in our vCenter environment, we’re still searching for the reason. Of course we used some VMware articles to get in a bit deeper. But for a successful we need to remove the file lock. I made a little procedure which worked best for us. The information also can be found in the VMware KB’s but sometimes it’s a bit confusing because you have different approaches in different versions.

Investigating virtual machine file locks on ESXi/ESX (10051)Unable to delete the virtual machine snapshot due to locked files (2017072)
Unable to perform operations on a virtual machine with a locked disk (1003397)
Investigating hosted virtual machine lock files (1003857)

Find locked files

It’s easiest to power off the VM. So you’re sure there shouldn’t be any lock anymore. If you’re not sure or have a lot of disks it can be a b*tch to check for locks using the vmkfstools -D. It’s doable, but I found a easier way using the touch command.

Start a SSH session to one of the ESXi hosts in the cluster where the VM resides you want to investigate.

Move to the location of the VM on the datastore: # cd /vmfs/volumes/<datastore>/<VMname>
Now use the touch * command to change the datestamp of all file in this directory. If you have  a lock you will get output like :

touch: VM1-flat.vmdk: Device or resource busy
touch: VM1_3-flat.vmdk: Device or resource busy

Who owns me?

We know now that the 2 files above are in some kind of lock and cannot be written to. Now we can move on using the vmkfstools command to see which ESXi host owns the file.

# vmkfstools -D VM1-flat.vmdk
Lock [type 10c00001 offset 13957120 v 448, hb offset 3510272
gen 41, mode 2, owner 00000000-00000000-0000-000000000000 mtime 845230
num 1 gblnum 0 gblgen 0 gblbrk 0]

We now see 2 things Mode 2 which indicates the kind of lock. See the list below with the kind of modes there can be:

  • mode 0 = no lock
  • mode 1 = is an exclusive lock (vmx file of a powered on VM, the currently used disk (flat or delta), *vswp, etc.)
  • mode 2 = is a read-only lock (e.g. on the ..-flat.vmdk of a running VM with snapshots)
  • mode 3 = is a multi-writer lock (e.g. used for MSCS clusters disks or FT VMs)

We see mode 2 which indicates a file lock. Normally there also is an owner, but this shows only zeroes so we can’t use this information.

In the few lines below we see this information about the Read Only Owner:

RO Owner[0] HB Offset 3510272 53baaace-0e95cf36-46a4-441ea15dea84

Addr <4, 5, 119>, gen 269, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 34359738368, nb 32768 tbz 7056, cow 0, newSinceEpoch 32768, zla 3, bs 1048576

The read only owner of this lock is MAC address 441ea15dea84 which converts to 44:1e:a1:5d:ea:84.

Find the MAC

A fast and easy way to find the corresponding MAC address is with powerCLI. Because we know the cluster where it happens I came up with the command line below:

get-cluster *001*|Get-VMHost|Get-VMHostNetworkAdapter|where {$_.Mac -eq "<strong>44:1e:a1:5d:ea:84</strong>"}|select vmhost,,mac

VMHost                                                                                  Mac
——                                                                                          —
ESX.host.net                              44:1e:a1:5d:ea:84
Now we put this host in maintanance and reboot. Normally the file lock should be gone now and we can consolidate the VM.

Warning: Failed to connect to the agentx master agent (hpilo:)

Warning: Failed to connect to the agentx master agent (hpilo:)


While doing by adminstrative tasks, I was wandering through the syslogs and I noticed a system returning the message below every few minutes.

hpHelper[14674]: Warning: Failed to connect to the agentx master agent (hpilo:)

Mmmz let’s try to reboot the system, this was no solution. I checked the settings of the ILO board of this system with a similar system (both HP DL380 Gen8) and saw some differences. One system without the problems was set to “agentless management” the other had some SNMP settings. I removed the SNMP settings and put it to agentless like the host without problems.

After that I restarted all the services to make sure it would reset the connection. To bad it didn’t work either.


With a little search I found this site.
This triggered me..might not be the problem on ESX side, but maybe on ILO side. So I resetted the ILO board and so the connection reinitialized. The AgentX messages disappeared and the problem seems to be solved.


Slow mouse performance Windows Server 2012 VM

Slow mouse ? Windows Server 2012 RDP ?

Recently we deployed some Windows 2012 server VM’s. Somehow I noticed that in the console but also RDP the mouse lags and is slow. Apparently I’m some one who notices this pretty quick, after a quick check I saw VMware tools where installed properly, correct drivers are loaded so this shouldn’t cause the problem.

After a little google search I found a blog with a simple solution we probably forgot while deploying the 2012 machines. It was just the little mouse pointer shadow :S

How do I turn this off ?

Go to the “control panel” and open “System”

Open “Advanced system settings and then click the “Settings” button at the performance field. In the tab “Visual effects” you see an option “Show shadows under mouse pointer”. After disabling this, the performance of the mouse was back to normal again.

I also decided to set the settings to “Adjust for best performance” because this will decrease any other almost not noticable lag 🙂

Migrating SQL HP SIM Database

Migrating SQL HP SIM Database

When migrating the HP SIM SQL database from 2005 to 2008 R2 we needed to change the database connection so there was no need to reinstall the entire application.

1) Stop the HP SIM service

– Start -> Run -> Services.msc- Stop the HP Systems Insight Manager


2) Go to the folder  “C:\program files\HP\Systems Insight\Manager\config”3) Open the file database.admin with a text editor

4) Edit the settings to with the new database properties

Normally hp.Database.hostName, hp.Database.instance, hp.Database.portNumber would be enough.

#--- Product Property Definitions ---
#Wed Dec 18 17:01:23 CET 2013

Save the file after editing

5) Open the file database.props with a text editor in the same directory as the database.admin file.
Edit the same properties as in the database.admin file. hp.Database.hostName, hp.Database.instance, hp.Database.portNumber

#--- Product Property Definitions ---
#Wed Dec 18 17:01:23 CET 2013

Save the file after editing

6) Go to the folder “C:\Program Files\HP\Systems Insight Manager\jboss\server\hpsim\deploy”

Search for files which end with *-ds.xm, open these and edit the host name here. In my example there only was one file hpsim-ds.xml in this file is a complete connection string which needs to be adjusted also. Open the file and search for the connection string:


It’s also possible that there are files like tmpXXXXXpmp-ds.xml  in the folder “\jboss\server\hpsim\tmp\deploy\” those need to be updated also.

7) Start the HP SIM Service again

Now wait a few minutes so everything can start, now check if the application works properly.

1 2 3 4 5 8