VMware vCenter Server + PSC Appliance 6.x deployment

Choices ?

Due the expansions of the limitations in the vSphere appliances, it now could be a worth replacing the Windows vCenter servers by appliances. Because I’m reasearching the upgrade to 6.x I need to make a decision to go for Windows or the Appliance. Too bad VMware’s Update manager is still a Windows-only product, but ok, I could live with that.

Because we have two sites I normally go for 2 single vCenter installations with their own databases, SSO, VUM etc. So they will be separate entities, no matter what happens on one of the sites, there is no relation, so the surviving one will keep working.

When digging through the new documentation and install guides there are basically a few major decisions to make. Because it’s not able to change this after deployment you have to think beforehand and
make the right decisions.

Below are a few “Major” choices for your design which I will explain later from my point of view. There are several good blog posts on the internet already.
Choice 1 : Embedded or external Platform Service Controller
Choice 2 : Internal or external Database
Choice 3 : One or more vCenters
Choice 4 : Windows, appliance or both ?
Choice 5 : New install or upgrade

Which are beautifully explained in the install guide:
http://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vsphere.install.doc%2FGUID-ACCD2814-0F0A-4786-96C0-8C9BB57A4616.html

At the moment I’m not going to explain or discuss the decisions and facts I used to choose for this setup.

Design6As you can see I chose to setup 1 vCenter per site (like I used to do before) and made a local external PSC for each site and connected them in the same “Site”. So basically the PSC’s will replicate with each other across the sites, while the vCenters are local.

What does that PSC do actually? Well it handles vCenter Single Sign On, vSphere License Service, VMware Certifcate Authority. Those components where previously part of the vCenter Installation, by seperating them, it now is also possible to link both PSCs.

Because I join the second PSC to the same site it’s possible to use Enhanced Linked Mode out of the box. The site below will explain what it means. It’s awesome that roles/permissions etc. now can be managed from a single point.
http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vsphere.install.doc/GUID-91EF7282-C45A-4E48-ADB0-5A4230A91FF2.html

Migrate Windows vCenter Server to vCenter Appliance?

Yes it’s  possible, haven’t tried it yet, but came accross this fling which let’s you migrate a Windows vCenter server with an external SQL Database to a vCenter Appliance with Embedded vPostgres Database

https://labs.vmware.com/flings/vcs-to-vcva-converter

Design  & Installation

For my testing purposes I decided to deploy 4 appliances, 2x vCenter server and 2x Platform Service Controller.

Instead of using the CD-ROM and follow the instructions (which I did already a few times), I decided to create a simple deployment script. It is possible to deploy the appliances with a pretty simple script and configuration file. Let’s dive in here:

You”ll need the installation CD and mount that to a Windows machine where you start the deployment using the deploy command which is located on the CD-ROM.

Secondly you need a few .json files which hold the configuration for the VCSA deployment.

To follow the example as in the picture I have 4 .json files, 2 for the vCenters and 2 for the PSC controllers.

Let’s start setting up the PSC Controllers and SSO Site, choose 1 to deploy first, I take PSC_Site1

The first part is the deployment information, what is the hostname where the PSC needs to be deployed too, fill in the username,password,datastore,network etc.

Next is the VCSA information, what is the root password of the appliance, does SSH need to be enabled and

After that there are some seperate configuration options for setting up the SSO domain. Because it’s the first one, the option “first-instance” is set to true, the rest looks self explanatory so fill it in like you designed it.

PSC Site 1 JSON

{
"__comments":
[
"Template to deploy PSC Controller Site1."
],

"deployment":
{
"esx.hostname":"ESX_Site1",
"esx.datastore":"Datastore_Site1",
"esx.username":"root",
"esx.password":"VMWare!",
"deployment.network":"VLAN1234",
"deployment.option":"infrastructure",
"appliance.name":"PSC_Site1",
"appliance.thin.disk.mode":false
},

"vcsa":
{
"system":
{
"root.password":"VCSA_VMWare!",
"ssh.enable":true
},

"sso":
{
"password":"SSO_VMWare!",
"domain-name":"vsphere.local",
"site-name":"vSphere_Test",
"first-instance":true
},

"networking":
{
"ip.family":"ipv4",
"mode":"static",
"ip":"192.168.1.214",
"prefix":"24",
"gateway":"192.168.1.254",
"dns.servers":"192.168.1.203",
"system.name":"SSO_Site1"
}
}
}

Nice, now we have the configuration file for the first External SSO server in the domain. In the JSON directory on the CD are some examples for the different installations. We now need to build the second external PSC controller on a different host in a different physical site, but still need to connect it to the primary site which is created above.

PSC Site 2 JSON


{
"__comments":
[
"Template to deploy PSC Controller Site2."
],

"deployment":
{
"esx.hostname":"ESX_Site2",
"esx.datastore":"Datastore_Site2",
"esx.username":"root",
"esx.password":"VMWare!",
"deployment.network":"VLAN1234",
"deployment.option":"infrastructure",
"appliance.name":"PSC_Site2",
"appliance.thin.disk.mode":true
},

"vcsa":
{
"system":
{
"root.password":"VCSA_VMWare!",
"ssh.enable":true
},

"sso":
{
"password":"SSO_VMWare!",
"domain-name":"vsphere.local",
"site-name":"vSphere_Test",
"replication-partner-hostname":"192.168.1.214"
},

"networking":
{
"ip.family":"ipv4",
"mode":"static",
"ip":"192.168.1.215",
"prefix":"24",
"gateway":"192.168.1.254",
"dns.servers":"192.168.1.203",
"system.name":"192.168.1.215"
}
}
}

As you can see, you configure the replication partner and site name where the second PSC needs to connect/register too.

Good, this will later on deploy the upper part of the picture and configure and SSO Domain which exists of 2 PSC Servers.

vCenter JSON

Let’s configure 2 vCenters, because the configuration is just the same, I’ll post only 1 example. I configure vCenter Site 1 here:


{
"__comments":
[
"Template to deploy vCenter Server Site1."
],

"deployment":
{
"esx.hostname":"ESX_Site1",
"esx.datastore":"Datastore_Site1",
"esx.username":"root",
"esx.password":"VMware!",
"deployment.network":"VLAN1234",
"deployment.option":"management-tiny",
"appliance.name":"vCenter_Site1",
"appliance.thin.disk.mode":true
},

"vcsa":
{
"system":
{
"root.password":"VCSA_VMware!",
"ssh.enable":true,
"platform.service.controller":"192.168.1.214"
},

"sso":
{
"password":"SSO_VMware!",
"domain-name":"vsphere.local",
"site-name":"vSphere_Test"
},

"networking":
{
"ip.family":"ipv4",
"mode":"static",
"ip":"192.168.1.216",
"prefix":"24",
"gateway":"192.168.1.254",
"dns.servers":"192.168.1.203",
"system.name":"192.168.1.216"
}
}
}

The second vCenter configuration file is the except the relation to the PSC which connects to the “local PSC”.
“platform.service.controller”:”192.168.1.215″

So let’s wrap it up, I have 4 .json files which hold the configuration of 1 external PSC controller, which is the first of the site. Then a second PSC is deployed which is connected to the first PSC “Site”.

After that we deploy two vCenters which connect to the PSC Site within their own site.

Good that’s a start. On the Installation CD there is  a file called  : vcsa-deploy.exe which can be found here Z:\vcsa-cli-installer\win32\vcsa-deploy.exe

All command line options an configuration can be found here:

http://www.vmware.com/files/pdf/products/vsphere/VMware-vsphere-60-vcenter-server-appliance-cmdline-install.pdf

Now let’s create a simple deployment batch file which contains the installation.

As you can see, you start the vcsa-deploy.exe, with the json file we created, save the log file to a directory and turn off the SSL verification.


Z:\vcsa-cli-installer\win32\vcsa-deploy.exe D:\JSON\PSC_Site1.json --log D:\JSON\PSC_Heerlen_install.log --no-esx-ssl-verify --verbose
Z:\vcsa-cli-installer\win32\vcsa-deploy.exe D:\JSON\PSC_Site2.json --log D:\JSON\PSC_Beek_install.log --no-esx-ssl-verify
Z:\vcsa-cli-installer\win32\vcsa-deploy.exe D:\JSON\VC_Site1.json --log D:\JSON\VC_Heerlen_install.log --no-esx-ssl-verify
Z:\vcsa-cli-installer\win32\vcsa-deploy.exe D:\JSON\VC_Site2.json --log D:\JSON\VC_Beek_install.log --no-esx-ssl-verify

Let’s fire it up  :

deplot

Cool let’s keep it running until it ‘s finished. I experienced that after it’s finished there seem to be some background tasks, so let it run for a few more minutes to let it sync and setup correctly.

Done?

Now let’s go to the webclient: https://192.168.1.216/vsphere-client

And login with “Administrator@vsphere.local” and the password you provided in the configuration. You’ll now see your 2 vCenters connected in the same interface.

Ah pretty, now let’s see what we can do.

For example go to the “Adminstration” tab, here we can see “Global permissions, licensing etc.”What about cross  vMotion to another datacenter, I read about that..let’s see:

Right click a virtual machine and select the “Migrate” option, choose the option to “Change both compute resource and storage” (because you don’t have the same shared storage on the other side, at least I don’t). Cool, now I can select other vCenters and underlying resources. Select the resource you want to move to, select the datastore, select another network if needed and click “Finish”.

Awesome that worked flawless.

I’m still doing some more testing and will update the text and pictures asap. This was a quick reference & braindump

 

Root disk full WTF?

Root disk full  WTF?

Suddenly somehow we got a virtual machine which couldn’t be powered on, vCenters events showed messages like:

Power On virtual machine
<Computer>
A general system error occurred: Unknown error

Mmm strange, but seen it before. I used the articles in each section to find the solution

Part 1

I started to look at the /var/spool/snmp directory, but this one was empty so it could not be the cause of filling the /root disk.

  1. Connect to the ESXi host using SSH. For more information, see Using ESXi Shell in ESXi 5.0 and 5.1 (2004746).
  2. Check if SNMP is creating too many .trp files in the /var/spool/snmp directory on the ESXi host by running the command:ls /var/spool/snmp | wc -lNote: If the output indicates that the value is 2000 or more, this may be causing the full inodes.
  3. Delete the .trp files in the /var/spool/snmp/ directory by running the commands:# cd /var/spool/snmp
    # for i in $(ls | grep trp); do rm -f $i;done
  4. Change directory to /etc/vmware/ and back up the snmp.xml file by running the commands:# cd /etc/vmware
    # mv snmp.xml snmp.xml.bkup
  5. Create a new file named snmp.xml and open it using a text editor. For more information, see Editing files on an ESX host using vi or nano (1020302).
  6. Copy and paste these contents to the file:<?xml version="1.0" encoding="ISO-8859-1"?>
    <config>
    <snmpSettings><enable>false</enable><port>161</port><syscontact></syscontact><syslocation></syslocation>
    <EnvEventSource>indications</EnvEventSource><communities></communities><loglevel>info</loglevel><authProtocol></authProtocol><privProtocol></privProtocol></snmpSettings>
    </config>
  7. Save and close the file.
  8. Reconfigure SNMP on the affected host by running the command:# esxcli system snmp set –-enable=true
  9. To confirm the SNMP services are running normally again, run the command:# esxcli system snmp getHere is an example of the output:
    /etc/vmware # esxcli system snmp get
    Authentication: Communities: Enable: true Engineid: 00000063000000a10a0121cf Hwsrc: indications Loglevel: info Notraps: Port: 161 Privacy: Remoteusers: Syscontact: Syslocation: Targets: Users: V3targets:

But to be sure I disabled SNMP like mentioned in the article.
ESXi 5.1 host becomes unresponsive when attempting a vMotion migration or a configuration change (2040707)

Part 2

In the first article there is also a hint that HP Gen8 hardware could have similar issues with another resolution. HAH! We use that hardware, so I went on to remove the hpHelper.log file. With the steps below

To remove the hpHelper.log file:

  1. Log in to the ESXi host as the root user.
  2. Stop the HP Helper management agent by running the command:/etc/init.d/hp-ams.sh stop
  3. To remove the hpHelper.log file, run the command:rm /var/log/hpHelper.log
  4. To restart the HP Helper management agent, run the command:/etc/init.d/hp-ams.sh start

ESXi ramdisk full due to /var/log/hpHelper.log file size (2055924)

After I started everything and mentioned the HPHelper.log file was super small I tried again. DAMN! Same error, guess there is something else which is filling the root disk, so my search continues.

Part 3

Somehow I got a clear moment and thought about the sfcdb-watchdog service. After a search with sfcbd and root disk full I found this article :

ESXi 5.x host is disconnected from vCenter Server due to sfcbd exhausting inodes (2037798)

With the help of the command vdf -h I noticed that there was only 8 MB Free on /root

Ramdisk                   Size      Used Available
root                           32M        24M         8M
etc                             28M     224K     27M
tmp                         192M       40K    191M
hostdstats            834M        3M    830M

Mmm that’s not much.

To free up inodes:
  1. Connect to the ESXi host using SSH.
  2. To stop the sfcbd service, run the command:/etc/init.d/sfcbd-watchdog stop
  3. Manually delete the files in the var/run/sfcb directory to free inodes.To remove the files, run the commands:cd /var/run/sfcb
    rm [0-2]*
    rm [3-6]*
    rm [7-9]*
    rm [a-c]*
    rm [d-f]*
  4. To restart the sfcbd service, run the command:/etc/init.d/sfcbd-watchdog startNote: You may need to restart management agents after step 4. For more information, see Restarting the Management agents on an ESXi or ESX host (1003490).

After doing this I had like 30MB free on /root.

This issue should be resolved in update below. Unfortunatly it was not possible for us to update these hosts before with the latest patches.

To manually resolve this issue, you can prevent the root file system from being impacted

  1. Connect to the ESXi host using SSH.
  2. Run this command to stop the sfcbd service:/etc/init.d/sfcbd-watchdog stop
  3. Enable this command to be run at boot time:esxcli system visorfs ramdisk add –name sfcbtickets –min-size 0 –max-size 1024 –permissions 0755 –target /var/run/sfcbFor more information, see Modifying the rc.local or local.sh file in ESX/ESXi to execute commands while booting (2043564).
  4. Run this command to start the sfcbd service:/etc/init.d/sfcbd-watchdog start
  5. Run this command and verify if the new ramdisk exists:esxcli system visorfs ramdisk list
Note: This issue may also occur if the SNMP agent is enabled. For more information, see vCenter Server and vmkernel logs report the message: The ramdisk ‘root’ is full (2033073).
After doing this I was able to succesfully start up the virtual machine.
Now it’s time to make sure we do the updates asap!
Another handy article to gather file system information

CDP settings ESXi host

On an ESX host you could use the command below to display the CDP setting on a switch


<strong>esxcfg-vswitch</strong> [options] [vswitch[:ports]]

#Removed the settings that don't mention the ESXi settings.
-B|--set-cdp                Set the CDP status for a given virtual switch.
To set pass one of "down", "listen", "advertise", "both".
-b|--get-cdp                Print the current CDP setting for this switch.
-h|--help                   Show this message.

~ #
-b|--get-cdp                Print the current CDP setting for this switch.
~ #  esxcfg-vswitch -b vsw-vms01
listen
~ #  Setting the CDP settings to "Down"
~ #  esxcfg-vswitch -B down vsw-vms01
~ #  esxcfg-vswitch -b vsw-vms01
down

 

With Powershell you could instead make this function and use this like:

get-cdp -vmhost “MyHost”


function get-cdp {
<#
.SYNOPSIS
Grab CDP info from NIC's who are connected
.DESCRIPTION
This function greps some CDP info from the switch
.NOTES
Source:  Internet
Authors: XXX
.PARAMETER VMHost
An array of entity names. Only clusters, datacenters or
ESX hosts are allowed.
Wildcards are supported. (mutually exclusive with -dsName)
.PARAMETER whatif
When set, the function will only list output to the console
and not register the found vmx files
.EXAMPLE
PS> get-cdp -vmhost "MyHost"
#>
param($VMHost)
$vmh = Get-VMHost $VMHost
If ($vmh.State -ne "Connected") {
Write-Output "Host $($vmh) state is not connected, skipping."
}
Else {
Get-View $vmh.ID | `
% { $esxname = $_.Name; Get-View $_.ConfigManager.NetworkSystem} | `
% { foreach ($physnic in $_.NetworkInfo.Pnic) {
$pnicInfo = $_.QueryNetworkHint($physnic.Device)
foreach( $hint in $pnicInfo ){
# Write-Host $esxname $physnic.Device
if ( $hint.ConnectedSwitchPort ) {
$hint.ConnectedSwitchPort | select @{n="VMHost";e={$esxname}},@{n="VMNic";e={$physnic.Device}},DevId,Address,PortId
}
else {
}
}
}
}
}
}

DL380/385 Gen8 Install ESXi4.1

Case: Missing NIC and storage drivers in our ESXi4.1 installation

First thing I noticed when the I tried to install ESXi4.1 on the Gen8 series is that our standard (VMware) image wasn’t able to install. Not by our scripted UDA and also manual failed.
With the scripted install I noticed the NIC drivers where not recognized and the hardware was unidentified which resulted in the NIC not connecting to our deployment server to grab the KS.CFG files.
The manual install failed by a missing Storage Adapter driver.

When I used the ESXi4.1 recommended image by HP with all the drivers inserted and loaded, the installation went fine. After the installation I used

esxcfg-scsidevs –l
esxcfg-nics –l

To see what hardware was exactly in the machine.

NIC: Broadcom NetXtreme BCM5719
Storage Array: HP System Array P420I
With this information I found the drivers on the HP site.
Driver: vmware-esx-drivers-scsi-hpsa-400.4.1.0-26OEM.x86_64
Driver : vmware-esx-drivers-net-tg3-400.3.123b.v40.1-1vmw.2.17.00000.x86_64

So ? How do we add this to our PXE deployment?
ESXi4.1 has a sort of local hardware database file with all components and vendors which are supported, and a list of which hardware uses what drivers.
Located in /etc/vmware the files are PCI.IDS and SIMPLE.MAP

PCI.IDS
In this file the hardware database is located, it’s a flat file. If the device ID is recognized during boot, ESX will give the corresponding name to it and it’s later used to load the correct drivers. Luckily I had 2 systems. One system I installed with the HP OEM VMware image, the other was my texting machine. I simply pulled the PCI.IDS file from the HP OEM server and saved it locally for later use.
In our case we are missing the HP Smart Array P420i

Example of PCI.IDS

103c Hewlett-Packard Company
323b Smart Array P222
103c 3350 Smart Array P222
103c 3351 Smart Array P420
103c 3352 Smart Array P421
103c 3353 Smart Array P822
103c 3354 Smart Array P420i
103c 3355 Smart Array P220i
103c 3356 Smart Array P721m

SIMPLE.MAP
This file represents a list of device ID’s and which driver modules they need to load. I also needed to add the correct drivers and modules here. Also in this case I just copied the simple.map file.
Example:
103c:323b 103c:3354 storage hpsa
The first explains the PCI device ID, the second category (storage) the last module is the driver that is loaded (hpsa).

So wrap up. Now the hardware devices can be recognized and the right drivers can be loaded. Too bad we still have older drivers in our image and need to update them first.

Logging in to the HP CD installed servers I figured out which driver versions are needed.

vmkload_mod -l|egrep "tg3|hpsa"
tg3 4 176
hpsa 2 76
~ # vmkload_mod -s hpsa|grep Version
Version: Version 4.1.0-26OEM, Build: 00000, Interface: ddi_9_1 Built on: Jan 19 2012
~ # vmkload_mod -s tg3|grep Version
Version: Version 3.123b.v40.1, Build: 00000, Interface: 9.0, Built on: Apr 3 2012

So now we now which drivers and versions are used to control the hardware.
Creating Custom Image

To inject drivers we can use the VMWare provided vibddi tool on a Linux/CentOS machine.
– Install vibddi

rpm -ivh vmware-esx-vibddi-.i386.rpm

– Installation instructions

Copy image file and extract
– Copy the original ISO using WINSCP to /opt/vmware/vibddi/iso

– Mount the original image:

Mount /var/public/smbmount/esx41iso/VMware-VMvisor-Installer-4.1.0.Update3-custom.iso /var/public/www/esx41i/ESXi41U3Cust –t= iso9660 (rw,loop=/dev/loop1)

– Copy the original imagedd.bz2 to a temporary folder /imagebuild/

cp /var/public/www/esx41i/ESXi41U3Cust/imagedd.bz2 /imagebuild[text]
[text]
total 405929
-r-xr-xr-x 1 root root 11225 Nov 6 22:36 a.z
-r-xr-xr-x 1 root root 2048 Nov 6 22:40 boot000.cat
-r-xr-xr-x 1 root root 2048 Nov 7 15:24 boot.cat
-r-xr-xr-x 1 root root 1162824 Nov 6 22:36 cimstg.tgz
-r-xr-xr-x 1 root root 15283377 Nov 6 22:36 cim.vgz
-r-xr-xr-x 1 root root 7306598 Nov 6 22:36 ienviron.vgz
-r-xr-xr-x 1 root root 312820483 Nov 7 15:14 imagedd.bz2
-r-xr-xr-x 1 root root 67 Nov 7 15:16 imagedd.md5
-r-xr-xr-x 1 root root 971501 Nov 6 22:40 install.vgz
-r-xr-xr-x 1 root root 12241 Nov 6 22:36 isolinux.bin
-r-xr-xr-x 1 root root 298 Nov 6 22:36 isolinux.cfg
-r-xr-xr-x 1 root root 47404 Nov 6 22:36 mboot.c32
-r-xr-xr-x 1 root root 51020 Nov 6 22:36 menu.c32
-r-xr-xr-x 1 root root 2593205 Nov 6 22:36 open_sou.txt
-r-xr-xr-x 1 root root 2156 Nov 6 22:36 readme.txt
-r-xr-xr-x 1 root root 72786024 Nov 6 22:36 sys.vgz
-r-xr-xr-x 1 root root 41155 Nov 6 22:36 tboot.gz
-r-xr-xr-x 1 root root 48339 Nov 6 22:36 vmkboot.gz
-r-xr-xr-x 1 root root 2489796 Nov 6 22:36 vmkernel.gz
-r-xr-xr-x 1 root root 34816 Nov 7 15:23 vmware_v.iso

– Now unzip the imagefile you copied

cd /imagebuild
bunzip2 imagedd.bz2

Adding VIB’s / offline bundles to the image
– Use VIBDDI to see which items are already in the image.

Vibddi –i imagedd –q

– Example:

< oem-vmware-esx-drivers-net-vxge (400.2.0.28.21239-1OEM) >
< oem-vmware-esx-drivers-scsi-3w-9xxx (400.2.26.08.036vm40-1OEM) >
< vmware-esx-firmware (4.1.0-3.26.800380) >
< CMPI-1.0 >
< VMW_CMPI_CPP-1.0.5 >
< vmkapi_1_1_0_0 >
< vmknexus1kvapi-3-26 >
< vmkepsecapi_1.0.0.0 >
< vmkvsepapi_2.0.0.0 >
< DriverAPI-9.0 >
< DriverAPI-9.1 >
< vmkuservsepapi_2.0.0.0 >
< vmware-esx-tools-light (4.1.0-3.26.800380) >

– Now we are going to inject the correct drivers , I downloaded and copied them to /bundles

vibddi -i /imagebuild/imagedd -v cross_oem-vmware-esx-drivers-scsi-hpvsa_400.4.1.0-22OEM.vib
Setting up configuration for esxupdate
This may take a few seconds…

– Repeat this for the other drivers. When you finished use the -q switch again to see if the correct drivers are injected. Now you see that they’re added.


vibddi -i /imagebuild/imagedd –q
< oem-vmware-esx-drivers-net-vxge (400.2.0.28.21239-1OEM) >
< oem-vmware-esx-drivers-scsi-3w-9xxx (400.2.26.08.036vm40-1OEM) >
< vmware-esx-drivers-net-tg3 (400.3.123b.v40.1-1vmw.2.17.00000) >
< vmware-esx-drivers-scsi-qla2xxx (400.841.k1.42.1-1vmw.2.17.00000) >
< vmware-esx-firmware (4.1.0-3.26.800380) >
< CMPI-1.0 >
< VMW_CMPI_CPP-1.0.5 >
< vmkapi_1_1_0_0 >
< vmknexus1kvapi-3-26 >
< vmkepsecapi_1.0.0.0 >
< vmkvsepapi_2.0.0.0 >
< DriverAPI-9.0 >
< DriverAPI-9.1 >
< vmkuservsepapi_2.0.0.0 >
< vmware-esx-tools-light (4.1.0-3.26.800380) >

Custom driver modules
– Download or extract the .o files from the offline bundles/vib files. Luckily I could capture them from the HP VMware OEM CD-ROM.
– The files we need are tg3.o and hpsa.o.
– Go back to your temp directory where your CD image is mounted and mount imagedd

mount imagedd /mnt -o loop,rw,offset=4210688
cd /mnt

total 91368
-rwxr-xr-x 1 root root 11225 Nov 6 22:21 a.z
-rwxr-xr-x 1 root root 128 Nov 6 22:21 boot.cfg
-rwxr-xr-x 1 root root 48339 Nov 6 22:21 b.z
-rwxr-xr-x 1 root root 1162824 Nov 6 22:21 cimstg.tgz
-rwxr-xr-x 1 root root 15283377 Nov 6 22:21 c.z
-rwxr-xr-x 1 root root 2489796 Nov 6 22:21 k.z
-rwxr-xr-x 1 root root 137 Nov 6 22:21 license.tgz
-rwxr-xr-x 1 root root 754246 Nov 6 22:21 m.z
-rwxr-xr-x 1 root root 950084 Nov 6 22:21 oem.tgz
-rwxr-xr-x 1 root root 3862 Nov 6 22:21 pkgdb.tgz
-rwxr-xr-x 1 root root 72786024 Nov 6 22:21 s.z
-rwxr-xr-x 1 root root 41155 Nov 6 22:21 tboot.gz
-rwxr-xr-x 1 root root 2030 Nov 6 22:21 vibddi

– Extract the oem.tgz file and copy the simple.map, pci.ids and .o files in the right directories.
tar xvzf oem.tgz

1) /usr/lib/vmware/vmkmod
Kopieer in deze map de .o files
2) /etc/vmware/
Kopieer naar deze map de sample.map en pci.ids bestanden

– Recreate oem.tgz

tar cvzf oem.tgz ./*

– Copy the oem.tgz back to imagedd

cp oem.tgz /mnt

– Unmount the imagedd

umount /mnt/imaged

– Recompress imagedd file

bzip2 /tmp/imagedd/imagedd

– Check the MD5 checksum and replace it with the checksum in the file : imagedd.md5

md5sum imaged.bz2

Create new ISO

cd /tmp/iso

mkisofs -o VMware-VMvisor-Installer-4.1.0.update1-348481.x86_64_customdriver.iso -b isolinux.bin -c BOOT.CAT-no-emul-boot -boot-load-size 4 -boot-info-table ./

You now have an image that is ready to install with, when you use a manual install.

PXE Boot
As we use PXE here, I added the image, rebooted and….FAIL!

WTH! Oh right…I forgot that the boot image needs the correct drives too, else the NIC can’t connect to the share..

To simply add the drivers to the PXE image I created another OEM.tgz file with the drivers needed for the PXE boot.

– I created a new folder /oem
– I then created the following folders in /oem
1) /usr/lib/vmware/vmkmod
2) /etc/vmware

In the first folder we put the .o files (drivers) and in the second folder the simple.map and pci.ids
Create a a new oem.tgz file to send with the PXE installation.

cd /tmp/oem/
tar –zcf /tmp/oem/oem.tgz etc lib sbin usr var

Copy the oem.tgz file to the location where the PXE files are loaded. In our case this is

/var/public/tftproot

Now add the –oem.tgz to your kernel options command line and you’re ready to fire up !

Good luck and have fun!

Install HP bundles ESX command line

Download the correct bundles from the HP site, I used our UDA as webserver where I uploaded the bundles too. It’s also possible to deploy them using the update manager. To install the HP bundles it’s needed to put the ESX host in maintenance mode. Below I posted some commmands to do this.


#Enter Maint. Mode
vicfg-hostops --server ESXHOST --username ROOT --password ROOT_PASS --operation enter
#Install HP Bundles
vihostupdate --server ESXHOST --username ROOT --password ROOT_PASS -install -bundle http://hp-esxi4.1uX-bundle-1.2-25.zip
vihostupdate --server ESXHOST --username ROOT --password ROOT_PASS -install -bundle http://hp-nmi-bundle-1.2.02.zip
#Reboot host
vicfg-hostops --server ESXHOST --username ROOT --password ROOT_PASS --operation reboot

#Exit maint
vicfg-hostops --server ESXHOST --username ROOT --password ROOT_PASS --operation exit

VMA add SNMP targets, enable and test on ESXhosts 4.1

First login to your VMa!

To add a SNMP target to your ESX host you can use the following command:
ESXhost: Fill in your ESX hostname or IP
ROOT: fill in the root user, or user with enough permissions to set the SNMP traps
ROOT_PASS: password of the user
snmptarget1: the first snmp target you want to send the SNMP traps too
162: port number to traps
community: the community

I divided the commands in two parts, so the first command line sets the targets on the ESX hosts and the 2nd command line enables and sends a test trap.
You can put the -E and -T option behind the first line, but I noticed it isn’t always working flawless.


#Add SNMP target
vicfg-snmp --server <b>ESXHOST </b>--username <b>ROOT </b>--password <b>ROOT_PASS</b> -t <b>snmptarget1</b>@<b>162</b>/<b>community</b>,<b>snmptarget2</b>@<b>162</b>/<b>community</b> -c <b>community</b>
#Enable SNMP TEST
vicfg-snmp --server <b>ESXHOST </b>--username <b>ROOT </b>--password <b>ROOT_PASS </b>-E -T
#Show SNMP settings
vicfg-snmp --server <b>ESXHOST </b>--username <b>ROOT </b>--password <b>ROOT_PASS </b>--show



To set the traps and enable this it’s not needed to put  a host in maintenance mode.
Aferwards, check on your target if the SNMP traps are received. Make sure that your SNMP target is reachable from the ESX hosts.

Because I didn’t want to do this for each server I made a little .SH script which sets this for all servers found in VIFP LISTSERVERS

See example:


vifp listservers|while read a b c; do /usr/bin/vicfg-snmp --server $a -username <b>root </b>--password <b>ROOT_PASS </b>-t snmptarget@162/patrol
vifp listservers|while read a b c;do /usr/bin/vicfg-snmp --server $a -username <b>root </b>--password <b>ROOT_PASS </b> -E -T;done

So now it’s only needed to start  the script to set the SNMP traps on all your servers. It’s also possible trough powerCLI but I tought this was easier.

Also added the SNMP trap settings to our kickstart script. So I’m always sure the SNMP traps are set as soon as a (new) server is installed.