As a person who enjoys weekends and sleep, my favorite friday night cutover is one that lasts approximately 30 seconds. To do this, you need three things:

-Enough resources to temporarily have two of everything
-A portgroup configured with an isolated VLAN
-A staging PC for use as an NTP server (I recommend Meinberg NTP for that)

After building and testing your new environment, you’ll want to “flip the nics” to make the new system live:

Connect-viserver vcenter01.dom.com -User admin -Password pass

Get-Cluster “Old-UC-Cluster” | Get-VM | Get-NetworkAdapter | set-networkadapter -Connected:$false -Confirm:$false

Get-Cluster “New-UC-Cluster” | Get-VM | Get-NetworkAdapter | Set-NetworkAdapter -Portgroup “VLAN 9” -Confirm:$false

This is in a sunny day scenario where everything in the whole cluster is disconnected while everything in another is connected. Sometimes you need to apply some logic to exclude some vms that happen to coexist within the same cluster:

Get-Cluster “New-UC-Cluster” | Get-VM | Get-NetworkAdapter | Where {$_.NetworkName -eq “VLAN 3” -and $_.Name -notlike “CUCMSUB2”} | Set-NetworkAdapter -Portgroup “VLAN 9” -Confirm:$false

UC applications aren’t only supported by Cisco on a virtual platform now, it’s the ONLY supported platform. As a “Collaboration Engineer” by title, I am usually focused on only a handful of applications, traditionally relying on the “Datacenter Guy” to provide the infrastructure and hope it works. But, it’s always good to see the whole picture.

In my opinion, virtualization is something everyone needs to know at least the basics of. You might not be the one adding a vlan to a switch anymore, but you still need to know networking.

Each VM contains two basic files: one .vmx (configuration) and one .vmdk (disc)
The .vmdk (disc) is considerably larger and contains all the bits a physical hard drive would. The .vmx (configuration) is a small, editable file containing all of the settings a physical bios chip would.

A few items you’ll find in a .vmx file:
numvcpus = “2”
memsize = “6144”
scsi0:0.fileName = “UCCX1.vmdk”
ethernet0.networkName = “Voice_VLAN”
guestOS = “rhel4”
sched.cpu.min = “1300”
sched.cpu.units = “mhz”
sched.cpu.shares = “normal”
sched.mem.minsize = “6144”
sched.mem.shares = “normal”
One task any Cisco UC Engineer will go through at least once is an upgrade from pre 8.6 to post 8.6 version. Two changes come about: an OS change from Redhat 4/5 to 6, and adapter change from flexible to vmxnet3.

One method is to edit the .vmx by finding it on the datastore, downloading the file, making the edits below, and import it back:
guestOS = “rhel6_64guest”
ethernet0.virtualDev = “vmxnet3”
An easier method if you are doing multiple servers/clusters at once is to use PowerCLI
Connect-VIServer -Server vcenter01 -User admin -Password pass
Set-ExecutionPolicy RemoteSigned
Set-VM -VM ‘UCCX Pub’ -GuestId “rhel6_64guest” -Confirm:$false
Get-VM ‘UCCX Pub’ | get-networkadapter | set-networkadapter -type vmxnet3

Adding to my post from monday, I had another issue with SRM not changing ip addresses on an Asterisk (CentOS) box, so I had to engineer a workaround..

For PRIMARY = 10.100.1.150 and DR = 10.100.2.150

1. On the linux box, we’ll create two bash scripts, chmod these 755 and stick these in your $PATH.

primary-to-dr.sh
#!/bin/sh
ifconfig eth0 10.100.2.150 netmask 255.255.255.0 up
route add default gw 10.100.2.1 eth0
done

dr-to-primary.sh
#!/bin/sh
ifconfig eth0 10.100.1.150 netmask 255.255.255.0 up
route add default gw 10.100.1.1 eth0
done

2. On the vCenter, install WinSCP and run. Create 2 new sessions with saved credentials and name primary and dr. Connect to each session and accept the certificates!
3. Ensure the following is in the windows PATH, if not, add this directory: C:Program Files (x86)WinSCP
4. Create the batch file below to call winscp, start the session and run the relevant script from step 1.

ifconfig.bat
@echo off
echo “Logging into host and running script”
If EXIST c:primary.txt (winscp primary /command “call primary-to-dr.sh”) ELSE (winscp dr /command “call dr-to-primary.sh”)
echo Exit Value %ERRORLEVEL%
TIMEOUT /t 5 /nobreak

In SRM run the bat file, no flags needed.

VMware advertises the ability to “perform fully automated orchestration of site failover and fallback with a single click.” While this technology is extremely easy to setup and use, it’s oftentimes the small things that cause the biggest headache. SRM uses dns to propogate ip changes from the primary to secondary virtual machines. With windows this is automatic: the servers simply register their new ip addresses in dns. For linux servers, however this is not so easy, especially in my case (an avaya phone system) which provides no access to the CLI. The fix is to create a script.

DNS Server – 192.168.10.1
Primary Site Avaya – 192.168.10.35
Failover Site Avaya- 192.168.15.35

First you’ll want to make sure you are running the SRM service as an account with domain admin rights (assuming you setup the database with windows authentication during setup).

Next you’ll need DNSCMD from the RSAT (remote server administration tools), which you can find here

Below is an example script:

@echo OFF
echo “Removing Records from DNS…”
dnscmd 192.168.10.1 /recorddelete dom.local AVAYA-SRV-01 A /f
echo Exit Value:  %ERRORLEVEL%
TIMEOUT /t 3 /nobreak
dnscmd 192.168.10.1 /recorddelete dom.local AVAYA-SRV-01 A /f
echo Exit Value:  %ERRORLEVEL%
TIMEOUT /t 3 /nobreak

@echo off
echo “Adding Records to DNS ..”
If EXIST c:primary.txt (dnscmd 192.168.10.1 /recordadd dom.local AVAYA-SRV-01 A 192.168.10.35) ELSE (dnscmd 192.168.16.58 /recordadd dom.local AVAYA-SRV-01 A 192.168.15.35)
echo Exit Value:  %ERRORLEVEL%
TIMEOUT /t 5 /nobreak

In SRM, you’ll call the script as:
C:windowssystem32cmd.exe /c c:srmscriptsdnschange.bat

Also, you may notice the “If EXIST c:primary.txt”
It was the most elegant way I could think of the check if the system was in a failover state. This file will exist only on the primary side. Credit to Derek Bickel on this one!

CIPC is a great tool for troubleshooting voice, but it kind of requires a sound card to work, which a VM doesn’t provide and sometimes that’s all you have. To get magic audio inside a VM that has no sound hardware, you can create a virtual sound card. It’s removed from the gui for some reason, but not that difficult. Just RDP to the VM and redirect the audio to your desktop!

Edit the .vmx file and add the following code: (it helps if you’re in notepad++ so the format is right)

sound.present = “TRUE”
sound.allowGuestConnectionControl = “FALSE”
sound.virtualDev = “hdaudio”
sound.fileName = “-1”
sound.autodetect = “TRUE”

If you do not have vCenter and you want to be able to collect performance stats past 1 hour, this option is greyed out in the vSphere client, however turning this on is pretty easy:

  1. SSH to the host
  2. Open the file “vi /etc/vmware/hostd/config.xml”
  3. Find the <historicalStatsEnabled> tag under <statssvc> and make this true
  4. Run “services.sh restart”