expresswayc-e

There are a number of deployment options for Expressway depending on your customers environment, which could lead you to having to come up with creative solutions. This is called engineering.

We know external services need to be discovered using “outside” dns, however what happens when a customer uses a single dns server for outside and inside networks? Well, The traversal zone will need to be a FQDN for TLS validation, so it needs to use dns to resolve the inside addresses, but clients on the public internet need to use dns to resolve the outside address of the Expressway E.

expc-mra_-_Edit_zone

Tandberg VCS aka Cisco Expressway runs on linux and uses a lightweight network services package called dnsmasq

Being a linux guy, the first thing I tried was editing the /etc/hosts file, but to prevent hacking, everything is on a read-only filesystem except for the /tandberg mount, which also had an etc but changes were not persistent. After poking around I found the dnsmasq.conf man pages here and an option to specify a “conf-dir” to include configuration files that are loaded on start.

For static A records, create a file called hosts.conf

/tandberg/etc/dnsmasq.conf.d/hosts.conf

address=/expe-mra.car.pnslabs.com/10.20.30.40

For static SRV records, create a file called srv.conf

/tandberg/etc/dnsmasq.conf.d/srv.conf

srv-host=_cisco-uds._tcp.car.pnslabs.com.,cucm.car.pnslabs.com.,8443,100,0

After adding the file, you’ll need to restart dnsmasq:

/etc/init.d/dnsmasq restart

 

Unlike other “report this problem” buttons, Jabber’s problem report doesn’t just fly into the cloud somewhere never to be seen again.

It actually contains a lot of useful information about things like: how services are discovered and assigned, what policies are defined, what devices are configured, etc. The problem report can be generated, but is located below:

\Users\AppData\Local\Cisco\Unified Communications\Jabber\CSF\Logs\csf-unified.log

One of the biggest questions I get is where configurations exist, and what takes priority ? Before looking at the file, I would have guessed 3 places, but as you can see below its actually 12!

1: TftpOverrideConfigStore [10] jabber-config.xml
2: LocalFileConfigStore [20] AppData\Roaming\Cisco\Unified Communications\Jabber\CSF\Config\jabberLocalConfig.xml
3: Ucm90ConfigStore [25] http://cucm.fqdn.com:6970/SPDefault.cnf.xml
4: PresenceAuthenticatorStore [28]
5: ServiceDiscoveryConfigStore [29]
6: PresenceConfigStore [30] Presence Server TFTP+CCMCIP Settings
7: RegistryConfigStore [31] HKCU\Software\Cisco Systems, Inc.\JabberWerxCpp
8: PresenceProductModeStore [35]
9: UrlProvisioningConfigStore [499]
10: BootstrapConfigStore [500] ProgramData\Cisco Systems\Cisco Jabber\jabber-bootstrap.properties
11: InMemoryConfigStore [600]
12: jabber-config-defaults.xml [2147483647] Program Files (x86)\Cisco Systems\Cisco Jabber\jabber-config-defaults.xml

Each of these stores contain configuration elements in the form of <tags>, retrieved from either the local machine, or through HTTP GET requests

When Jabber is launched, Jabber first tries to read jabberLocalConfig.xml for cached settings, if any exists. It then attempts to reach the domain controller of the machine you are running on for SSO. If both of those fail, only then will it give you a login prompt. If no cached username is found, email is prompted for and DISCOVERY begins.

retrieve email address and parse domain:

Getting LastLoggedInUserName from ConfigFeatureSet
Getting servicesDomain from ConfigFeatureSet
Getting PresenceDomain from ConfigFeatureSet
ServicesDomain key not found, parsing email address.
Retrieved Email Address jhalpert@car.pnslabs.com
Retrieved DNS Domain ‘car.pnslabs.com’ from ‘Email Address from UserInput’
Domain is set to: car.pnslabs.com

check for webex im:

makeCasLookupForDomain result is ‘Code: NOT_WEBEX_CUSTOMER; CasUrl: http://loginp.webexconnect.com/cas/FederatedSSO?org=car.pnslabs.com;

check SRV records for on-prem and mra:

*—–* DNS query _cisco-uds._tcp.car.pnslabs.com. has succeeded.
*—–* DNS query _collab-edge._tls.car.pnslabs.com. has failed: DNS name does not exist.

determine home cluster:

*—–* Making HTTP request to: https://cucm.car.pnslabs.com:8443/cucm-uds/clusterUser?email=jhalpert@car.pnslabs.com

fetch user data:

https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert
https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert/devices
https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert/credentials
https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert/extensions/
https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert/subscribedServices
https://cucm.car.pnslabs.com:8443/cucm-uds/user/jhalpert/speedDials
https://cucm.car.pnslabs.com:6970/SPDefault.cnf.xml
https://cucm.car.pnslabs.com:6970/global-settings.xml

set location:

setCurrentLocationOption start: description: Jeffs House ,id: fff8ffb0-1332-4da7-ba78-647b35aa25b2
IsGatewayAddressSubnetExists: gateway address: 00::50::56::E3::E5::50, subnet: 172.16.13.0, locations: 1

The need to share dial plan information between multiple call control nodes has been around since gatekeeper, but I find the latest iteration, GDPR, particularly smooth.

Gatekeeper had its downsides: additional hardware costs, lack of URI or +E164 support, and reliance on H323 protocol. CUCM 8 introduced CCD (Call Control Discovery) using SAF (Service Advertisement Framework), but since it required end-to-end EIGRP and a complex configuration, I’ve never known anyone to actually use it. With CUCM 9 came Intercluster Lookup Service (ILS) I did a write up on a year or so ago. The first iteration only supported URIs but was onto something. ILS adds a route string to the database, which tags Directory URIs on each node. Route strings use sip route patterns as opposed to traditional route patterns to dynamically route.

GDPR is simply the addition of numbers and patterns tagged with a route string in ILS. GDPR is supported in CUCM 10+ and uses the following new system partitions:

Directory URI: As soon as the directory number is associated to the end user, this becomes the primary URI on the directory number page.

dna
directoryuri
Global Learned E164 Numbers / Global Learned Enterprise Numbers:

On the directory number page, two new fields “add enterprise alternate number” and “add +E164 alternate number” have been added. Simply check “Advertise Globally via ILS” and these numbers will show up on ILS neighbor clusters as “Global Learned Enterprise Numbers” or “Global Learned E164 Numbers”

numbers

Global Learned E164 Patterns / Global Learned Enterprise Patterns:

Under Call Routing, a new option “Global Dial Plan Replication” has been added. Advertised Patterns is where you can add entire patterns you want to advertise. I find this is easier to manage rather than advertising each number individually; although its nice to have that granularity, it’s usually not necessary.

advertised

To view learned Numbers, Patterns, and Directory URIs, you can find these under “Global Dial Plan Replication” and “View…“, or you can run a traditional route plan report. This is much improved to the “utils ils lookup” needed with CUCM 9.

The default IM address scheme is to take the userid and append @defaultdomain to the end. This may work for smaller deployments, but caused a lot of confusion when multiple domains or subdomains were used. Flexible Jabber ID allows the IM address scheme to be mapped to Directory URI which is then mapped to either mail or msRTCSIP in LDAP.

jid

Changing the IM address scheme to Flexible JID is simple:

    • Stop the following services:
      Cisco Presence Engine
      Cisco SIP Proxy
      Cisco XCP Router
      Cisco Sync Agent
      Cisco Client Profile Agent
    • Presence > Settings > Advanced Configuration:
      Change IM Address Scheme to Directory URI and save.

imaddress

You can now browse to Presence > Domains to see a list of all domains in your presence environment.

You may find that some should not be there, as a lot of people like to add external contacts in their active directory with an ipPhone field populated. This will cause issues down the road if you deploy Mobile and Remote Access and want to do XMPP Federation. You’ll need to be able to sign for each domain. You’ll want to filter these out either by ldap filter into CUCM, or UC Service Profile. The Default Domain above also gets put into the CSR, so make sure to change that as well.

You can run the sql query below to clear the domains from the CSR once they are removed from the system:

run sql select * from impresencedomains

run sql delete from impresencedomains where pkid=’pkid_from_step_1′

To take advantage of Flexible Jabber ID, you must be running Jabber 10.6.6 or higher !

I’m not sure why Cisco didn’t include “custom file” option to update lines like you can phones, but if you have the list of numbers to change call forwarding rules, you can use the following queries which are a part of the numplan table. Call Forward All rules are in another table callforwarddynamic which i’ll cover later:

cfbvoicemailenabled = call forward busy external set to voicemail (boolean)
cfbdestination = call forward busy external destination
cfbintvoicemailenabled = call forward busy internal set to voicemail (boolean)
cfbintdestination = call forward busy internal destination
cfnavoicemailenabled = call forward no answer external set to voicemail (boolean)
cfnadestination = call forward no answer external destination
cfnaintvoicemailenabled = call forward no answer internal set to voicemail (boolean)
cfnaintdestination = call forward no answer internal destination

An example query below:

run sql update numplan set cfbvoicemailenabled = (‘f’), cfbintvoicemailenabled = (‘f’), cfnavoicemailenabled = (‘f’), cfnaintvoicemailenabled = (‘f’) where dnorpattern = “1111111”

Making bulk changes in CUCM can be done in 3 ways:

  1. bulk operations built into cucm – great but very limited
  2. import/export of tar file – all details – concise but requires change freeze
  3. sql operations – sometimes the best option

The following example I needed to update each line text label to add “Agent” to the beginning. Built in operations would only allow me to change the label all together, not append text. Import / Export would work fine, but for 10,000 phones this would take a full day to process. In this case SQL was my best option. The query below matches the line and updates the label. I used excel and the concatenate function to pump out all of the required operations.

run sql update devicenumplanmap set (label) = (‘Agent – 1112001’) where fknumplan = (select pkid from numplan where dnorpattern = “1112001”)

The vlookup function in excel is one of the most powerful tools to a voice engineer, which at times can seem more like being a database administrator. The logic here is simple: find the user id from the table on the left in the user id column on the table in the right. When you find it, return the value in the row x number of rows away.

=VLOOKUP(lookup_value, table_array, col_index_num, [range_lookup])

lookup_value = userid

table_array = the entire table on the right. the leftmost column must contain the lookup value

col_index_num = how many rows to the right to grab the value to return

range_lookup = false always!

vlookup

So I took the CCIE lab and failed my first attempt. I realize now it’s more about efficiency and speed than skill. 8 hours goes by in what seems like an hour. So I decided what I needed to do was practice the lab by repetition. After creating a mock lab, I made snapshots in vmware, then created a Power CLI script and bat file to revert to the snapshot and power back on. In one click I can just start all over and do it again!

revert.ps1:

Connect-VIServer -Server vcenter -User root -Password blahblah

$VMs = Get-Content ‘C:\vmlist.txt’
Get-Snapshot -VM $VMs -Name ccie | Foreach-Object { Set-VM -VM $_.VM -Snapshot $_ -Confirm: $false }
Start-VM -VM $VMs

vmlist.txt:

HQ-PUB
HQ-SUB
SB-PUB
HQ-CUC
HQ-CCX
HQ-IMP

startover.bat:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -PSConsoleFile “C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\vim.psc1” -NoExit -Command “& ‘C:\revert.ps1′”

 

EDI (Enhanced Directory Integration) can use the logged on user domain credentials to search LDAP directory, or a saved set of credentials in either the Service Profile or jabber-config.xml via the <ConnectionUsername> and <ConnectionPassword> tags. It’s only supported with Jabber for Windows since it uses the built-in Windows API.

BDI (Basic Directory Integration) uses stored credentials in either the Service Profile or jabber-config.xml via the <BDIConnectionUsername> and <BDIConnectionPassword> tags to search LDAP directory. Storing credentials in the service profile is highly recommended, since the jabber-config.xml is plaintext.

When both service profile and jabber-config.xml define a parameter, the service profile will take priority.

UDS (User Directory Service) uses CUCM to provide directory lookup services. UDS is the only supported method when MRA is used, however because CUCM is now providing directory lookups and not LDAP, the load must be considered: a node can support UDS contact service connections for up to 50% of the maximum device registrations supported by the server.

jabber-directory

Multiple Unity Connection Clusters may be networked together to provide cross-server features such as sign-in, transfer and live reply. An example would be a dial by name in a company directory being able to search multiple clusters and route the call appropriately.

First thing you’ll need is to define a route pattern in CUCM that will point to each Unity cluster. The CSS of each Unity Connection cluster should have access to this route pattern. In the following example I will use: ##991 to Site A and ##992 to Site B.

Next, add a HTTP(S) link under Networking, Branch Management. This adds the site under Networking, Locations. Notice you’ll only need to do this on one cluster- it will automatically add it to the neighbor cluster.

Now you’ll add the route pattern you created in step one to each cluster’s respective locations and check both “Allow Cross-Server..” options. This is the target dial string, so the route pattern going to site B will go on site A’s cluster.

Under Advanced, Conversations you’ll need to check “Respond to Cross-Server Handoff Requests”

OK at this point you can check under Users and you should see users from both clusters to tell you HTTP(s) networking is working, but before we can make the features work, take the steps below to create some partitioning:

  1. Create a “hidden” and an “intersite” partition on both clusters.
  2. Create a site-specific “reply-all” search space including each sites default partition (it’s own first, then its neighbors) and the intersite partition.
  3. Create a shared “master distribution list” search space, including all partitions, starting with site specific partitions, then the intersite partition, then the hidden.
  4. Rename the system distribution list on each cluster and change it to use the hidden partition.
  5. Create a new “master distribution list” to use the master distribution list partition.
  6. Change the system directory handler to use the new master distribution list search space

Now we just need to configure the clusters to respond to cross cluster requests. This uses a series of dtmf codes to hand the call off between servers. You’ll need to run the following via CLI to enable this:

run cuc dbquery unitydirdb execute procedure csp_ConfigurationCreate
(pName=’HandoffForwardRemoteForward’::lvarchar, pParentFullName=’System.Conversations.CrossBox’::lvarchar, pType=11, pValueBool=1, pRequiresRestart=1)

run cuc dbquery unitydirdb execute procedure csp_ConfigurationModify (pName=’HandoffForwardRemoteForward’::lvarchar, pParentFullName=’System.Conversations.CrossBox’::lvarchar, pValueBool=1)

Almost done – the only thing left is a call handler to answer the call.

  1. Copy the opening greeting to create a “CrossCluster_CallHandler.
    1. Under caller input, uncheck “ignore further input” under #.
  2. Create a Direct Routing Rule.
    1. Condition “Dialed Number Equals ##992” (this is what it answers to)
    2. “Send Call to” should go to the “CrossCluster_CallHandler”

See the official documentation here