1. NAT Exemption – don’t ever nat this
  2. Static Policy NAT – nat to something when this goes to that
  3. Static NAT – nat this to that
  4. Dynamic Policy NAT/PAT – nat to something when these go to that
  5. Identity NAT – don’t nat this when i’m talking
  6. Dynamic NAT – nat all these to this
  7. Dynamic PAT –  nat all these ports to this

 

First we’ll discuss a basic ipsec tunnel between two ISRs:

vpn1

RTR-A
Define IKE Phase 1 Policy (ISAKMP)
crytpo isakmp policy 10
encryption aes 256
authentication pre-share
hash sha
group 2

Define pre-shared key
crypto isakmp key pr3sh4r3dk3y address 22.2.1.1 no-xauth

Define IKE Phase 2 Policy (IPSEC)
crypto ipsec transform-set aes-sha esp-aes 256 esp-sha-hmac

Create ACL to match interesting traffic
access-list 150 permit ip host 192.168.1.1 host 192.168.2.1

Create Crypto Map
crypto map vpn-a-to-b 10 ipsec-isakmp
set peer 22.2.1.1
set transform-set aes-sha
match address 150

Apply Crypto Map to Interface
interface Gi0/0
crypto map vpn-a-to-b

Create a route
ip route 192.168.2.1 255.255.255.255 Gi0/0

RTR-B
Define IKE Phase 1 Policy (ISAKMP)
crytpo isakmp policy 10
encryption aes 256
authentication pre-share
hash sha
group 2

Define pre-shared key
crypto isakmp key pr3sh4r3dk3y address 22.1.1.1 no-xauth

Define IKE Phase 2 Policy (IPSEC)
crypto ipsec transform-set aes-sha esp-aes 256 esp-sha-hmac

Create ACL to match interesting traffic
access-list 150 permit ip host 192.168.2.1 host 192.168.1.1

Create Crypto Map
crypto map vpn-b-to-a 10 ipsec-isakmp
set peer 192.168.1.1
set transform-set aes-sha
match address 150

Apply Crypto Map to Interface
interface Gi0/0
crypto map vpn-b-to-a

Create a route
ip route 192.168.1.1 255.255.255.255 Gi0/0

Now the more likely scenario –

two hosts need to talk to the same company and the ips overlap

vpn2

We’ll use the same instructions, but making some adjustments to RTR-B

Create ACL to match interesting traffic (using natted address this time)
access-list 150 permit ip host 192.168.11.1 host 192.168.2.1

Create route-map
route-map B-to-C permit 10
match ip address B-to-C-ACL

Create ACL to match traffic in the scenario
ip access-list extended B-to-C-ACL
permit ip host 192.168.1.1 host 192.168.2.1

Create NAT statement and apply to interfaces
ip nat inside source static 192.168.1.1 192.168.11.1 route-map B-to-C-ACL extendable

int Gi0/1
description inside
ip nat inside

int Gi0/0
description outside
ip nat outside

 This can be done a little simpler with an ASA using a static policy nat

access-list policy-nat extended permit ip host 192.168.1.1 host 192.168.2.1
static (inside,outside) 192.168.11.1  access-list policy-nat

staticpolicyasa

Unity Connection can be integrated with CUCM using 2 methods, SCCP or SIP, the high level differences and similarities are detailed below;

SCCP Integration

  • Requires SCCP Ports, along with Line Group,  Hunt List & Hunt Pilot
  • Has dedicated DNs for MWI on/off

SIP Integration

  • Requires a SIP Trunk pointing to Unity Connection
  • Requires a route pattern to send calls to the SIP trunk (or a route group if there is a clustered Unity setup)
  • Does not require MWI DNs, uses SIP NOTIFY messages

Both integrations require a VM Pilot and a VM Profile. The VM Pilot is not a dialable DN/pattern, it’s more of a ‘speed dial’ for the phones to use, so when we change the VM profile for a DN it’s effectively a speed dial button for the VM pilot number references in the VM profile.

SCCP Integration – CUC Configuration
Firstly we have to create a ‘Phone System’ This is the highest-level element of the integration configuration, it will contain a port group which will then contain individual ports.

The options here control the integration between the two, including settings for MWIs (and the ability to initiate a synchronization in case the MWIs have become inconsistent for some reason). It also allows you to enable/disable loop detection (which is on by default), either by Extension (default) or by DTMF. Is it used to guard against scenarios such as breaking out of an auto-attendant system call handler to call a particular user and that DN they call is set to forward to VM because they are away from their desk. I need to add more information here.  The Phone View feature is controlled from here – enabling/disabling the feature along with the username/password for Unity Connection to use (must be an application user on CUCM with CTI control of the required devices) – along with outgoing call restrictions (unrestricted/blocked/blocked during a given time period)

You must also go to ‘Edit’ and select ‘CUCM AXL Servers’ and add the CUCM servers with their IPs or hostnames if CUC uses DNS, ensure the port is set to 8443. Add in the user/pass for AXL access (must be created in CUCM with the ‘Standard AXL access’ role) and hit ‘Test’ for each CUCM server to ensure there is no error.

Once this is done, the next step is to create a port group. A port group is given a name, a device name prefix along with more detailed MWI settings consisting of MWI On/MWI Off extensions and some timers for MWI. The defaults are pretty sane I think. The Device Name Prefix is important, as it must match at CUC and CUCM.

The port group configuration also allows you to specify CUCM & TFTP servers along with detailed timer settings to be used if there are issues with the integration (typically the defaults are sane and work fine with CUCM, but these settings may need changed if the integration is to another voice system like Asterix/FreePBX). Finally, you can also control the codecs that Unity Connection will advertise during the capabilities exchange of any call setup. The default appears to be G711ulaw and G722. iLBC/G711alaw/G729 are also available.

The final step is to create ports. These ports can have 4 functions enabled on each port; Answer calls/MWI notification/MWI requests/TRAP notifications. With this in mind, groups of ports could be assigned separate functions or all ports could be assigned all functions, although doing this means you may run in to issues in not being able to ensure that users will almost always be able to get a port for checking voicemail if everybody happened to be recording a new greeting using all ports there would be none free for VM-checking. If a group of ports were enabled for answering calls only, then these would only every be used for users calling in to Unity Connection (for VM/call handlers/etc).

SCCP Integration – CUCM  Configuration
There is a handy wizard at Advanced Features -> VM -> VM Port Wizard

First off, you enter the device name prefix. Ensure this matches what you configure at the CUC side, otherwise the ports will not register to CUCM correctly! Then you tell the wizard how many ports you want to create during the wizard.

When you hit next you are asked for the typical line/device information for the ports/DNs to be used, including Device Pool/CSS/Location/etc. The CSS for VM port devices/VM pot DNs is used by Unity Connection for calling out on any of these ports. Unity Connection has restriction tables that can block calls before they ever leave CUC, which is handy – we can give the ports full access but restrict it from the CUC side. Why do we set it twice in the wizard? Because a VM port is the same as any other ‘thing’ that can dial – it uses the line/device approach where the line and device are assigned a CSS.

After configuration the device/DNs you can chose to add the numbers to a new or existing line group, or do it manually. Thereafter you must add the line group to a hunt list, and add that hunt list to a hunt pilot. The hunt list must have the ‘for voicemail usage’ box ticket, although I’m not sure why. I wonder if I can find out…

Then we need to configure a VM Pilot and VM Profile. The VM pilot is the number used to dial in to Unity Connection. For an SCCP integration it is a number that is assigned a CSS that can reach the Hunt Pilot eventually containing the VM ports. This is then assigned to the VM profile assigned to each DN. I’m not sure why the VM Pilot has a CSS, does the calling party inherit that CSS when pressing the ‘Messages’ button or something? Update: from a bit of Googling, I think the CSS assigned there is used when a device forwards to voicemail.

We also need to ensure we configure the MWI On/MWI Off extensions in CUCM . Go to Advanced Features -> Voicemail -> Message Waiting and hit ‘add’ to add a new MWI DN, enter the number/partition/CSS and whether it is for on or off. Rather (un)helpfully, the MWI On icon is green and MWI off is red – despite the MWI light itself being red! The MWI DNs are assigned a partition which allows you to restrict whether users can dial the MWI directly (i.e the users devices/lines are assigned CSS’ that don’t contain the partition they are in). Unity Connection dials these with a spoofed calling number/ANI of the DN we wish to set an MWI for (I belive!), then CUCM signals that DN to display MWI. The voicemail port CSS must be able to reach the MWI DNs for this to work!

SIP Integration – CUC Configuration

Add a new port group, ensure the type is set to ‘SIP’ rather than ‘SCCP’ the default port/protocol settings are usually ok, add the IPv4 address of the CUCM at the bottom. Hit save and then click on the related link to add ports. You can also at this stage go to ‘Edit’ and add details for any other CUCM servers so it isn’t reliant on the single server configured on the main page. A reset of the port group is required after this.

We must then create ports the same as we do with SCCP, this is because this is a licensing requirement in Unity Connection – it is how they limit the number of calls in to the system.

Done! =D

SIP Integration – CUCM Configuration

Create a SIP trunk (default service type of ‘none’), give it a name/device pool/etc. Ensure inbound significant digits is set to all and the CSS’ and AAR settings are valid to ensure any outcall to the PSTN works. Disable outbound calling/called party xforms unless required. Ensure ‘Redirecting iversion header delivery – outbound’ is ticked so that the RDNIS info is included in the setup messages.

Enter the IP of the CUC server as the destination. Hit save & reset the trunk.

At this point, we need to create a new SIP Trunk Security Profile and ensure the following settings are selected;

  • Accept Out-of-Dialog REFER
  • Accept unsolicited notification
  • Accept replaces header

If we’re creating a new SIP Trunk Security Profile for other Cisco UC Applications, it may make sense to also tick the box that mentions ‘SUBSCRIBE’ (I can’t remember the exact wording of it off the top of my head!) – it will be required for CUPS.

Assign this new SIP Security Profile to the SIP trunk & reset.

Next, create a route pattern that matches the VM pilot, set the destination to be the CUC SIP Trunk and hit save. Untick any PSTN settings.

Credit goes to http://protocol41.wordpress.com for this one!

Suppose we are a service provider carrying multiple tenants across our network. This is actually pretty simple and doesn’t require carrier-grade equipment.

In this example we’ll have an inner tag (customer) and an outer tag (provider). We’ll add 14 to the max mtu size on the provider switch to accommodate for the second tag.

The outer tag uses both the “switchport mode access” to define the customer and “switchport mode dot1q-tunnel” to tell that it’s twice encapsulated.

You can “hop off” the outer tag at the CPE depending on what vlan you assign them.

This is the basics of q-in-q. With this you can provide “metro-e” or “private clouds” really its just basic vlans inside of vlans!
8021qinq

 

So can you also do router-on-a-router-on-a-stick ? Sure can..

2921(config)# interface gigabitethernet 0/1/0
2921(config-if)# dot1q tunneling ethertype 0x9100
2921(config-if)# interface gigabitethernet 1/1/0.1
2921(config-subif)# encapsulation dot1q 100 second-dot1q 200

Read more about it : at cisco’s site

monitor session 1 type erspan-source
erspan-id 1
vrf default
destination ip 10.1.x.y
source vlan 1-2,4,6-7,10,12,14,16,18,20,22,28,30,33,40,50,60,100,122,199-200,202 both
source vlan 211-212,222,230,236,240,998-999 both
no shut

monitor erspan origin ip-address y.y.y.y global

destination ip is of course your L3 based sniffing tool…like ExtraHop
y.y.y.y in this example is a management IP on my default VRF.

Often when troubleshooting an issue, having a good test can take a while to get ready. With CSIM, an undocumented IOS command, it is possible to simulate an outbound call, originating from your voice gateway directly. Use the csim start dialstring hidden command to initiate simulated calls to whichever real-world E.164 number is desired. This allows you to determine whether you can properly go offhook, send digits, and complete a call to the destination phone.

2951-VGW#debug isdn q931
debug isdn q931 is              ON.
2951-VGW#csim start 19105557245
csim: called number = 19105557245, loop count = 1 ping count = 0

Apr 13 22:19:42.179: ISDN Se0/1/0:23 Q931: Sending SETUP  callref = 0x09CD callID = 0x8955 switch = primary-ni interface = User
Apr 13 22:19:42.179: ISDN Se0/1/0:23 Q931: TX -> SETUP pd = 8  callref = 0x09CD
Bearer Capability i = 0x8090A2
Standard = CCITT
Transfer Capability = Speech
Transfer Mode = Circuit
Transfer Rate = 64 kbit/s
Channel ID i = 0xA98397
Exclusive, Channel 23
Called Party Number i = 0x80, ‘19105557245’
Plan:Unknown, Type:Unknown
Apr 13 22:19:42.227: ISDN Se0/1/0:23 Q931: RX <- CALL_PROC pd = 8  callref = 0x89CD
Channel ID i = 0xA98397
Exclusive, Channel 23
Apr 13 22:19:44.863: ISDN Se0/1/0:23 Q931: RX <- PROGRESS pd = 8  callref = 0x89CD
Progress Ind i = 0x8188 – In-band info or appropriate now available
Apr 13 22:19:52.376: ISDN Se0/1/0:23 Q931: RX <- CONNECT pd = 8  callref = 0x89CD
Apr 13 22:19:52.376: ISDN Se0/1/0:23 Q931: TX -> CONNECT_ACK pd = 8  callref = 0x09CD
csim err csimDisconnected recvd DISC cid(42503)
csim: loop = 1, failed = 1
csim: call attempted = 1, setup failed = 1, tone failed = 0
Apr 13 22:19:57.144: ISDN Se0/1/0:23 Q931: RX <- DISCONNECT pd = 8  callref = 0x89CD
Cause i = 0x8290 – Normal call clearing
Apr 13 22:19:57.144: ISDN Se0/1/0:23 Q931: TX -> RELEASE pd = 8  callref = 0x09CD
Apr 13 22:19:57.156: ISDN Se0/1/0:23 Q931: RX <- RELEASE_COMP pd = 8  callref = 0x89CDter no mo
Apr 13 22:19:58.741: //-1/xxxxxxxxxxxx/SIP/Msg/ccsipDisplayMsg:

Despite the call working in this case CSIM will always display failed=1.  There is no real explanation for this.

If the call does not complete, you could perform a “show dial-peer voice sum” to match dialed digits, or my favorite “show dialplan number

2911-VGW#show dialplan num 19105557245
Macro Exp.: 19105557245

VoiceOverIpPeer1000
peer type = voice, system default peer = FALSE, information type = voice,
description = `SIP_OUT_11_DIGIT’,
tag = 1000, destination-pattern = `1[2-9]………’,
voice reg type = 0, corresponding tag = 0,
allow watch = FALSE
answer-address = `’, preference=0,
CLID Restriction = None
CLID Network Number = `’
CLID Second Number sent
CLID Override RDNIS = disabled,
rtp-ssrc mux = system
source carrier-id = `’, target carrier-id = `’,
source trunk-group-label = `’,  target trunk-group-label = `’,
numbering Type = `unknown’
group = 1000, Admin state is up, Operation state is up,
incoming called-number = `’, connections/maximum = 0/unlimited,
bandwidth/maximum = 0/unlimited,
DTMF Relay = enabled,
modem transport = disabled,
URI classes:
Incoming (Request) =
Incoming (Via) =
Incoming (To) =
Incoming (From) =
Destination =
huntstop = disabled,
in bound application associated: ‘DEFAULT’
out bound application associated: ”
dnis-map =
permission :both
incoming COR list:maximum capability
outgoing COR list:minimum requirement
outgoing LPCOR:
Translation profile (Incoming):
Translation profile (Outgoing):
incoming call blocking:
translation-profile = `’
disconnect-cause = `no-service’
advertise 0x40 capacity_update_timer 25 addrFamily 4 oldAddrFamily 4
mailbox selection policy: none
type = voip, session-target = `sip-server’,
technology prefix:
settle-call = disabled
ip media DSCP = cs5, ip media rsvp-pass DSCP = ef
ip media rsvp-fail DSCP = ef, ip signaling DSCP = cs4,
ip video rsvp-none DSCP = af41,ip video rsvp-pass DSCP = af41
ip video rsvp-fail DSCP = af41,
ip defending Priority = 0, ip preemption priority = 0
ip policy locator voice:
ip policy locator video:
UDP checksum = disabled,
session-protocol = sipv2, session-transport = system,
req-qos = best-effort, acc-qos = best-effort,
req-qos video = best-effort, acc-qos video = best-effort,
req-qos audio def bandwidth = 64, req-qos audio max bandwidth = 0,
req-qos video def bandwidth = 384, req-qos video max bandwidth = 0,
dtmf-relay = sip-notify,
dtmf-relay = rtp-nte,
RTP dynamic payload type values: NTE = 101
Cisco: NSE=100, fax=96, fax-ack=97, dtmf=121, fax-relay=122
CAS=123, TTY=119, ClearChan=125, PCM switch over u-law=0,
A-law=8, GSMAMR-NB=117 iLBC=116, AAC-ld=114, iSAC=124
lmr_tone=0, nte_tone=0
h263+=118, h264=119
G726r16 using static payload
G726r24 using static payload
RTP comfort noise payload type = 19
fax rate = disable,   payload size =  20 bytes
fax protocol = system
fax-relay ecm enable
Fax Relay ans enabled
Fax Relay SG3-to-G3 Enabled (by system configuration)
fax NSF = 0xAD0051 (default)
codec = g711ulaw,   payload size =  160 bytes,
video codec = None
voice class codec = `’
voice class sip session refresh system
voice class sip rsvp-fail-policy voice post-alert mandatory keep-alive interval 30
voice class sip rsvp-fail-policy voice post-alert optional keep-alive interval 30
voice class sip rsvp-fail-policy video post-alert mandatory keep-alive interval 30
voice class sip rsvp-fail-policy video post-alert optional keep-alive interval 30
voice class sip profiles = 1
text relay = disabled
Media Setting = forking (disabled) flow-through (global)stats-disconnect (disabled)
Expect factor = 10, Icpif = 20,
Playout Mode is set to adaptive,
Initial 60 ms, Max 1000 ms
Playout-delay Minimum mode is set to default, value 40 ms
Fax nominal 300 ms
Max Redirects = 1, signaling-type = cas,
VAD = disabled, Poor QOV Trap = disabled,
Source Interface = NONE
voice class sip url = system,
voice class sip tel-config url = system,
voice class sip rel1xx = system,
voice class sip anat = system,
voice class sip outbound-proxy = “system”,
voice class sip associate registered-number = system,
voice class sip asserted-id system,
voice class sip privacy system
voice class sip e911 = system,
voice class sip history-info = system,
voice class sip reset timer expires 183 = system,
voice class sip pass-thru headers = system,
voice class sip pass-thru content unsupp = system,
voice class sip pass-thru content sdp = system,
voice class sip copy-list = system,
voice class sip g729 annexb-all = system,
voice class sip early-offer forced = enable,
voice calss sip delay-offer forced = disable,
voice class sip negotiate cisco = system,
voice class sip block 180 = system,
voice class sip block 183 = system,
voice class sip block 181 = system,
voice class sip preloaded-route = system,
voice class sip random-contact = system,
voice class sip random-request-uri validate = system,
voice class sip call-route p-called-party-id = system,
voice class sip call-route history-info = system,
voice class sip call-route url = system,
voice class sip privacy-policy send-always = system,
voice class sip privacy-policy passthru = system,
voice class sip privacy-policy strip history-info = system,
voice class sip privacy-policy strip diversion = system,
voice class sip send 180 sdp = system,
voice class sip map resp-code 181 = system,
voice class sip bind control = system,
voice class sip bind media = system,
voice class sip bandwidth audio = system,
voice class sip bandwidth video = system,
voice class sip encap clear-channel = system,
voice class sip error-code-override options-keepalive failure = system,
voice class sip error-code-override cac-bandwidth failure = system,
voice class sip calltype-video = false
voice class sip registration passthrough = System
voice class sip authenticate redirecting-number  = system,
voice class sip referto-passing = system,
redirect ip2ip = disabled
local peer = false
probe disabled,
Secure RTP: system (use the global setting)
mobility=0, snr=, snr_noan=, snr_delay=0, snr_timeout=0
snr calling-number local=disabled, snr ring-stop=disabled, snr answer-too-soon timer=0
voice class perm tag = `’
Time elapsed since last clearing of voice call statistics never
Connect Time = 303893, Charged Units = 0,
Successful Calls = 37, Failed Calls = 9, Incomplete Calls = 0
Accepted Calls = 0, Refused Calls = 0,
Bandwidth CAC Accepted Calls = 0, Bandwidth CAC Refused Calls = 0,
Last Disconnect Cause is “10  “,
Last Disconnect Text is “normal call clearing (16)”,
Last Setup Time = 510566383.
Last Disconnect Time = 510567472.
Matched: 19105557245   Digits: 2   Matched pattern: 1[2-9]………
Target: sip-server

Sometimes your asked to tune down certain hosts only during business hours to conserve bandwidth. I always tell them the easiest way to control this is at the switch at a physical level using tcl scripts and scheduled kron jobs. In this example, the port is limited to 1MB at 8:00 AM daily, but the limit is removed at 5:00PM.

First, create the script

You can do this in notepad and upload using tftp. I prefer using the cli directly:

switch#tclsh
switch(tcl)#puts [open “flash:portlimit1.tcl” w+] { ios_config “interface Gi1/0/10” “speed 10” “bandwidth 10” “srr-queue bandwidth limit 10” }
switch(tcl)#puts [open “flash:portlimit0.tcl” w+] { ios_config “interface Gi1/0/10” “speed 1000” “bandwidth 1000” “no srr-queue bandwidth limit 10” }

Next, define the kron policies

kron policy-list turnUp
tclsh portlimit0.tcl

kron policy-list turnDown
tclsh portlimit1.tcl

Then, schedule the kron job

kron occurence open at 08:00 recurring
policy-list turnDown

kron occurence closed at 17:00 recurring
policy-list turnUp

It surprises me how many times people invest the money in having a redundant edge, but fail to do the simple things: plug into different power sources and have their switch be a single point of failure. With this configuration, you could lose a power source, a firewall, a switch in the stack, and 75% of your cables and still function!

hapair2

Configure Switch Stack:

vlan 10
name LAN

vlan 777
name ISP1

interface Port-channel1
description Primary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
!
interface Port-channel2
description Secondary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
!
interface GigabitEthernet1/0/47
description Primary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet1/0/48
description Secondary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
channel-group 2 mode active
!
interface GigabitEthernet2/0/47
description Primary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
channel-group 1 mode active
!
interface GigabitEthernet2/0/48
description Secondary ASA
switchport trunk allowed vlan 10,777
switchport mode trunk
channel-group 2 mode active
!

Configuration Primary ASA:

interface GigabitEthernet0/0
channel-group 1 mode active
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/1
channel-group 1 mode active
no nameif
no security-level
no ip address
!
interface Port-channel1
no nameif
no security-level
no ip address
!
interface Port-channel1.10
vlan 10
nameif LAN
security-level 100
ip address 172.17.10.10 255.255.255.0 standby 172.17.10.11
!
interface Port-channel1.777
vlan 777
nameif ISP1
security-level 0
ip address 24.222.111.25 255.255.255.240

interface GigabitEthernet0/5
description LAN/STATE Failover Interface

failover
failover lan unit primary
failover lan interface failover GigabitEthernet0/5
failover link failover GigabitEthernet0/5
failover interface ip failover 10.255.255.1 255.255.255.252 standby 10.255.255.2
monitor-interface LAN

Configure Secondary ASA:

failover
failover lan unit secondary
failover lan interface failover GigabitEthernet0/5
failover link failover GigabitEthernet0/5
failover interface ip failover 10.255.255.1 255.255.255.252 standby 10.255.255.2
monitor-interface LAN

CIPC is a great tool for troubleshooting voice, but it kind of requires a sound card to work, which a VM doesn’t provide and sometimes that’s all you have. To get magic audio inside a VM that has no sound hardware, you can create a virtual sound card. It’s removed from the gui for some reason, but not that difficult. Just RDP to the VM and redirect the audio to your desktop!

Edit the .vmx file and add the following code: (it helps if you’re in notepad++ so the format is right)

sound.present = “TRUE”
sound.allowGuestConnectionControl = “FALSE”
sound.virtualDev = “hdaudio”
sound.fileName = “-1”
sound.autodetect = “TRUE”

vmaudio

Recently when replacing an ASA5510 with a new ASA5545X, I noticed the nat was not working for a couple public ips. I used the following command to capture any packets attempting to hit that ip. The problem ended up being an arp cache issue that had to involve the provider. Something to keep in mind with fiber services – the device on site is not a layer 3 device, it only converts fiber to copper. The device that needed to be flushed was at the provider end, miles away.

cap o type raw-data int OUTSIDE match ip any host 60.33.70.69

show cap