Network Stability Through Resilience Engineering
Once again, we’re prompted to write because there isn’t a lot of documentation on a particular subject, and we’re interested in passing along configurations that we got to work.
SRST is something that’s been around for some time, but using CME for SRST is relatively new. Now, by “relatively” we mean it’s actually been a few years (since CME v4.0), so one would think that more information about it would be out there, but such is not the case. And, in fact, we’ve run across a lot of people who don’t even know that the configurations exist, which is a shame because it’s pretty cool.
But before we get folks too worked up, we feel the need to point out – in the interest of full disclosure – that there seem to be three downsides to using CME as opposed to regular SRST configurations. First, a CME/SRST ISR is only able to register the same number of phones that it would be able to handle if it was being employed as a stand-alone CME product. Standard SRST configurations usually allow a voice gateway to register more phones.
Second, CME/SRST works best for SCCP phones. We understand there are some workarounds for SIP endpoints, but they are not well documented (shocker) and we haven’t tried them yet.
The final issue is one that is hard to explain until we talk about a few other things first.
But if the environment is mostly an SCCP phone shop, there are a lot of upsides to using CME for SRST. For one thing, the phones have more features available like call park, call pickup, and call pickup groups. It’s possible to do paging. Softkey templates can be controlled. And it’s possible to set up the same type of class of restriction CME uses so that employees can’t suddenly call Moscow when CUCM is down.
The configuration is different, though, and – as we say – there isn’t a lot of information about it. The following are the commands that worked for us. Our examples use a gateway that is communicating with CUCM using H.323, although we’ve also done it with gateways connected to CUCM through a SIP trunk. The latter works the same as long as the phrase “session protocol sipv2” and an appropriate dtmf-relay command is added to the dial-peers handling calls going to and coming from CUCM.
We are beginning with the assumption that the voice gateway is already configured, and is able to send and receive calls to and from the PSTN.
The steps to configure SRST are as follows:
1) Do some research. Make sure that the firmware load on the phones will work with both the version of CUCM that will be used as well as the version of CME that will be used.
2) Get the device configured in CUCM . Navigate to System > SRST and click “Add New”. The “Name” will be the name of the router, the “Port” will be 2000, the “IP Address” is the IP of the interface on the router that points toward CUCM. One note here is that for our SIP configuration, we didn’t put anything in the “SIP Network/IP Address” field. This is for an IP address of an SRST device that SIP phones will communicate with during a WAN outage. We didn’t use any SIP phones in our network.
3) Add the SRST device to the appropriate device pool. Navigate to System > Device Pool then find and select the pool with which the SRST device should be associated. Click on the “SRST Reference” drop-down and select the device that was created in the previous step. Save everything, and do a Reset. This will reset the phones in the device pool.
4) It’s not a bad idea at this point to look at the configuration of a few selected phones (either on the phone’s web page or on the phone itself) and make sure that the SRST gateway is listed as one of the phones’ CM devices.
The next set of steps deal with entering the commands on the ISR. Before continuing, we want to give a brief overview of the concept behind the configurations. Many of the terms used will be most familiar to those who have experience configuring CME.
First of all, we will be enabling CME using the “telephony-service” command as opposed to “call-manager-fallback”.
Each directory number will be created as an “ephone-dn” just as each one is when setting up CME. But we can also create an “ephone-dn-template” to set default configurations for all of the directory numbers. We used this for both voice translation-rules and corlists.
One big “gotcha” here is that the ephone-dns are supposed to be thought of as separate entities from the phones (or, “ephones” as CME calls them). DNs aren’t really “on” the phones, they’re just “paired with” ephones. Furthermore, the ephones can be seen as being outside of CME and the ephone-dns as being inside of CME. So, conceptually, calls are placed out of an ephone and come into an ephone-dn. In order for calls from ephones to have translations and corlists applied correctly, therefore, ephone-dns or ephone-dn-templates have to be configured with incoming corlists and incoming translation-profiles.
By the same token, ephone-dns and ephone-dn-templates have to have outgoing corlists and translations for calls leaving them and going out to their paired ephones.
One other word before we get into the steps: folks who have configured standard SRST are not used to having to configure ephone-dns. That’s OK, though, because people who have worked with CME are used to having to configure ephones and they won’t be doing that here. So there’s plenty to make everyone uncomfortable!
Any CME person will say that configuring an ephone-dn means being able to control what features are available when calls are placed from that number. But any SRST person will say that the beauty of SRST is that the ISR configures the phones automatically.
CME/SRST is a compromise between those two ideals. Ephone-dns are created manually so the administrator has more control over what features can be used, but the ephones are still created by the router after they inform CME about the ephone-dns with which they are supposed to be paired during SRST.
So, finally, here are the steps for configuring the CME router. All are done from the CLI:
1) If voice translation rules or CoR will be used, it’s a good idea to configure them first. The following are examples:
voice translation-rule 9
rule 1 /^9/ //
voice translation-profile DIGITSTRIP-9
translate called 9
dial-peer cor custom
dial-peer cor list LOCAL-IN
dial-peer cor list LD-IN
dial-peer cor list INTL-IN
dial-peer cor list LOCAL-OUT
dial-peer cor list LD-OUT
dial-peer cor list INTL-OUT
2) The next thing to create is the ephone-template. What’s accomplished with this step is the configuration of the softkeys the phones will have during an SRST event. Remember that this cannot be configured until the “telephony-service” command has been issued. The following is an example:
softkeys connected Park Hold Endcall Trnsfer Confrn Join ConfList RmLstC
softkeys idle Newcall Redial Cfwdall Dnd
softkeys hold Resume Newcall Join
softkeys ringing Answer Dnd
softkeys seized Redial Endcall Cfwdall Meetme
softkeys remote-in-use Newcall
softkeys alerting Endcall Callback
3) After the ephone-template is done, the ephone-dn-template should be configured. This is going to push out things like the corlists and translation-patterns to the DNs. Always remember that a corlist or translation applied to this template is going to be the “incoming” portion of the list (as opposed to the dial-peer which is the “outgoing” portion of the list). Realize that we were only concerned with controlling the flow of calls coming from an internal phone and going to the PSTN (if we wanted to control calls going from the PSTN to the internal phones, we’d also have a “corlist outgoing” statement). The following is an example:
corlist incoming LD-IN
translation-profile incoming 2
Although we don’t show it in this article, the outgoing corlists will go on the outgoing dial-peers pointing to the PSTN.
4) At this point, it’s finally time to deal with the rest of the telephony-service commands. The following is an example:
srst mode auto-provision none
srst ephone template 1
srst dn template 1
srst dn line-mode single
max-dn 290 preference 0
ip source-address 10.0.0.1
no service directed-pickup
max-conferences 4 gain -6
system message Local Mode
sdspfarm units 1
sdspfarm tag 1 CFB -03
It’s worth pointing out a few things in the above configuration. The commands beginning with “srst” are the SRST commands. Huge surprise. The ephone template and dn template reference what was created above. We’ll talk about the line-mode in a minute. The big thing to note is that it’s strongly recommended that the command “srst mode auto-provision none” be used as opposed to any other possibility. The reason is that if there is an SRST situation, the other versions of this command will cause the devices that are created on-the-fly to be written to the running-configuration of the ISR. If this happens, they won’t go away when connectivity to CUCM is restored. We are told that this can cause reachability problems when the phones fall back to CUCM. We actually didn’t see the problems in our testing, but we take the warning seriously. Plus if someone comes along and saves the configurations during SRST, the commands won’t leave until they’re manually deleted.
Just like CME, the “max-ephones” and “max-dn” lines are required. They tell the router how many phones and extensions will be allowed to be created. Also the “ip source address” line is required to get everyone talking to each other. The address in that command should be the one on the CME interface that communicates with CUCM (in other words, the address that was used in step 2 of the section about setting up SRST in CUCM).
Most of the rest of the configurations are optional depending on whether conferencing will be required during an SRST event, and whether music on hold will be used.
The only other required configuration is “create cnf-files”. This takes the configurations and creates new files that will be downloaded to the phones if they ever have to register to CME during SRST.
5) Since we were planning on using conferencing both within and outside of an SRST event, we configured it:
sccp local Loopback0
sccp ccm 10.0.0.1 identifier 2 priority 2 version 7.0
sccp ccm 10.120.0.1 identifier 1 priority 1 version 7.0
sccp ccm group 10
bind interface Loopback0
associate ccm 1 priority 1
associate ccm 2 priority 2
associate profile 12 register CFB-03
dsp services dspfarm
dspfarm profile 12 conference
description conference bridge
maximum sessions 6
associate application SCCP
ephone-dn 500 octo-line
ephone-dn 501 octo-line
In these configurations, “10.0.0.1” is the IP of this device (the one used in the previous step) so phones in SRST mode will call upon this device to act as their conference bridge. “10.120.0.1” is the IP of CUCM. The words, “version 7.0” deal with the version of CME and the version of CUCM that are used in this environment. There is no “8.0” available, by the way, but 7.0 works with an 8.0 system.
We only created ad-hoc conferences for this client, but Meet-Me conferences would have worked too.
Finally, “CFB-03” is the name of the conference bridge. Notice it has to be the same as the name referenced under the “telephony-service” command (in step 4).
6) The next thing we did was create the ephone-dns. The following are some typical examples:
ephone-dn 2 dual-line
ephone-dn 3 octo-line
Note that you don’t create speed-dial numbers. They will pop up on the phones that expect them. BLFs, on the other hand, will not. They won’t even show up as speed dials.
In the above example a few things are going on. In this environment, we have both private numbers and DIDs. The private numbers begin with a “3” and the DIDs begin with a “4” (the 10-digit number is cut down to the last 4 digits by the telco before it’s sent to our client’s office). In the first example, the ephone-dn is a single line. We know this because if no line-type is mentioned it falls back on the default line type configured in “telephony-service”. In our case this was “srst dn line-mode single”. A single line DN is not able to do call-waiting. A “dual-line” DN can do call waiting with just one other person. An “octo-line” can do call waiting with up to 8 people. The reader may have noticed that all of our conference lines were “octo” too.
The “number” is just the actual DN for the line. The “label” is what the display will say next to the line button on the phone. We could also have put a department name or a person’s name there. The “name” is what will be used as caller-ID for calls going to or coming from the line.
Notice the only line that has the “preference 1” command is the one that’s a DID line. There’s a reason for this that we’ll get to in a second. Stick around, it’s juicy.
7) A couple other things we did for this client was create a call park number and a hunt list. They were configured as follows:
name Park Slot 1
park-slot timeout 60 limit 3
ephone-hunt peer 1
list 4001, 4002
The Park configuration will display a call park number of *123 to the person who does the parking. That number will then have to be called from another phone to pick up the person on hold. The “name” given to the park slot will also be displayed.
The “park-slot” command tells CME that this is a call park number. It also says, in this example, that CME should wait 60 seconds before reminding the person who put the call in park that someone is still waiting on hold. It reminds the person 2 times and then after the third interval of 60 seconds, it hangs up.
The hunt group has a pilot of 4000 which triggers it. Extension 4001 will ring first. After 15 seconds of ringing, 4002 will ring. After another 15 seconds the number of hops runs out, and the destination of last resort, 4003, will ring.
8) Finally we get to the dial-peers. This is the thing that drove us out of our tiny, little minds, and it’s the reason we wrote this blog.
First, we’re assuming that the gateway in question is an H.323 (or SIP) voice gateway that was working before we messed with it. We’re also assuming that there are already incoming and outgoing dial-peers configured that point calls from CUCM to the PSTN and from the PSTN back to CUCM. If a person has followed all of the above steps, and they are not yet in SRST mode, they will notice that they can make calls from anywhere out to the PSTN. They will also notice that internal phones can make calls to other internal numbers. But no calls from the PSTN can be made to any internal DID. In effect, all that hard work has broken inbound calling.
The reason for all of this can be seen by doing a “show dialplan number” followed by an internal DID that someone may want to call from the PSTN. The readout displays the fact that the called number matches a dial-peer up in the 20000 range that hasn’t been configured by the administrator. What’s happening is that when an ephone-dn is created, a dial-peer is also created (automatically) that corresponds to the ephone-dn. This dial-peer isn’t shown in a “show running-config”, but one can see it using a “show dial-peer summary”.
This show command also displays the fact that the dial-peer has the same preference (0) as the one that had previously been created to point outside calls to CUCM. Since that manually created dial-peer usually has a destination pattern with wildcards (ex. “destination-pattern 4…”) and the automatically-created dial-peer matches the DN exactly (ex. “destination-pattern 4001$”) which one will the gateway choose? You betcha. It’ll choose the one that has a more exact match. And since the dial-peer with the exact match points to an ephone-dn that doesn’t have an ephone paired to it yet (if we’re not in SRST), the call goes nowhere.
Remember in step 6 when we put a “preference 1” on ephone-dn 3? That was the first thing we had to do to fix this issue. Putting that statement into the ephone-dn causes the preference to also be put into the automatically created dial-peer.
The second thing was to manually create a dial-peer for that phone, point it to CUCM, and allow it to have the default preference of 0 which is a better preference than 1.
And realize that a specific dial-peer has to be created for EVERY phone that has a DID assigned to it. Think back to the beginning of the article when we said there were three problems with using CME for SRST. This is the third. If the ISR is used in an office with several hundred DIDs, not only does an administrator have to create an ephone-dn for every number, the person also has to create a dial-peer for every number that’s a DID. That could result in one huge running-configuration.
Realize, also, that the destination-pattern in each dial-peer MUST be the exact DN followed by a dollar sign ($). This symbol is the “end of string” match and it basically means “this many digits and no more”. It’s considered by CME to be a better match than just the digit string and nothing else. So “destination-pattern 4001$” is a better match than “destination-pattern 4001”. The dollar sign is in the destination-pattern of the dial-peer that’s created automatically. If the dollar sign isn’t in the dial-peer created manually, the automatic dial-peer will still be the one chosen even if it has a less attractive preference number.
With all that knowledge, the following is what we created for DN 4001. Remember this is taking calls coming from the PSTN and sending them to CUCM, so it has to be a voip dial-peer:
dial-peer voice 2000 voip
session target ipv4:10.120.0.1
The last two configurations are optional depending on the specific environment.
As always, if anyone has any questions, send them along. We’ll be happy to answer them.
Network Stability Through Resilience Engineering
Cloud Security 101
BGP Traffic Engineering
Nick has over 20 years of experience in Security Operations and Security Sales. He is an avid student of cybersecurity and regularly engages with the Infosec community at events like BSides, RVASec, Derbycon and more. The son of an FBI forensics director, Nick holds a B.S. in Criminal Justice and is one of Cisco’s Fire Jumper Elite members. When he’s not working, he writes cyberpunk and punches aliens on his Playstation.
Virgilio “Bong” has sixteen years of professional experience in IT industry from academe, technical and customer support, pre-sales, post sales, project management, training and enablement. He has worked in Cisco Technical Assistance Center (TAC) as a member of the WAN and LAN Switching team. Bong now works for Tech Data as the Field Solutions Architect with a focus on Cisco Security and holds a few Cisco certifications including Fire Jumper Elite.
John is our CTO and the practice lead for a talented team of consultants focused on designing and delivering scalable and secure infrastructure solutions to customers across multiple industry verticals and technologies. Previously he has held several positions including Executive Director/Chief Architect for Global Network Services at JPMorgan Chase. In that capacity, he led a team managing network architecture and services. Prior to his role at JPMorgan Chase, John was a Distinguished Engineer at Cisco working across a number of verticals including Higher Education, Finance, Retail, Government, and Health Care.
He is an expert in working with groups to identify business needs, and align technology strategies to enable business strategies, building in agility and scalability to allow for future changes. John is experienced in the architecture and design of highly available, secure, network infrastructure and data centers, and has worked on projects worldwide. He has worked in both the business and regulatory environments for the design and deployment of complex IT infrastructures.