Bank of America rolling out VOIP

18.11.2006
Craig Hinkley, senior vice president of network services for Bank of America Corp., is overseeing an enormous voice-over-IP (VOIP) project with Electronic Data Services Inc. (EDS) that will bring 180,000 IP phones to the bank's U.S. operations. Although the massive project isn't expected to be completed for another two years or so, Hinkley talked this week about how the effort is going.

Excerpts from that interview follow:

You talked at VoiceCon about the bank's VOIP project in February 2005. Is it still on target? Yes, the target is still 180,000 IP phones, and it's going along nicely. They are Cisco IP-based phones, and we've actually got the program split into three separate areas for branch retail, enterprise and call centers. We split it into three efforts to focus on each subprogram and to get it running without trying to boil the ocean, so to speak.

In 2005, the first order of business was to get the consumer retail program up and running. Of 180,000 total phones, we have roughly 60,000 in retail, 60,000 in enterprise and 60,000 in a number of contact centers. We first focused on retail, with 6,000 branches, and had to get that program up and running. It's doing well, with 800 out of 6,000 branches completed. We are doing this with EDS, our outsourcer that we contracted with in 2003, which will deploy and manage it.

What about enterprise and contact centers? Enterprise is started, and we've completed 50 enterprise locations. We're kicking off the contact center effort before the end of the year, and the first location will be a medium-size contact center for internal operations used to serve bank associates in the Charlotte, N.C., area. We're adopting the Microsoft methodology to eat your own dog food, working with our people first.

So, today, you have how many phones installed? We are north of 20,000. It's going very well. From a technology perspective, it's going well, with the processes and the program. I would say that what we've done in the last 18 months is focus on establishing the program, processes and standards of what we're deploying. What we've had to do is take heritage PBX switches that have been out there 10 to 15 years and peel back the layers of the onion to understand things and determine the standard telephony interaction model to support the business. We're realizing we can now deploy ubiquitous features and functions with IP. It translates to the business, so I can have now a common telephony interaction model in the U.S. We've had lot of conversations with divisional executives about how we want VOIP nationally to support business processes. It's not just been a technology transformation, but a business process transformation.

But with 20,000 phones after nearly two years out of 180,000 expected, are you still on schedule? We are a little off pace on the schedule, but that's not contributory to the technology. It's really us making sure we have the right deployment processes and the right technology standards. We started out a little slower out of the gate to understand how to use the technology in our environment. The last thing we wanted to do was go out and just deploy technology and come back and realize that all we did was duplicate what was out there. What's out there today is 468 PBX TDM [time division multiplexer] switches. The last thing we wanted to do was create a like-for-like situation.

How many IP-PBXs or equivalents will you install? That's one of the things we're doing -- significantly reducing that physical footprint of PBXs needed to support the same number of endpoints. With PBX virtualization, it will be about a third of what we have today. It could be less.

There are some carriers and technology companies, such as IBM, that might have larger VOIP rollouts, but do you know of any others by nontechnology companies that are larger? I don't know of any project that's larger that a single enterprise is doing.

What are your overall costs? We're a bank. We don't go there. It's confidential.

EDS is doing all the installs and ongoing support? Yes.

How is the technology working, such as sound quality, on the phones? From a quality and technology perspective, it's working extremely well for us. We went through a very extensive planning process around standardization of the technology. [We] also went through a process of making sure that we provide the same level of service in the VOIP world that is comparable to what users had before. From a quality perspective, it's been right on as far as delivering with our expectations.

You upgraded the IP network backbone to help provide that quality? In terms of the upgrades, we had a long-term strategy that the bank has been executing for more than three years. We had already implemented an Optical Carrier 49 dual-IP backbone over three years ago. So we already had a backbone speed capability to support bringing our voice communications onto the infrastructure. We had refreshed a significant portion of our network switches as part of a standardization program. So even before we deployed VOIP, we were moving into an environment that had received the latest and greatest technology to provide quality of service. That kept quality of service high, and from a security perspective, we could still guarantee voice availability as well.

In 2005, you gave a speech mentioning how this VOIP project would challenge your organization to bring together voice and data IT workers. How has that gone? It was not a problem, but candidly, it was a potential factor we recognized at the start. We clearly clarified that this project was a transformation, not just from a technology perspective, but from a people-and-process perspective as well. We had already worked hard on integrating the data and voice teams, and EDS is responsible for deploying and operating the network. We work transparently with EDS, so we were in there working with them in terms of their operational readiness to support VOIP in 2004 and 2005. We worked with them to make sure that integration within the voice and data teams occurred from Day One.

When is the project supposed to be completed? While we're a little offtrack, we still have our program completion set for late 2008 or early 2009. While we went out slower than expected, we still have enough runway to meet our commitments. We've already taken the time to structure the program, and make sure we have standards in place. Now we are starting to turn the engines on.

Have you added any communications features that weren't used at the bank before, such as unifying voice mail and e-mail? Candidly, we have not yet deployed unified communications in terms of a single box for e-mail and voice mail. We've not done that. We are looking at a multigenerational rollout where we deploy that unified communications capability. What we'll be doing is phasing that in over time. We won't wait until 2008 but will layer that in as we reach penetration in certain areas. We have not deployed any new applications, but they have a new phone with additional features out of the box.

Is there a killer communications application coming from Bank of America? Some banks are talking about mobile phone banking, for example. I wouldn't want to tip our hand.

Are any of your IP phones working as wireless LAN phones? There are around 2,000 of the Cisco wireless IP 7920 phones.

And what is the benefit of these? The benefit of wireless 802.11 technology is analogous to the benefit provided by cordless phones, except with more robust information security control and the ability to continue consolidation of voice and data infrastructures. Currently, the bank uses the 7920s for banking center personnel to have mobility to interact with customers within the retail store environment and still have service customers calling into the store.

Would Bank of America be interested in having the Wi-Fi voice phones work in a dual-mode fashion, handing off the calls to the cellular network at some point? We are exploring the use of dual-band phones as the technology progresses. The integration of these technologies and the ability to reduce the number of devices an associate requires drives it, and it reduces total cost of ownership for hardware.

So on the painful to painless scale, how has this project gone for you personally? I like scales. On a scale where one is completely painless, and 10 is a headache going to the office every morning, I would say it's probably been about a 4. There's certainly been pain because you are going through such a transformation across the magic triangle of people, process and technology. With that said, you are impacting business processes and the way the associates interact with technology. There's been pain in that, but that's been countered by the fact that the technology worked well, and we've had a good solid base to work from; we've had good solid executive support for the program. It's a key enabler for the bank and has been given appropriate support. Hence, we've very much moving forward, full steam ahead.