Newly hired as IT vice president, Bill Chapman believed the only way to accomplish this shift was to embrace a component-based software architecture that used common capabilities wherever possible, instead of a rat's nest of point-to-point integration. "The concept was to create generic interconnectivity throughout all significant applications and connections, including external ones," says Chapman, who is now Avnet's CTO. Another key requirement: Use existing systems wherever possible to keep costs down. "Distribution is a lean business, so we want to leverage our solutions over and over again," he says.
Integrating the diverse systems required a modular, flexible approach for which a service-based architecture -- later to be known as SOA -- is a natural. But before Avnet could develop common services, it needed a consistent operational data model that ensured consistent results when transactions were executed, no matter who the customer was or which Avnet system was doing the work.
Building a common data model
"What we do with the data is where the SOA core exists," says Sean Valcamp, director of enterprise integration. Hired shortly after Chapman, Valcamp and his team of five data architects designed the basic component architecture that defined a separate operational data layer as the groundwork for what became the company's SOA environment.
Using a canonical model, they created the service definitions for each of the various domains in the data hierarchy, as well as the structures and relationships among them. The team then implemented the data models into an ODS (operational data store) to augment application knowledge bases and business transactions.
Avnet, for example, created a model to represent people, organizations, and business units -- as well as companies and relationships between companies and people -- to supplement LDAP profiles. This was crucial because it let Avnet track relationships between suppliers and customers, so, for instance, a customer who resells only Hewlett-Packard products couldn't see IBM product pricing.
Linking inside and out
Avnet initially focused on simplifying the integration of its many internal systems. Rip-and-replace was out of the question because of the high cost. But Chapman also wanted to reduce the cost and complexity of maintaining existing point-to-point connections.
The answer was to leverage Avnet's investment in webMethods middleware, replacing point-to-point connectors with generalized services that ensured any change to either a process or data model was consistently applied to all systems around the globe. "The goal was to represent common business processes across different suppliers," Chapman says. The architecture team mapped out the functions in all of the legacy systems to determine the capabilities, inconsistencies, and gaps. Then developers began creating services to handle translation, fill in missing pieces, and otherwise ensure common functionality across all systems. Essentially, Avnet pursued systems integration via services.
Chapman had always envisioned extending the same approach to external clients, since he expected them to use multiple channels to interact with Avnet when the e-commerce offering went live. But that day came sooner than expected, when HP, a major client, said in early 2000 that it wanted Avnet to support the RosettaNet e-commerce protocols. "That exposed the need to change our internal systems infrastructure," Chapman recalls. The company still relied on direct connections to and from its clients, with individual APIs for each. To do RosettaNet right, that needed to change.
The solution was to separate the customer interactions from internal systems, so customers could choose their preferred way of connecting using an extranet portal without affecting the operations of Avnet's internal systems. "We could now create a custom layer to adapt to our customer needs: b-to-b your way, not b-to-b our way," Chapman says. Those choices ranged from direct links to customer ERP systems to Web pages that enabled customers to enter orders manually.
As it had done with its internal systems, Avnet deployed wrappers around key legacy applications and created new services for missing functions, all accessed through generic APIs at the portal. In some cases, that meant providing adapters for the client to use; in others, it meant transforming data to whatever format the customer preferred, from IDoc to EDI to RosettaNet.
The M&A challenge
Not long after the RosettaNet effort, Avnet faced a new integration challenge: The company purchased IBM distributor the Savoir Technology Group in summer 2000. Along with the business came new applications that Avnet wanted to make available to both HP and its storage product clients.
Avnet, however, first simply had to make the Savoir front end work with the Avnet back end. "We took all their front-end apps and had them running on our system in three months," Chapman notes. The company performed this trick by creating new services to translate Savoir's data and basic functions into Avnet's common model.
With back-end support in place, Avnet could extend its Savoir front end to all its customers. It did so by refactoring Savoir's 31 applications -- including order status, returns management, invoice management, customer security management, lead management, credit reporting, and invoicing -- as services that would run alongside legacy Avnet systems, rather than forcing all systems to rely on a single Savoir platform.
Today, information arrives in various forms from various clients, passing into a set of services that translates it according to the operational data layer's standards while flagging anomalies along the way. Those services then move the data to Avnet's various ERP and transaction systems in the formats required, as well as ensure process consistency no matter what applications a particular Avnet division happens to use.
The end result is a loosely coupled framework inside and out. "This architecture allows any application within the Avnet environment to readily provide services to other applications or to trading partners through any one or combination of communication vehicles, such as XML, objects, JMS [Java Message Service], FTP, RosettaNet, EDI, and e-mail," Valcamp says.
Building block by block
As Avnet built out its services, it established common-sense criteria to ensure a successful outcome. First, each service needed to be reusable elsewhere, so its development costs could be spread out. Second, the service abstraction had to be able to accommodate data representations that were likely to change frequently. Third, the locations of the data sources themselves had to be able to change often. As Valcamp notes, "All you are concerned with is calling an interface, not where the interface is."
At the same time, Avnet wanted to reuse its existing technology and human skills wherever possible. For example, the company was using webMethods as its messaging middleware when the services effort began in 1999. There was no commercial ESB (enterprise service bus) at the time. So Avnet used that middleware as its common gateway to route data and function calls among services, portals, and applications, as well as to provision its service policies, such as those for security.
Where webMethods didn't provide needed ESB functionality, Avnet added it, such as augmented translation services for mapping EDI into XML. As webMethods added more service-oriented capabilities over time -- such as a services directory, common SOAP implementation, and SOA governance capabilities -- Avnet used off-the-shelf webMethods features rather than continue to support its homegrown code. By late 2002, Avnet had deployed an upgraded webMethods as its ESB.
Similarly, because Web services standards and JMS were still evolving, Avnet had to build its own messaging hooks using webMethods or proprietary interfaces to SAP, AS/400 applications, and IMS-based systems for translation, routing, and common communication standards. "Our needs were ahead of where the product suites were," Valcamp says.
Another example of leveraging existing technology is Avnet's continued use of multiple ERP systems, whose number grew as the company went on a global acquisition spree. The Technology Solutions business unit in Europe, for example, uses the Command Frida ERP system. Although the system has been around for more than a decade, Chapman says, "With SOA we can augment it to make it a state-of-the-art system rather than retire it. Now we can decide the retirement road map based on IT issues, not business process needs."
When the company eventually replaces Frida with an SAP system, the SOA approach means that the processes, data flow, and data model won't change, so the business will be able to operate as is rather than adapt to SAP's approach -- or require an expensive integration effort to make SAP work the Avnet way, he says.
Similarly, Avnet can continue to let its programming staff use the development environments they know best, such as Java and Microsoft .Net. As long as services have the same interfaces and data models, they can exchange data and results consistently, Valcamp says. "We were not going to drop .Net to do just Java, or vice versa," he notes. The key to successful heterogeneity, Chapman says, is in the services' design. Developers need to take the time to ensure they are truly abstracted, with no dependencies based on the flavor of technology.
Calculating the payoff
The SOA approach has also provided measurable benefits for application development, Chapman says, such as a 30 percent more efficient quote-to-order process, with an associated reduction in cycle times for order management to minutes from more than a day. Development time at Avnet was reduced by 45 percent for IT projects using SOA versus those that weren't.
Some benefits are harder to quantify, because they come from increased business activity, presumably due to the company's ability to execute faster and more effectively.
Chapman believes, for example, that Avnet boosted its software distribution business from tens of millions in sales to hundreds of millions of dollars by providing EDI links to its customer base over a two-month period. And he says that a new integrated quoting process, implemented as a set of services, reduced one division's labor usage by 20,000 hours per year -- an efficiency gain of 20 percent -- and simplified the process for clients, as well.
Chapman says the new order process alone has boosted revenues by more than $200 million without staffing increases. With ROI like that, the pain of transforming architecture is worth it.