What's in a certification?

14.12.2006
I certainly wasn't expecting a rooster to start crowing as I hit question 50 on my information security certification exam this past Saturday. Then again, not much had gone as I'd anticipated. Soon after number 50, a noisy cow was driven back to the nearby hillside, and the din outside the wide-open school lunchroom windows was reduced to the distant clatter of cars and honking on the nearby outskirts of Pune, India.

I was the only American in the room. Afterward, several people asked why I'd take an exam half a world away from home. Why here indeed, and why at all? I wondered that myself, having completed the higher-level certification several years ago.

Certainly there's value in security certifications, even if respect for many of the vendor-specific certifications -- notably Microsoft's Certified Systems Administrator () and Software Engineer () -- has decreased. But the broad idea of professional certification hasn't fallen out of favor, and certifications still make sense in the information technology or security industry.

In fact, I think Microsoft's current perception problem is due to specific missteps: flooding the market with certified administrators and software developers, and since-reversed mistakes related to the forced expiry of certifications according to product release cycles. Cisco, for example, has managed to retain a bit more cachet for the Cisco Certified Network Administrator () and Internetwork Engineer () simply through reasonable rigor and a touch of scarcity. The percentage of people with insufficient among CCNAs may be the same as among MCSAs, for example, but there's a clear difference in perception.

Two decades ago, Novell made important early strides in vendor-driven certification with its Certified Netware Engineer () designation. The CNE was not only one of the first well-marketed extensions of engineering designations from other industries, but it provided a much-needed bridge between entry-level network cable-jockey and support jobs into the realm of respected professional roles.

Through this widely recognizable structured route, technically competent IT workers who might lack the social skills necessary to advance in a highly social professional environment could assert their merit without limitation from managers, employers and even industry. Potential advancement, new employers and peers could in turn recognize a competent individual by the designation.

That was the idea, at least. As more companies adopted the model, pressure increased for enterprise software and network clients to adopt vendor-certified implementation processes and people. At the same time, test mills expanded from the already-lucrative college and graduate school exam-prep market into the realm of professional IT certifications -- and churned out waves of certificate holders with no experience. Adding a third axis were a few vendor-independent organizations, making much noise about their certifications denoting distinguished experts rather than just competence. Reality, as usual, was somewhere in the middle.

Microsoft and other vendors started to bind their certifications to products, and then to specific versions of those projects. (Novell, interestingly, went against the grain when it acquired UnixWare in 1993, broadening and renaming the CNE designation to Certified Novell Engineer.) While this is great for short-term projects where a product revision cycle is longer than the average tenure of an entry-level employee or temp, it reduces relevance for clients seeking to make a long-term investment in qualified persons -- i.e. hiring.

Worse, some of the independent certifications expanded too quickly and, in my own experience, quality fell through the floor. The Global Information Assurance Certification () series offered by the SANS Institute got off to a good start in a few technical areas, but when I sat for the GIAC Information Security Officer () certification exams (there were two), they had numerous repeated questions, spelling and grammatical errors and other indications of lax review and immaturity. The paper I wrote for the certification -- which SANS later posted to its "most popular security resources" page -- was returned with erroneous edits and off-base comments. Never have I been so annoyed about passing an exam.

The Certified Information Systems Security Professional () exam administered by the International Information Systems Security Certification Consortium () was clearly better prepared, edited and reviewed, even in its first iteration. When I went through the process, some of the ISC2 test preparation materials and even the guidance during the exam were labeled "Beta" or "Version 0.9."

No annoyance there, even without knowledge of the newness of the program, because there was a clear sense of respect for applied knowledge over rote behavior. The book upon which much of the CISSP program was based, "Information Systems Security: A Pracitioner's Reference" written by Phil Fites and Martin Kratz with its rather dated 15 domains of security, is still a useful and insightful tome. (It's long out of print, but can be found .)

The most pleasant experience -- if there's any pleasure in quickly being put in one's place -- was sitting for the Certified Information Systems Auditor () examination some years ago. Administered by the Information Systems Audit and Control Association (ISACA), the test accurately covered what it was supposed to, had no errors of organzation of grammar as I recall, and honestly was pretty hard. Twenty-five years of test administration and review was clearly evident. But that wasn't all -- in order to receive the designation, I had to provide proof of work experience, and find two people in the industry to vouch for my qualification and capability. That's a lot closer to the notion of a journeyman.

After this most recent test, scheduled away from home to match this month's work schedule, I met with several members of the Pune ISACA chapter and was invited to give a short talk to the membership. I now know that many of the concerns about certification are the same worldwide. Confidence in the quality and applicability of any certification is offset by uncertainty about its perception and worth in the market.

It boils down to this: The purpose of any professional designation beyond a simple marketing label is to assert capability quickly and efficiently, and shorten the social-dance portion of a potentially productive conversation. A certification, like the notion of a journeyman, means much more if it includes peer evaluation. Being able to assert that professional peers would trust and depend on you means more than any point-in-time evaluation of product-specific or even scenario-specific functional skills.

It's an idea that crosses both industry and international boundaries. This is particularly important for those who might sit a world away from their clients or partners. I might not be able to watch the eyes of a distant potential colleague or service provider while discussing a topic that would reveal knowledge or ignorance, but others can. And in that, there's a world of difference.

Note: Many thanks to Kumar M.S. and the ISACA Pune Board for inviting me to speak and share experiences at the chapter meeting in Pune, Maharashtra. I'm deeply encouraged by the interest and devotion of the information security community members, both new and old. Thank you.

Jon Espenschied has been at play in the security industry for enough years to become enthusiastic, blas', cynical, jaded, content and enthusiastic again. He is currently a security consultant in Seattle, where his advice has been ignored by CEOs, auditors and sysadmins alike.