Maley: Source code analysis is also a critical part of our CA2 process. Early on in the certification process that's what we require and it helps us tremendously. But application flaws are not the only thing we look at with our pen testing. Both are critical to our risk mitigation and I don't see one replacing the other. They really go together. With PCI DSS, an important ingredient is vulnerability scanning. An automated pen testing tool allows me to go through and review vulnerability scans and see in real time what kinds of weaknesses can be exploited. I don't see that as something you can replace.

Maley: We don't randomly go out and pen test things. We don't have that kind of time. We use it at a specific point in the CA2 process. We also use it as a specific piece of the compliance process. Meantime, if we suspect something like an SQL injection attack against a certain app, we go back and do pen testing. One innocuous Web page with job descriptions was subject to such attacks. Through pen testing we were able to extract info about every state employee and their dependents through that page. So we shut it down and did a thorough investigation. We keep all our log files and were able to pinpoint the point in time where attackers started trying to target the data. That's the kind of success we have had.