Of course, the reliability of those third-party certifiers would depend on the quality of their test suites. If every certifier gins up its own tests, that quality could be all over the map.

But it doesn't have to be -- not if SANS and Mitre and their partners sponsor development of a standard test suite and then make it freely available.

Think about it. Those third-party certification companies would gladly use that test suite, because the certifiers would be off the hook for any top-25 errors the test suite fails to find.

Software providers would happily use the test suite to make sure their code would achieve third-party certification on the first pass.

Security companies would fall all over themselves to discover top-25 errors that could get past the test suite. They'd issue their press releases, the test suite would be updated, and the new version would be the new standard.