Amateurs and pros vie to build new crypto standard

07.11.2008

Hashing is computer talk for finding a way to take a message -- an e-mail, for example -- and representing it with a unique number that appears to be random. Hashing was developed as a way to reduce the computing overhead when programs are doing things like scanning files to see if they've been changed. It's much quicker to compare two hash values than to scan through whole files for changes.

In a cryptographic hash, the number is encrypted, creating a digital signature that can be verified using public key cryptography. In practice, these digital signatures are used to confirm, for example, that a Web site really is the site it claims to be, or that an e-mail message is from the person who claims to have sent it, and that it hasn't been tampered with along the way.

Starting in 2004, researchers led by Shandong University's Wang Xiaoyun found weaknesses in the MD5 and SHA-1 hash algorithms. They discovered that it was easier than had been thought to create two numbers that share the same hash value. In cryptographic parlance this is called a collision, and it's considered a very bad thing because it undermines the integrity of the cryptographic system.

"It's made everybody nervous," said Rivest, an engineering and computer science professor at the Massachusetts Institute of Technology. He led the team that submitted the algorithm in NIST's contest.

The hack showed just why people were nervous: Using MD5, researchers were able to create different .pdf files that shared the same hash value. To illustrate why this is a problem, they published the hash value of the pdf file containing the name of their 2008 U.S. presidential election pick, and then created pdfs with the names of every single candidate, all of which shared that same hash.