NSA's phone-records program puts focus on data mining

25.05.2006

The RAID technology that's designed to back up and protect data takes up extra space, Winter noted. Moreover, although the amount of data that disks can contain per spindle doubles every year, the rates at which the disks spin and at which the arms that hold the read-write heads move haven't changed much, according to Winter. "The result of that is, to get good performance on a normal data warehouse, you have to leave the disks partly empty," he said.

Some analysts argue that social network analysis, the data mining technique used most often to determine interconnections between people, isn't particularly effective with call data records alone.

"If the only data you have is what phone number calls what number and how long they talk, trying to figure out who is a terrorist through this 'bottom-up approach' is impossible," said Valdis Krebs, a Cleveland-based consultant who has done work for many defense and federal government IT contractors.

But Brobst said that social network analysis has long been used by telephone companies to do sophisticated calculations for purposes such as figuring out how to best structure their friends-and-family calling plans to appeal to customers and maximize their profits. "The whole point is that you don't know exactly what you're looking for, so you use data mining to search for patterns," Brobst said. "Going the other direction is easy."

Not surprisingly, the NSA isn't talking about its data collection and mining activities. "Given the nature of the work we do, it would be irresponsible to comment on actual or alleged operational issues; therefore, we have no information to provide," NSA spokesman Don Weber said via e-mail. "However, it is important to note that NSA takes its legal responsibilities seriously and operates within the law."