Google researcher targets Web's structured data

30.01.2009
Internet search engines have focused largely on crawling text on Web pages, but Google is knee-deep in research about how to analyze and organize structured data, a company scientist said Friday.

"There's a lot of structured data out on the Web and we're not doing a good job of presenting it to our users," said during a talk at the New England Database Day conference at the Massachusetts Institute of Technology,

Halevy was referring in part to so-called "deep Web" sources, such as the databases that sit behind form-driven Web sites like Cars.com or Realtor.com. Google has been submitting queries to various forms for , retrieving the resulting Web pages and including them in its search index if the information looks useful.

But the company also wants to analyze the data found in structured tables on many Web sites, Halevy said, offering as an example a table on a Web page that lists the U.S. presidents.

And there are reams of those tables -- Google's index turned up 14 billion of them, according to Halevy. He "realized very quickly that over 98 percent of these are not that interesting," but even after significant filtering there remain about 154 million tables worth indexing, he said.

One of Google's ultimate goals is to provide results that organize "aspects" of a search query, particularly an exploratory one such as "Vietnam travel," as opposed to a query for a specific fact such as "Vietnam population," Halevy said. The former query would produce information about visa requirements, weather and tour packages, for example.