Next generation Internet searching

From today's CNET News.com: Academia's quest for the ultimate search tool:

The University of California at Berkeley is creating an interdisciplinary center for advanced search technologies and is in talks with search giants including Google to join the project, CNET News.com has learned.

The project is one of many efforts at U.S. universities designed to address the explosive growth of Internet search and the complex issues that have arisen in the field.

U.C. Berkeley, birthplace of early search highflier Inktomi and the school where Google CEO Eric Schmidt got his computer science doctoral degree, is bringing together roughly 20 faculty members from various departments to cross-pollinate work on search technology, said Robert Wilensky, the center's director. The principal areas of focus: privacy, fraud, multimedia search and personalization....

The search problems of today are different from those of five years ago. With books, scholarly papers and television programs being digitized and put online, the technology necessary to search through the material needs to be that much better. People need a way to trust the information they find and to ask more-complex questions with search tools so they can extract knowledge or ideas....

CMU is also working under a government grant on a longer-term project called Javelin, focused on question-and-answer search technology. Google, MSN, Ask Jeeves and others already help people find quick answers for word definitions or encyclopedia facts like "What is the population of Los Angeles?" But for complex queries like "What is the cheapest flight from San Francisco to London?" or "Which university has the largest computer science department?" finding answers is still like doing long division.

"This is dynamic information," Carbonell said. "You must parse the question, look for answers in multiple places and do a comparison. There are multiple steps, and we're looking at how to do it in one step and provide a trace for the user."...

Stanford, the Massachusetts Institute of Technology and many other universities are working to solve problems presented by the library of tomorrow, which will be largely digitized. Sifting through and organizing billions of digital documents will require new search technology.

MIT, for example, has teamed with the World Wide Web Consortium to create next-generation search technology using the Semantic Web, in an overarching project called Simile.