Rethinking Internet Search Engines

New start-ups are rethinking how search engines can work. So far, we have text-based searching, augmented with some algorithm that ranks sites based on the number of links to that site, and possibly other factors to estimate the value of the page. What other ways could we devise to sort through the huge volume of the World Wide Web? For millenia, the main information problem was finding it. Now, our problem is to sort through the mass of it for the most reliable and useful bits.

It's worth checking out Search Engine Showdown here to see what is happening. They include a very helpful news feature and reviews of the largest search engines. They have a very handy table that lays out the features of the search engines they review side-by-side, for easy comparison. But let's think outside the box. I was struck by ALEXA search technique which looks at where other web surfers have gone before and after visiting a site. They don't rely so heavily on relevance of the text on the site, but use surfer behavior as a guide to what is connected on the Web. What are some other features that could help manage the ocean of knowledge on the Web?

Hakia Inc., a New York-based start-up, is engaged in building what it calls a "meaning-based" search engine, said Melek Pulatkonak, chief operating officer, at a Tuesday conference hosted by Piper Jaffray in New York.

"Search is primitive now," Pulatkonak said. Instead of simply matching search terms to popular results, Pulatkonak said Hakia hopes to create an engine that understands what someone searching the Web is really looking for -- answering a user's "why" question, or giving results that are time-sensitive, like "investment conferences taking place right now."

While Pulatkonak gave few technical details, she said the company's engine can understand what Web pages mean, not just what words they contain.

See the Hakia Beta at this link. I tested it briefly and am not sure my socks have been knocked off. But I plan on taking more time to look.

Another model, that takes advantage of IA (Intelligence Augmentation, see post here), is Cha-Cha here. Cha Cha claims to be the worlds largest live search site. I tested it with the same query I used on Hakia and Google, looking at "law office knowledge management." The difference with Cha Cha is that you connect for a live chat with a "guide." It's a lot like having live chat reference, except you know nothing of the guide's qualifications. They were searching "the deep web," PDF files and databases within websites.

I have been fascinated in the past by knowledge mapping techniques. I wonder if anybody is working on that as a way to augment a search engine? I guess it's already sort of there with "more like this," and "Search Results Plus" on Westlaw. Other ideas?

The Thinker, by Rodin, is from the University of Kentucky.