New Search Engines: Wolfram Alpha & Google Public Data

The Boston Globe today has a story by Hiawatha Bray, "A hungry little number cruncher," featuring mainly Wolfram Alpha.

Computers can't think, but they can count.

That's the idea behind Wolfram Alpha, a new search service that could be as much of a game-changer as Wikipedia or Google. Alpha, created by renowned mathematician, author, and entrepreneur Stephen Wolfram, uses fast computers and vast statistical databases to answer questions just as a human would - a human with advanced degrees in math.

"The goal is to see just how much of the world's knowledge can be made computable," Wolfram said.

Quite a lot, it seems. At lightning speed, Alpha serves up detailed, number-based answers about everything from algebra problems to poker odds.

"It's a compelling new service," said search engine industry analyst Danny Sullivan. "It's very much one to watch."

A few Internet mavens have even suggested Alpha could threaten Google's dominance in Internet search.

Wolfram scoffs at such talk, saying that Alpha is a supplement to traditional search, not a substitute. He's interested in partnering with other search companies, though he wouldn't name names.

Google is exploring similar ideas. On the same day last week that Wolfram demonstrated Alpha at Harvard Law School, Google launched its new Public Data service, which automatically responds to questions about US unemployment or population with graphs generated from the government's statistical databases.

Google Public Data isn't nearly as versatile or powerful as Wolfram Alpha. And unlike Alpha, it didn't grow out of a theoretical quest to make machines think. Google project manager Ola Rosling had a much simpler goal: "Democratizing the access and usefulness of public data, by making it easy to find and use."

Rosling said Google's launch wasn't an attempt to step on Wolfram Alpha's debut. But he said Google plans to make Public Data more like Alpha, enabling it to crunch numbers into practical information for everyday use. "We're definitely on the same side of the street," Rosling said. "But the street is pretty long and pretty broad."

For now, Google Public Search is rudimentary. Users who type queries like "unemployment in Massachusetts" see a graph at the top of the page. Clicking on it provides a more detailed view of the data, as well as links to let you compare Massachusetts to other states and break out the numbers by county.

But Wolfram Alpha, which will be available for public use later this month at wolframalpha.com, offers detailed, math-based responses to a huge variety of questions. To Wolfram, anything that can be expressed as a set of numbers is a computable problem. So he and his colleagues have filed away vast amounts of numerical data on a host of subjects. Alpha uses these numbers to generate practical information for scientists, business people, or anybody else.

Type "International Space Station" into Google, and you will be directed to millions of pages of information about the orbiting science lab. Type the same query into Wolfram Alpha, and you will get a map of the station's precise location over Earth at the moment you typed your query. It will also tell you the next time the station will pass over Boston. It's all calculated instantly from statistics provided by the National Aeronautics and Space Administration.

Apart from Alpha's knack with numbers, the service is good at analyzing a user's questions and surmising what he is looking for. (snip)

But Alpha is a long way from perfect. Vast amounts of useful data are nowhere to be found. (snip)

Wolfram admits Alpha is still relatively ignorant. His team of developers is constantly adding statistical databases to expand the range of the service.

He's also still trying to figure out how to make money from Alpha.

Wolfram said a basic version of the service will always be free, perhaps with advertising support or a corporate sponsor.

In addition, he's planning a paid subscription version that will offer advanced features, such as the ability to download statistical data directly into a desktop spreadsheet program.

Wolfram has no interest in selling the business to Google or any other company that might use Alpha technology to enhance its own search offerings.

"My goal is to make this an authoritative source for knowledge on the Web," he said. "I fully expect to be working on developing this for decades to come."

The Wolfram Alpha search engine should definitely be on librarians's bookmarks as a tool when it becomes available to the public. And here is a nice video and blog post explaining how to use the Google Public Data feature in the Google search engine.

We just launched a new search feature that makes it easy to find and compare public data. So for example, when comparing Santa Clara county data to the national unemployment rate, it becomes clear not only that Santa Clara's peak during 2002-2003 was really dramatic, but also that the recent increase is a bit more drastic than the national rate. (there are images on the page -- go and see it; I'm extracting some text to make it clearer that this is a widget on a regular Google search)

If you go to Google.com and type in [unemployment rate] or [population] followed by a U.S. state or county, you will see the most recent estimates.

Once you click the link, you'll go to an interactive chart that lets you add and remove data for different geographical areas.

In both search engines, they plan to keep adding more. They are searching existing data sets that have been gathered over decades (or longer) by lots of organizations, government agencies and more. These are tools to make the mountains of data more easily accessible to searchers on the web and more easily digestible through visual aids such as graphs and charts. Pretty wonderful aids!