SinbadSearch: The web without a back button

I am a huge fan of data, or screen-scraping. It allows a programmer to abstract a complex user interface (like a website) and have it make sense at a new, lower level so anyone can take advantage of the site's not-so-obvious features or to make your web application much more quickly accessible. For instance, you could possibly write a news aggregator for whitehouse.gov if you were so inclined.

There have been few examples of some neat rich clients that use web services and screen-scraping attempting to make the use of one or more websites an easier experience. Apple has come through with a really cool application that never took off called "Sherlock", and there was also it's "inspiration" to Apple: "Watson".

These apps, I feel, did not harness the full potential they had and work only as searching tools. It seems almost as if the developers added another complex layer, rather then simplifying the process. There is a new player on the market, SinbadSearch, making it's effort to basically do… exactly the same thing as these two by simplifying web searches from popular sites.

I congratulate the developers of SinbadSearch and wish all kinds of success for them. But I have a few critiques to it's design and more in general to everyone making a data scraping application.

What's good about SinbadSearch is that it takes data from very different sources and displays it all in a very similar fashion. This is very cool and makes it very easy for a user to adapt to different websites and search engines quickly and easily.

But, the problem I see with an app like Sinbad or Watson or Sherlock is that they are not very high-level abstractions of the web data, but simply function as a stripped down web browser without most of the browser's intuitive and genius features (a back button, bookmarks, tabs) that have evolved throughout the years. The problem is that these rich clients are a simple aggregation of data, rather than a simplification of design.

So how do we fix that? Do we just add on those features that make a web browser amazing to our scraping apps? I say, no… only because those features add complexity which is obviously not a design goal of this application.

What I want to see happen is not just the paradigm of "search, view results, refine search, view results, repeat" but new, creative and inventive ideas that transcend the browser experience.

A current example is The iTunes Music Store. iTunes is a great product, and Apple really did a great job integrating it's online store into the interface. While it still retains some browser features, like home back and forward, it also enhances the interface my integrating into your music player, providing search results for new music the same way you would search through your library. This is absolutely brilliant.

How about making suggestions for your searches? Predictive searches like Google Suggest?

Analyze the data returned from a search string and start figuring out how the links on each page relate to your search and find possible better or related results.

Or how about some site interaction?

Let's allow wikipedia articles to be opened in an Rich WYSIWYG editor and make changes with Sinbad, instead of just displaying the HTML. I have a widget for that!

Let's be able to gather prices from different sources (like half.com, amazon, barnesandnoble, etc) and pick the best price and enter in order information (credit card, address) within the confines of a pretty rich client.

I see a future in this kind of stuff. For now, SinbadSearch is solving a problem that doesn't really exist. We know how to enter a text string into a website to search it. We know how to grep the results. Let's make it more convenient for the user to use much more of the website than just the search feature.

related links
Sinbad Search
Watson
Sherlock (wikipedia article)