New York Times on their own use of collective search intelligence

Here’s a short piece from the NYT Tech Blog on how the New York Times is using realtime analysis of site search to improve results.

Regular readers will know that we’ve been doing this over at the Powerhouse on our OPAC for a long time. The principles are the same and the use of actual users search relationships can greatly assist the navigation of other users.

As the NYT says,

Also Try recommends relevant suggestions of related searches, based on a fairly simple formula. The basic principal is to cluster together all the search queries submitted to nytimes.com in the past week. Then we do a bit of processing to ensure that a cluster has enough variance, and that each node in the cluster actually returns results. Since the system operates on the last week of user submitted queries, the suggestions are very timely and evolve with the news. This helps with the newsiness of the results.

We use a similar method for our ‘similar searches’ and then also implement a Levenshtein algorithm to present a ‘did you mean’ set of results if the search term has no results. Read more on this from my post back in August 2007.

Regular readers will also be wondering where all the new posts on Fresh & New have gone . . . . a whole host are coming very soon. I’ve been busy working on my papers for Museums and the Web 2008 amongst other things.

We are also about to make live a slew of new OPAC features which will add a completely new set of user enhancements to the collection database. We’ve been working a lot on ‘context’ and I’ll be announcing the enhancements soon.

Happy new year.