I'm not sure why, but the phrase "real-time search" has somehow largely become associated with the phrase "Twitter search." While much of the conversation around real-time
search centers conceptually on
Twitter search and other engine algorithms that remix their data, the fact is that the Google crawl and many other viable
contributors are critical to getting a fresh and relevant answer in real-time, though they are often lost in the discussion. Make no mistake about it, crawler-based real-time search and
status-update search are both important to the successful development of a state-of-the-art
robust
and relevant real-time search engine.
Real-time search is an entirely different animal from its two main components, "traditional" crawler search, and keyword
discovery via social streams. The end goal in "real-time" search is about getting the best results now or within a recent period of time, versus getting the best results
over history. It is entirely different from the keyword monitoring of a social stream for discovery, and is also different from fresh, crawler-based results, even if they are delivered within a
shorter timeframe. Perhaps the most unique aspects of real-time search (in its current incarnation) are that they are keyword-triggered, viewed in reverse chronological order (or other recency
parameters), and that many of the best results are collated by a human audience. These attributes ultimately differentiate it from either typical keyword-triggered stream view, or keyword search
in your favorite search box.
Sources of "in-the-moment" content, aside from Twitter: Beyond Twitter and Google's up-to-the minute crawl, there are many
other elements of discovery and search that are not often found in a discussion about real-time search, but can be very helpful in obtaining the right answer at the right moment (either individually,
through social discovery, or crawler-based search). Here are just a few examples that, with a little bit of social propagation and a robust crawl, can add to the real-time search experience:
advertisement
advertisement
Forum chatter: Forums and bulletin boards are the core of the Internet's continuing evolution, and were "social" long before the term was coined
in 2004. While forum content is often timely and useful, it has long been buried in the search results. To see an example of how the recency of forum posts is improving via Google search, see
this query for results crawled in the last 24 hours for "obama" sorted in chronological
order.
Microformats: Microformatted attributes (semantic interpretation of information like reviews and locations, among many other uses) could become very useful in
real-time search, as adoption of these formats continues to increase. The Google crawl could conceivably sort this segmented data to help you find what you are looking for in very short
order.
Comments. User participation in the comments sections of online publications and blogs has exploded in the last year. For example, it is not uncommon for
sites like the Huffington Post to get 1,000 or more timely comments on a single story or post. But much of the potential usefulness of this data is currently wasted, as it is either unedited, or
unsorted (much as Twitter streams are today). Adoption of an algorithmic/social ranking approach to comments could change this (see Digg as an example of how mass volumes of comments are parsed
to become useful to its readers).
Google Hot Trends. If people think that Twitter's measly top 10 trending topics are
the apex of social search discovery, then I would recommend they check out Google Hot Trends, or other buzz indexes for some more fascinating
sources of what is being talked about right now, and what people are searching for, right now.
Are we there yet, are we there yet, are we there yet?
Real-time
is in its earliest stages, and while it is fun to prognosticate about the possibilities, the charge with the major engines is clear. Larry Page has stated that real-time is a priority of Google search. Evolution in Twitter search seems to be moving at a snail's pace,
though it is has been reported that company strategists are rethinking their approach. But
there is still a key place for Twitter, and nothing could be better for real-time search than the combining of the two in some fashion. But don't start planning your strategy around
real-time search just yet. This is an innovation that is years in the making, and it is much bigger than Twitter alone.