• by Laurie Sullivan , Staff Writer @lauriesullivan, September 21, 2010

Google research published earlier this year, but recently resurfaced, provides insight into features that appear to focus on Google Instant. This two-part study measures failure and behavioral signals, suggesting people still struggle with search tasks. The findings have become an important addition to an existing body of research focusing on successful search strategies.  

The study also acknowledges search engines can identify signals in real time when searchers have difficulties finding information. Combining these signals with those related to success and failure search strategies discovered in earlier research could be used to build a model to predict user satisfaction in a search session. Google could use the model to learn how often users leave search engines unhappy or how often they become frustrated and need intervention during the search session.

Before getting into the research it's interesting to note Conductor published Monday findings on the impact Google Instant on search behavior and site traffic. The search company studied the search traffic for ten high traffic Web sites across multiple verticals for the week prior to Google Instant and compared it with the week immediately following the launch. Taking at more than 880,000 visits, conductor found in both the week pre- and post-launch, three-quarters of traffic came from search terms with three words or less. And, long tail visits, four words or more, did not increase significantly after the launch.

As for Google's research, failure allowed the Mountain View, Calif., researchers to determine what happens when searchers face serious difficulties finding a piece of information and the signals that identify frustration. The approach combines small-scale lab studies and log analyses to gain a qualitative understanding of how users' search behavior changes when they start having difficulties in search tasks. The researchers created hypotheses testing it with online studies that focus on closed informational search tasks where search success becomes easy to measure.

Researchers conducted two experiments: Usability Lab Study, and Online Study. By analyzing users' behavior in tasks searchers fail on and comparing these behaviors to behaviors in tasks where they succeed, the group aimed to identify signals in user behavior that suggest what it takes to frustrate searchers.

Twenty three people took part in the first study to find answers to questions like what is the name of the Dave Matthews Band studio located outside of Charlottesville, or find the models' names who fell on the runway due to the footwear they wore in a recent Prada fashion show. Google examined the searches they performed and the sites visited to find the answers. The second study of 179 searchers based on processes included questions like "find a map drawing of the flight path of a space shuttle that flew in 2008," or name the "athletic director at Mater Dei High School in Santa Ana California?" (which I would know because my daughter graduated from the high school.) 

Based on the data, Google researchers determined when people have difficulties in a search task, they will spend more time on the search result page, use more natural language/question type queries and/or use advanced operators in their queries, and have a more unsystematic query refinement process.

In the second part of the study results show that in unsuccessful tasks people formulate more question queries, use advanced operators more often, spend a longer time on the results page, formulate the longest query somewhere in the middle of the search session, and spend a larger proportion of the task time on the search results page.

The research also explains early Google research has mainly focused on the differences between experts and novices or mined the log data to find metrics that correlate with success. Instead of looking at individual differences by pre-categorizing users into experts or novices or having to rely only on log data where the level of success is difficult to confirm. For this study, Google researchers used their own perception on difficult vs. easy outlined in the paper.

Google says the research paper was published at the CHI conference in April 2010. It was submitted for publication on Sept. 17, 2009, and the data collection took place earlier in 2009. Google says the paper is not related to Google Instant, (but the findings appear to have provided fodder for change).