Analyzing search logs from two different commercial search engines, Microsoft researchers found that results repeated in about 40% of multi-query search sessions and users engage differently with repeats than with results shown for the first time. Not a difficult concept to follow.
The research demonstrates how repeated results within search sessions are incorporated into ranking for personalizing search results. The engines used to demonstrate the model promote documents that the searchers are more likely to click on in the future. It calls on previous work that analyzes repeated query results, interpreting clicks, learning from clicks, and user interest modeling. Researchers used query logs from Bing and Yandex to make their point.
After verifying repetition, the researchers analyzed click-through rates (CTRs), which seems most interesting to me. It turns out that repeated search results have substantially lower CTRs than those displayed for the first time. Some repeats are often missed by the user. The earlier impressions are more likely to be clicked, followed by those that have been displayed but skipped, according to the research.
The No. 1 position in the organic results was the only exception. Another interesting point is that there is a consistent drop in CTR across all four rank positions for results that were previously skipped, so once a user skips a result they rarely go back and click on it. So why serve it a second time? Results on Bing and Yandex showed similar trends.
Researchers found that by modeling different aspect of repetition in addition to clicks are important in leveraging repetition and the behaviors associated with it. Future work will consider richer feature sets and long-term repetition of search results in cross-session search tasks. The research shows that combining long-term click-statistic signals in the re-ranking model increased gains, and the overall impact of re-ranking according to non-personalized relevance metrics is minimal.