Staying Minutes Ahead Of The Next Twitter Trend

If online ad targeting continues to follow the lead of Wall Street in relying on automated systems that work at real-time speeds, then we may be looking at a future where some marketer chase trends by the nano-second. Among stock traders, machine trading has become so efficient and and inhumanly fast that algorithms hunt down every hint of a trend in order to detect and capitalize on it before competitors dilute the first mover advantage.

That was the analogy that came to mind when MIT announced that one of its research teams is claiming to get ahead of the Twitter trending curve. In coming weeks, Associate Professor Devavrat Shah and his student Stanislav Nokolov will present  a new algorithm that can anticipate what topics will appear atop the Twitter trending stories list an average 90 minutes before they appear. The duo, which is from MIT’s Interdisciplinary Workshop on Information and Decision in Social Networks, says that the algorithm can make these predictions with 95% accuracy, and many times they can tell four and five hours out what topics will be appearing as a Twitter Trend.

The researchers say they use machine learning algorithms that pore over data sets of topics that have and have not trended to find patterns. They say that the big difference in their approach is that the search for patterns makes no assumptions about the patterns it should find. In other words, the search imposes no hypothesis of what a trend should and should resemble. If I am understanding what these guys do correctly, they are letting “the data decide,” as Prof. Shah describes it.

I will just mangle getting the details right, so let me quote at length from MIT’s explanation of the process: “In particular, their algorithm compares changes over time in the number of tweets about each new topic to the changes over time of every sample in the training set. Samples whose statistics resemble those of the new topic are given more weight in predicting whether the new topic will trend or not. In effect, Shah explains, each sample ‘votes’ on whether the new topic will trend, but some samples’ votes count more than others’. The weighted votes are then combined, giving a probabilistic estimate of the likelihood that the new topic will trend.”

The researchers used 200 Twitter topics that did trend and another 200 that did not trend as the sample set that “voted” on the topics analyzed. They say that accuracy will only improve by expanding the data sets. They also suggest that the basic approach could be applicable to a number of cases where a quantity changes over time, such as ticket sales or stocks.

More immediately, the algorithm certainly could be applied by Twitter itself or third parties in selling keywords and topics to advertisers looking to get the same edge on targeting that a nano-second stock trader gets on micro-shifts in the market. Ultimately, it is an open question what kinds of products and offers really would benefit from this kind of 90-minute advance knowledge of social media trending. But the idea of real-time targeting in advance of trends certainly is an interesting one if applied to a wider variety of ad types, social and search platforms. Perhaps even more valuable for marketers would be segmenting the trends so they could aggregate social inventory by about-to-trend areas. And of course, this MIT-bred algorithm is addressing only the content side of the equation, not the audience. Ideally, you want to know with whom a topic is trending.

And of course, this is just the sort of thing that invites an algorithm arms race, with everyone trying to predict trends earlier and earlier in the cycle. Perhaps the natural logic of a real-time information morass like the Internet presumes faster is by definition better. But there must be diminishing returns to this. Sure, these kinds of predictive analytics applied to ad targeting can help a platform identify a trend, and lock in the most precious inventory at an advantageous price before anyone else. But unless there is a real strategy and messaging to apply against that “win,” what value does it add? The machine can tell you where to aim, but at some point it takes us sluggish humans to ensure you don’t shoot blanks.     

Recommend (2)
2 comments about "Staying Minutes Ahead Of The Next Twitter Trend ".
  1. Pete Austin from Triggered Messaging , November 3, 2012 at 2:44 p.m.
    Machine Trading is about making bad predictions about what will happen in a few milliseconds (they only need to be right slightly more than a random guess to make money), while this is about allegedly making good predictions (95%!) an hour and a half ahead. Apart from these both using computers, I don't see the similarity. But can the researchers get 95% accuracy? Of course - anyone can do that. Just choose twitter topics completely at random, guess they will not trend, and you'll get 99%+ accurancy. Unless these researchers pin down exactly what they, are predicting a lot more precisely than in this report, then it's not interesting to marketers.
  2. Paula Lynn from Who Else Unlimited , November 30, 2012 at 7:28 p.m.
    It's not a tell us what you think but we'll tell you what to think.