• by Mark Lieberman , Op-Ed Contributor, March 21, 2011

Last week I attended a Roundtable Breakfast at the Paley Center with Google's Marissa Mayer. She mentioned a content-recommendation study at Carnegie Mellon that piqued my interest. The project, dubbed Elvis, had found that collaborative filtering  -- aka recommendations  -- is most effective and most interesting when it relies not on a largest-possible sample of participants, but on a midsize sample that allows for serendipity.

When a population of up to about 50 subjects used Elvis for music recommendations, the results were random and unreliable: The user whose profile best matched yours actually didn't share much in common with you. At the other end of the spectrum, with a too-large user set of 2500-plus, recommendations were uninteresting and predictable; you met your musical twin.

It was in the middle that things got interesting. The best collaborative-filtering algorithms, Mayer said, incorporate a stochastic element to provide a little randomness. It's what makes the PANDORA Internet Radio System, which I love, inspire such loyalty from its users. 

It's in the middle of the continuum that things get interesting for media analytics as well.

Case in point: Just a few years ago, if you had a television spot you wanted to air, you didn't need to know much about your audience. What did the Ratings Book say? Were you targeting men or women? Older or younger?

Then a bit more sophistication was added. Gen X or Baby Boomers? Urban or suburban? And recently, a little bit more.  Want women who like action movies? Now you can have them. But the decisions were still pretty simple, because your options remained fairly limited. 

Like the Model T, you could get any color you wanted as long as it was black or white (or maybe now grey).

Not anymore. With the explosion of digital databases, we've gained the ability to slice and dice the data to find exactly the right consumer for a given product. But now just the opposite problem exists: the risk of paralysis by analysis.  Too much data, too many choices.  A veritable Tower of Babel.

So how do we provide a compelling media analytics solution to marketers who want to define and target niche populations and measure the results of their efforts?

Aim for  the middle of that analytics continuum.

Give advertisers enough flexibility to customize their own solutions, but not so much that they become overwhelmed by the possibilities (no paralysis by analysis here) or put off by the blunt instruments of the past, the amorphous "women 18 to 49" and its equivalent segments. Guide them in thinking about the numbers, without defining their segments for them. And let them define their own segments in whatever way best supports their business, then leave the system and processes to do the rest.

All without eliminating the serendipity that will keep the analysts engaged and customers loyal.