• by Jon Last , Columnist, February 9, 2011

Full disclosure ... I'm writing this before the Super Bowl. So, chances are that by the time you read this, you've already consumed countless analyses on which of the $3 million, 30-second TV spots aired during the big game on Sunday were most compelling, funny, recalled, breakthrough, profligate and, on the other end of the spectrum ... actually effective at building the brands that invested in them.

Notice that I am taking care, above, to label the ultimate outcome of a Super Bowl advertising investment to be "effectively building the sponsoring brand." I've purposely moved away from the objective of the ad in question actually selling something, though I am sure that in your readings of the past day or so, there's a purported expert or two that claims to have cracked the code to determine which Super spots did just that. For those buying into that claim, I will gladly sell you a bridge that links Manhattan to the future home of the New Jersey Nets.

As a marketing researcher, I've often and loudly espoused that the notion of accurately quantifying the direct impact of a single advertisement on consumer purchase behavior is often folly. Absent rigorous experimental design, it is virtually impossible.

Add onto this all of the hype, rehash and conversation that surrounds Super Bowl advertising, and I'd maintain that from a methodological perspective, you'd need to send your test sample into a sensory deprivation tank for about three months surrounding the game to remove all of the ancillary exposures that ultimately influence a consumer's mindset and cloud one's cognitive ability to attribute a behavior to a single execution.

Now this isn't necessarily a bad thing. As others have surely mentioned over the past several days, brands that benefit the most from sports marketing are insightful enough to build integrative campaigns that leverage various marketing mix elements in complementary ways that remain true to the brand and relate to the specific activation.

In a test environment, good research can isolate these variable initiatives and measure their impact against different conditions and/or targets. (We call these multi-variate tests). From there, one can actually examine the incremental impact of different campaign elements on brand perceptions. I will continue to strongly advocate that this is a valuable tool that any sports marketer should consider deploying.

But that aside, no doubt by the time you read this, we will all have seen a variety of lists and rankings of the "best" and "worst" Super Bowl ads. Many research firms and others masquerading as researchers will have proclaimed which ads were the funniest, the most likeable, etc. There will be "real time" polls testing unaided and aided recall (no doubt clouded by recollection of prior Super Bowl advertisers), scrapes of the blogosphere to measure largely anecdotal online conversation and people packed in "labs" and other unnatural environments, turning dials as they watch the advertising and react on a moment-by-moment basis. These results will be fun to review, in some cases inspiring and in most cases largely useless in getting a real take on the efficacy of the ads and the campaigns that spawned them.

In this sound byte society that we live in, many will embrace flawed methodologies to proclaim victory. But unlike a single football game or the ads within it, victory can't be attributable to a singular 30-second increment. Rather, a championship is the result of a carefully planned, multi-faceted and consistently analyzed campaign. Effective and regular brand perception tracking within a sponsor-blind and natural environment can help sports marketers and property holders know the real score.

This Just In: Top Major League Sports
1 National Football League
2 Major League Baseball
3 National Basketball Association
4 National Hockey League
Source: Brand Keys Customer Loyalty Engagement Index 2011