• by Erik Sass , June 21, 2006

The need for new audience metrics and a palpable dissatisfaction with existing methods dominated the first day of and important gathering of top advertising and media researchers in New York this week. Overall, the execs and pundits leading Tuesday's discussions at the Advertising Research Foundation's Audience Measurement Symposium painted a picture of an industry pulled in different directions and lacking a consensus on some important measurement standards.

One outspoken panelist, Jack Wakshlag, chief research officer for Turner Broadcasting System, for example, decried a lack of syndicated data for mobile video viewing. That, he said, is leaving Turner to its own resources.

"CNN Wireless is available to 140 million wireless data devices, but how many people are actually viewing it? We can only tell from our server lines--that's the best kind of data we've got. That's my world," he said, adding, "We do the best we can to get information from syndicated services, but we're doing a lot more than that."

Similarly, Tom Evans, senior vice president of research for ABC Radio, responded to a question from an Arbitron employee about developing more refined measurement methods by saying: "That's your job!" As it stands now, Evans said as an example, ABC programming carried on satellite radio is essentially going unmeasured--meaning that ABC sales reps "can't go in and say: 'On terrestrial radio I'm going to give you this reach and frequency, and I can take satellite radio and roll that in'--that's not possible now, because the numbers just aren't there."

TBS' Wakshlag also assailed measurement systems using self-reported data from individual consumers, noting a high degree of inaccuracy both in raw numbers and self-attributed causal explanations: "While people are very good at explaining their actions, those explanations aren't necessarily true, and we fool ourselves when we say they are."

Here, Wakshlag cited a recent study from Ball State in which observers actually followed subjects around, noting: "You get a sense of the complexity of the world they live in, and the impossibility of asking them to summarize their behavior." Displaying a graphic of one person's actual observed media behavior, Wakshlag recalled that the subject's own estimates deviated by 62 percent. "For example, she made 40 phone calls in one day--and you're going to ask her how much time she spent on the phone while using the computer, or while watching television, and what influence that had on her behavior?"

However the main clash came in a lunchtime debate between Joe Plummer, the ARF's chief research officer, and Erwin Ephron, a noted media pundit, over "engagement." Plummer began by defending "engagement"--and the ARF--against charges of incoherence and obscurity from the "trade press": "It seems we haven't done a great job explaining that there is a definition, and a method of measurement" presented at an ARF conference on the subject in March.