• by Mark Green , Op-Ed Contributor, December 1, 2005

Measuring media can be a nasty business. There is always understandable pressure to make measurements simple and inexpensive. While good in theory, this pressure can quickly lead to bad decisions. There are two fundamentals to quality measurement: objective and representative. These fundamentals are often stretched in the struggle to evolve media analyses beyond media exposures to understanding engagement. Today's media measurement conundrums are "what to measure" and "how to measure it." Academia posits that effective management requires key performance indicators (KPIs).

The KPI measurement that is seizing the imagination of today's marketers is media engagement. Integration's Market ContactAudit has gained popularity for its strong viewpoint: that asking consumers what media influences them is the way to capture this information. This challenges the traditional thinking of the Advertising Research Foundation (ARF) in its Media Model. Instead of asking consumers simple questions like "what influences you," the ARF Media Model uses clinical gradations with many KPIs: exposure (those exposed to media), attentiveness (those who recall a brand name), communication (those who recall a brand message), and persuasion (those who shift their brand intentions).

Asking consumers simple questions is more reasonable. By contrast, the ARF Media Model is complicated and expensive to implement. But before rushing to judgment, we need to pause and reflect upon which measures are truly objective and representative. As a rule, simpler is better. It makes sense that "what to measure" ought to be media effectiveness -- essentially, how well your campaign persuaded your target audience.

This cut to the chase is what Market ContactAudit advocates. Others prefer analyzing media engagement and message effectiveness separately, since these two areas are traditionally organized into business silos: one shop for media planning, one for creative messaging. Crispin and its ilk are battering the conglomerate fortresses, arguing that the world of business silos is dead in marketing communications. The strength of the MCA/Crispin argument is hard to ignore.

Yet real world business systems require methods and processes to be effectively managed. This brings us to the hard question of "how to measure it," while being objective and representative.

The debates over being objective weigh "recalled behavior" or "claimed viewpoints" against observations of behavior. Is what someone tells you more important than what you observe that person do? Television diaries are recalled behavior, while passive electronic meters of television tuning are observations of behavior.

The debates around being representative weigh Internet polling and mall intercepts against random samples of U.S. populations. At best, Internet polls randomly select from unknown slices of active Internet users, and mall intercepts randomly select from unknown slices of mallgoers. These studies are essentially large focus groups. They fish for information but need follow-up research with real random samples to calibrate how representative the findings really are.

Upon reflection, the ARF Media Model starts to make sense. If you ask people apparently independent questions that are in reality interrelated, you can extract real behavior from what they think and remember. This is done by asking the Media Model questions at different times or places in the questionnaire, then aligning answers to edit out conflicting statements.

With this in-depth approach, questionnaires are the way to go, and yes, the Internet is probably the best way to administrate them. But to make it representative, you would also need to solicit by phone (using random digit dialing), by mail (selecting random households from Census-stratified areas across the U.S.), or in-person (using the same selection technique).

It may be more expensive, but the results are objective and representative. If you build this as an ongoing omnibus study for sponsoring brands and businesses, the resulting KPIs could become part of a system to effectively manage communication investments

Mark Green is senior vice president, media services, VNU Global Modeling & Analytics, and the founding partner of the Media Learning Institute. (mark128green@aol.com) He is a regular contributor to MEDIA magazine. This column is republished from the November issue.