• by September 27, 2007

I have a confession to make. From the 4^th of July to Labor Day, I don't read trade journals and I don't surf the media industry Web sites. Don't tell anyone. I take my sons to baseball games and watch the U.S. Open Series on ESPN. My wife and I Netflix independent movies and stay up to all hours arguing whether a melancholic Bill Murray would have been a better Bernard than Jeff Daniels in "The Squid and the Whale." We take the family to the shore. It's summer, that's what we do.

This year, as has been the case for the last several years, my laptop did not join us on holiday. How does that work out, you ask? Well, aside from the fact that we have no cable and are saddled with a single 13-inch television in the house where we stay: not so bad. We get three channels and lots of static. When it rains, (and it rained a lot this year) we play cards and board games with the kids. But I do suffer withdrawal pains. On the last day, while my lovely bride took the kids to a museum, I got lunch at the Internet café in town. While eating an enormous hamburger -- or "salad," as I later recalled to my wife -- I got my fix of news, the Devil Rays and preseason football. It wasn't long before I took a peek at my email. "Just one," I told myself. In a note from a friend, I found a quote from Arbitron's chief researcher, Bob Patchen:

[Arbitron] has begun moving more quickly to remove "poor responders" from its panels -- but it's taking longer than expected to replace those households with more cooperative panelists.

Now, the whole point of a vacation is to relax and unwind, reason one why I don't bring my laptop with me anymore. Quotes like these just spin me up. So I calmly picked up my newspaper and turned my back on the lifeline to Madison Avenue. "That can wait until tomorrow," I told myself.

Well, it's tomorrow.

Moving quickly to remove poor responders? Replace those households with more cooperative panelists? Whatever happened to the random in random sampling? Raise your hand if you think you would have gotten away with suggesting such a solution in your Stats class back in college. We all know small samples are problematic, but at some point you have to draw a line in the sand. The theory behind your business is the sample, Arbitron. You can't just toss somebody out because they don't want to wear your silly pager. I am constantly amazed by the arrogance displayed by both Arbitron and Nielsen. Have they no shame? Do they think they can play fast and loose with sampling rules? What happened to technology that works? The PPM design is 15+ years old. Anyone seen a cell phone from 1992? Does anyone want to trade me their Apple iPhone for a mint Motorola bag phone I've got in the garage?

There's more:

Arbitron plans to improve its overall sample performance, Patchen said, with a range of measures, including adding recruiters and support staff, using a higher installed sample in "problem demos" and during the summer, and raising incentives for hard-to-recruit young and young black households.

Let's see. It sounds to me like they are going to ignore those in the sample that make life difficult for them. Next, will they call the panelists morning, noon and night? I am sure that will work. I know my compliance needle pegs when my wife nags me about taking out the garbage. What effect does increasing incentives have on the research? Does anyone at Arbitron have an opinion on that? Does anyone at Arbitron care? When is enough enough? I can hear the uproar now: "You have no idea how much work goes into what Arbitron does," or "There are a number of good people doing really good work there," and my favorite: "It is the only data we have!"

It boils down to this. If there is a shared characteristic within the "poor responders" group and that characteristic drives media consumption, then the influence of that characteristic is not measured. Still not concerned? Let's apply the concept. Suppose most teenagers feel that carrying the PPM is "geeky" and decide (en masse) to become "poor responders". If you believe Bob Patchen, Arbitron would replace those randomly selected teenagers with others who don't care if they are "geeky." All is well. chirps Arbitron. Not a problem, you say? Fine. You listen to Ryan Seacrest, I'll hit Pandora and iTunes, thank you very much. Now you know why radio is dying.

We all agree that research can be difficult, but we in the industry have allowed Arbitron to become what it is. When is Arbitron going to deliver on the faith the industry has placed in it? There's nothing wrong with not being perfect. The PPM technology is 20 years old, but it is substantially better than diaries. We should demand better meters, larger samples and integrated reporting.

Or we could make it easy on everyone and institute some competition. I can hear the catcalls now ... "There's only one currency!" Well, the last time I bought a house, there were three credit reporting agencies, so that argument doesn't fly with me. And don't think for a minute that we don't have the same problem in television. Last time I checked, Ryan Seacrest was everywhere on television, too. It is our mess, let's clean it up already. Summer's over. Time to get back to work.