• by Joe Mandese  @mp_joemandese, May 21, 2009

In an apparent effort to appease client anxiety sparked by controversial research it released last month showing flaws in its national people meter ratings system currently underestimate television viewing by about 8%, Nielsen Wednesday issued a revised interpretation saying that actually is not the case, even though two senior Nielsen executives went out of their way to highlight the "underreporting" in an interview with MediaDailyNews. In fact, they said the finding was consistent with a series of "validation" studies Nielsen has been conducting periodically ever since it introduced the people meter ratings system in 1987.

"A significant question was raised by clients, resulting from that communication, which we would like to follow up on here. Specifically, we were asked whether the .92 viewing index (8% fewer viewers based on the people meter compared to the coincidental) were equivalent to an 8% underreporting of ratings," Nielsen told clients Wednesday, adding, "The answer is no."

That's a very different view than the one taken by the two top executives - Pat McDonough, senior vice president of Planning Policy & Analysis; and Bruce Hoynoski, Chief Research Officer - when they originally briefed MDN on April 29th. The significance of that finding was reconfirmed in another off-the-record meeting with another senior Nielsen executive last week, but an executive familiar with the controversy sparked by the research said Nielsen has issued the revised perspective to allay the concerns of some top network TV clients.

"Since Nielsen released its first findings (4/29) from the button-pushing study, 'scared' clients asked/begged Nielsen to 'clarify' the 'inflammatory' statements that upset client management," one knowledgeable executive told MDN.

The controversial research came from the first release of what Nielsen said is an ongoing series of studies it is conducting on its "live" TV ratings panel to better understand, and ideally to improve, flaws in the way people push their people meter buttons. The studies, known as "telephone coincidentals," contact Nielsen households by phone while they are watching TV and ask them what they are watching, and then compare the results with what has been recorded by their people meters. The studies have found both under-reporting and over-reporting by various respondents, but Nielsen executives said the net effect has been that the national people meter sample underestimates its TV viewing by 8% due to improper use of their people meters.

Nielsen's Hoynoski previously said that percentage has been consistent with similar studies Nielsen conducted in the late 1980s and during the 1990s, and that the new, ongoing research is being conducted to identify the most egregious situations, and to develop ways to improve the accuracy of their reporting. The most recent research showed the worst offenders are Hispanic households and households where multiple viewers were present. In fact, the research showed that the more people that are present in a room while watching TV, the greater the level of under-reporting.

While no two audience research methods will yield exactly the same findings, Nielsen executives originally said the telephone studies were considered a valid, objective source for calibrating the accuracy of the people meters.

But on Wednesday, they issued a new, 786-word advisory claiming the findings are actually "relative," and not "absolute," citing several reasons why the study should not be considered a "quantitative" measure of the people meter's accuracy (see below).

Another reason why Nielsen executives may be back-pedaling on the findings now, is because the company was hit with a federal antitrust suit by one of its clients, Sunbeam Television Corp., on May 1, which included numerous allegations that its people meter system produces inaccurate ratings.

In Wednesday's communiqué, Nielsen now says the under-reporting was "driven by the study design, which did not tabulate certain cases where viewing was potentially overstated. In particular, in order to collect viewing information from a respondent, he or she had to be present in a room where the TV was on. This is so that he or she can most accurately report tuning in the room. This by definition would not include instances where the respondent's button was pushed in another room where he or she was not present. When these instances are tabulated and taken into consideration, they offset much of the under-reported results. This was not done however, as we were focused on viewing in the room the respondent was in when the phone rang."

The communiqué then outlines the following reasons why, "the viewing index should be used as a relative rather than absolute metric:"

*1. As noted above, not all data were tabulated. If the person who answered the phone (the respondent) was not watching television and not in a room with a television set on, we terminated the interview and then tabulated the results only if the People Meter tuning records agreed with the coincidental - that is, the respondent was not logged into any television set in the home. This eliminated situations where the People Meter records disagreed with the respondent's reported tuning status (instances where the People Meter buttons were actually pushed when the respondent reported that he/she was not watching, resulting in an over-estimation of viewing). In fact, there were 35 such cases, which when added to the data analysis raised the overall viewing index to .97 among household members.

* 2. Not all viewing situations in the household were recorded. Only viewing (and non-viewing) situations in the room that the respondent was in when the phone rang were reported in this study. So compliance for all persons in other rooms, whether a single adult or unsupervised children, were excluded.

* 3. Not all calls yielded a complete interview. In this first wave, 68.7% of the calls resulted in a completed interview.

* 4. Nielsen identifies itself at the time of the call. While we continue to study the impact, the fact remains that in many cases this can introduce some response bias, influencing how the respondent answers the questions.

* 5. With the coincidental, we could only be certain of the true viewing status of the respondent and not those in the room with him or her. If, for example, another household member is in the room with the television set on and not paying attention, it is conceivable that they had rightfully not pressed their People Meter button and that the respondent incorrectly reported them as viewing (or vice-versa).

"While the above factors render this data inappropriate for use as an absolute statement of over or under reporting, the true value of this study is to identify which groups need more attention and to track progress we make with our improvement efforts," Nielsen concluded.