• by Joe Mandese  @mp_joemandese, November 19, 2012

On Oct. 29th, when Superstorm Sandy knocked more than 8 million households off the grid in some of the nation's largest media markets, it created unprecedented challenges for the national and local TV marketplaces, and especially for the company that measures it. While Nielsen has disclosed some of the impact on local ratings in some of the most severely affected markets -- New York, Philadelphia, Boston, Baltimore and Akron -- it has disclosed nothing to date about how it has been dealing with the impact on national TV ratings. As it turns out, Nielsen has been relying on a process known as mathematical “weighting” that gives some of the weight of those homes that were still reporting data to those that were not.

While weighting is routinely used in the TV ratings business to adjust for the under-representation of some portion of Nielsen’s sample -- typically homes that are difficult to recruit or maintain in its panels -- those weights are usually based on specific types of households that Nielsen can compare with other similar households in its sample, not an entire region of the country that simply stopped watching television due to the loss of power, and maybe even their homes. Some experts believe that may have led to some significant overstatement of national TV ratings during one of the most valuable advertising periods of the year -- the first several weeks of November.

Nielsen has scheduled a special webinar Tuesday morning to brief its clients on its storm “recovery” efforts, and is expected to disclose more information about the total impact on its sample, what data has been withheld, and how it has been processing ratings throughout the period.

What is known to date is that the impact was so severe that Nielsen has already informed clients that it would withhold all its survey data for the entire month of November for the New York market, the nation’s largest, and that it was withholding several days of data from four other local people meter markets -- Philadelphia, Boston, Baltimore and Akron (see below) -- which combined represent more than 13% of Nielsen’s national TV ratings sample.

A Nielsen spokesperson asserted that the company has not excluded data for the entire month of November for the New York market, because some clients would still receive daily data “as of Nov. 9 forward.” She said Nielsen is only withholding the New York market’s “book,” which is an average of ratings for the entire month for the market.

In any case, Nielsen has had some significant portions of its local and national samples affected -- including households residing outside the five people meter markets disclosed to date -- and has been utilizing mathematical weighting to produce national audience estimates.

“If Nielsen has been using the weighting method, it has been overstating, or inflating, ratings, because it assumes those people had access to television, but they did not. They’ve been turning non-viewing households into viewing households and that’s wrong. And it’s producing ersatz data,” says a research executive who is knowledgeable about Nielsen’s methods.

The executive said it’s especially disconcerting that Nielsen has been doing this without disclosing it publicly during a period when some of the most valuable national advertising buys are running, and when it has been reporting audience estimates for television programming that is vital to national interests, such as election night coverage.

During tomorrow’s webinar, Nielsen will answer some of these questions, and will explain some of the methods it has been using as a stopgap measure to deal with the unprecedented impact of the storm, including how it will replace some households in the sample that have been permanently lost.

While that’s not unprecedented -- Nielsen’s spokesperson noted that the same thing happened after Hurricane Katrina destroyed many of its sample households in New Orleans -- it most likely has never impacted the national sample the way Sandy has over the past several weeks.