Commentary

Nielsen's Plan To Improve TV Measurement: Progress, Not Perfection

Last week Nielsen announced key rollout dates for its previously stated measurement enhancements, most of which had been announced last year.   Nielsen plans to utilize several new statistical methodologies across both local and national measurement services.  Since the announcement, Nielsen has received quite a lot of backlash (as happens with most big changes), but we at MAGNA support the initiatives as an important step in moving television measurement forward. 

Nielsen has begun adding what will ultimately about 2,000 new electronic meters to the National panel, using the same technology and methodology already in place. Additionally, over the course of this year, 15 LPM markets are expected to be expanded by 200-300 meters each, effectively adding over 3,000 meters to the National sample by the time the expansion is completed.  Nielsen also plans to leverage the 31 set meter markets, which currently only produce household-level data electronically (with demographics coming from diaries). 

advertisement

advertisement

The most controversial part of the plan is the way in which Nielsen will assign demographics to both the aforementioned set meter markets and to future markets that will be measured by a new device called a Code Reader.  This new technique is being named the “Viewer Assignment Methodology,” which uses statistical modeling to determine who in the household is viewing the program, by using National People Meter homes in a defined territory area, in the same time zone.  This allows for the attribution of demographic viewing in homes where only household-level tuning was recorded.

With these changes, Nielsen will be able to remove archaic paper diaries from local measurement in a total of 45 markets—the 31 set meter markets, and an additional 14 markets that previously relied on diaries only.  In the 14 diary-only markets, Nielsen will utilize its new metering device known as the “Code Reader”— a small, easy-to-install device that relies on audio codes and signatures to match viewing.  The Code Reader only measures household level tuning, so demographic estimates will come from the Viewing Assignment Methodology. 

Judging by the quotes we’ve seen in the press, some of our colleagues are concerned with using modeling for demographic estimates. Here is why we support it:

We believe this is the fastest and most cost effective way to add to the existing sample and help stabilize ratings, without passing on major expenses to clients.  Unreliable diary measurement is being phased out. Even in homes where only household-level tuning is captured, Nielsen knows the demographic makeup of the residents.People Meter homes, in which persons viewing is directly, electronically captured, will serve as the “source of truth.”

Also, Developing an audience modeling approach now will help Nielsen if/when they decide to pursue using set top box data in the future (and we feel a hybrid approach is where they will ultimately go). Demographic viewing always has to be modeled when using return path data. And, Nielsen has demonstrated a commitment to working with both clients and the MRC to ensure their modeling process is consistent and transparent.

To paraphrase Voltaire, let’s not let “perfect” be the enemy of the good.

18 comments about " Nielsen's Plan To Improve TV Measurement: Progress, Not Perfection".
Check to receive email when comments are posted.
  1. Alan Westendal from West End Communications/Consul, February 26, 2015 at 8:33 a.m.

    Based solely on what you wrote it looks like Nielsen is taking the detailed information about a population sample which information was obtained by one methodology, and applying it to another, similar but larger population where the more detailed information is not available. That, of course, is the rationale behind extrapolating something –a behavior pattern, say -- observed in a sample of a population onto the whole population from which the sample was drawn. Call it modeling or whatever, it's not a new technique and not much of a trick.

    Extrapolation like this is much cheaper than finding a way to collect more data from an enlarged sample. It also diminishes (relative to a method that actually uses a bigger sample) the likelihood that the system will detect the early stages of subtle changes in behavior, and actually makes the resulting findings less valuable, since they give the illusion of being more accurate, when in fact by glossing over small but possibly important nascent changes In behavior they are less accurate.

  2. Ed Papazian from Media Dynamics Inc, February 26, 2015 at 8:47 a.m.

    Alan is correct about the diminished ability of such systems to detect actual changes in viewer preferences between platforms and individual shows, until these changes are of sufficient magnitude that their effects are all too obvious. In, effect, you are getting a built in system which leans heavily on consistency of data but does not necessarily give you the ability to zero in on trends that are just starting to develop---and might be predictive of future behavior. Only a much larger sample, using only the actual findings from its "respondents", can give you that degree of sensitivity.

  3. Ed Papazian from Media Dynamics Inc, February 26, 2015 at 9:27 a.m.

    In fairness to Nielsen, I should point out that as the stations are unwilling ----or unable---to foot the bill for dramatically expanded sample sizes in their markets, it is understandable that Nielsen attempts some solution to the problems caused by "unstable" ratings. Since the buyers insist on ratings for every channel and TV show, no matter how small the audience levels are, this puts Nielsen in a bind. Either say "no"---due to the lack of financial support for larger samples---or "yes" and try something like this modelling scheme. I should also add that the rules for spot and national TV are quite different, with the agencies being much more careful and concerned about data accuracy when "network TV" is involved. I hope that the agencies will give spot a more even break in this regard ----but I'm not holding my breath waiting for that to happen.

  4. Alan Westendal from West End Communications/Consul, February 26, 2015 at 9:37 a.m.

    I wasn't trying to be unfair to Nielsen. If they want to posit that their new offering is an important enhancement, and customers go along with that, that's fine.

  5. Joshua Chasin from comScore, February 26, 2015 at 10:48 a.m.

    Keep in mind that this system is replacing a different system whereby one sample is used to collect one set of data via by one methodology, which is then applied to a different sample from which a different set of data is collected via a different methodology. This is a replacement of paper diaries and meter/diary integration, a technique which had it's heyday in the '60s. It is not a retrenchment from people meters. Paper diaries are notoriously slow at detecting subtle changes in viewer preferences (especially given that in most markets they are only placed four months a year, and I think even in the largest diary markets, only seven months.)

  6. Ed Papazian from Media Dynamics Inc, February 26, 2015 at 10:54 a.m.

    Alan, I wasn't saying that you were being unfair to Nielsen. I'm merely pointing out that Nielsen is between a rock and a hard place in this matter, since a truly huge increase in sample size is needed to obtain stable ratings for all parties---big and small----in local markets, but nobody is stepping up to pay the added cost. As a result, Nielsen's clients---mainly the stations in financial terms---have no choice but to accept this type of "solution", no matter what they think of it, privately. I doubt that many are really OK with it, however.

  7. Nicholas Schiavone from Nicholas P. Schiavone, LLC, February 26, 2015 at 5:24 p.m.

    Dear Stefanie,
    I appreciate learning your perspective.
    It is my intention to address as many of your mistaken (and correct) judgments in due course over the days ahead. The topic is that important!
    However, before quoting Voltaire, you must acknowledge that every thoughtful critic (like Ed Papazian) sought quality not "perfection." The perfection defense is a serious abdication of the researcher's fiduciary responsibility.
    Further, with all due respect to MAGNA, what MAGNA pays for the Nielsen so-called" service is a tiny fraction of major broadcasters and cablecasters pay for their deliverables. If ratings were your currency & life-blood you would take a less "academic" perspective. Since when does putting the word "statistical" before the word "methodologies" endow them with a carte blanche defense? Extrapolation is a "sin" when it comes to tabulating ratings currency. Well the cabin has closed and I must thankfully turn off my mobile device. There is much more to be discussed. In the meantime, you should seriously consider making certain that MAGNA plays an integral, professional role in the E&Y Audit & the MRC Accreditation process.
    Onwards & upwards!
    Sincerely,
    Nicholas P. Schiavone
    Nicholas P. Schiavone, LLC

  8. Alan Westendal from West End Communications/Consul, February 26, 2015 at 6:07 p.m.

    Well, Mr. Papazian, I take your point. If all parties are OK with the largely cosmetic enhancement to Nielsen's data product, that's fine. I expect that Nielsen's customers, who expect to pay very little extra will, indeed, get very little extra.

    Perhaps instead of sourcing Voltaire for the perfect vs good adage, Ms Morales might have found whoever first said: "There ain't no such thing as a free lunch."

  9. Nicholas Schiavone from Nicholas P. Schiavone, LLC, February 26, 2015 at 6:42 p.m.

    Ms. Morales should also acquaint herself with the concept of "Lies, Damned Lies and (certain Nielsen) Statistics" which are based on Nielsen's possession of certain delusional powers...like the ability to turn a set-meter home into a people-meter home without installing a people meter in the panel household undergoing magical transformation.
    By the way, Stefanie, I've yet to read any of the backlash to which you refer. Once again, critics of Nielsen's ill-conceived "sham" plan are seeking quality, not perfection.
    It would also be helpful if you shared a CV that made clear the experience & expertise you are speaking from with respect to quantitative methods & statistics. In the meantime, set aside Voltaire and read what Dr. Ed Deming had to say about research quality and the proper use of statistical methods.
    Onwards & upwards.
    Sincerely,
    N. P. Schiavone

  10. Nicholas Schiavone from Nicholas P. Schiavone, LLC, February 26, 2015 at 7:35 p.m.

    I trust MediaPost abides by an equal time Amendment that would permit as full-throated a critique of Nielsen Nonsense, as it does a full-throated ecomium of Nielsen's Short-cut To National Sample Expansion. Once again, if MAGNA paid Media Company rates, as opposed to the rate charged to friends of the Nielsen Family (i.e., advertising agencies), one might hear a different tune coming from the other side of the aisle. I now wonder whether Stefanie & Company didn't have a coach for her tribute to "progress." Let's be clear: Nielsen is devolving, not evolving.
    Onwards & upwards.
    NPS

  11. Nicholas Schiavone from Nicholas P. Schiavone, LLC, February 26, 2015 at 10:18 p.m.

    I fully agree with Alan Westendal's commentary. As I am too tired to take Nielsen & MAGNA to task tonight for pie-eyed optimism or just an error of judgment, I shall instead provide limited background on Alan's apt quote reference:
    INVESTOPEDIA EXPLAINS:
    'THERE AIN'T NO SUCH THING AS A FREE LUNCH - TANSTAAFL'
    "This phrase is thought to have originated because many saloons in the U.S. used to provide free lunches to their patrons,
    but required them to purchase drinks in order to get them.
    Although the phrase is a double negative,
    it is not intended to be interpreted as such.
    Therefore, the alternate acronym TINSTAAFL is often used."
    Thank you, Alan, for that ray of light.
    Hope must wait, I'm afraid.
    NPS

  12. Stephen McClellan from MediaPost, February 27, 2015 at 9:17 a.m.

    Nick, MediaPost would be more than happy to offer you the same platform as Stefanie for an equal time post. 800 words give or take, a thousand words max. Think you're off to a good start already with your eloquent comments but you're certainly entitled to the "main stage" for a broader take if you'd like. Send to mcclellan@mediapost.com and thanks for your interest in doing so.

  13. Timothy Daly from ITN networks, February 27, 2015 at 3:18 p.m.

    Quality is important, and I think Stephanie is saying that the diaries are extremely low quality and need to be retired. The MRC took away diary accreditation almost 5 years ago. My take is that the Nielsen plan for combining people information and HH meter information from two different panels , with the National People Meter (NPM) panel replacing the diary panel, is an increase in quality on the local NSI side. Not only is the NPM better than the diary in quality, the proposed method for combining the two data sets is more logical than how the meter and diary data have been combined in the past. We have yet to see any real impact data yet, so there is much to learn. Thanks to Stephanie for weighing in; and Nick, yes, "more people meters" is the path to quality, but please agree that continuing diaries is the path to low quality.

  14. Jon Currie from Currie Communications, Inc., February 27, 2015 at 3:19 p.m.

    As Jimmy Fallon might say on Fridays, "Thank you Nielsen, for bringing new life to Currie Communications' ratings practice. Thank you Nielsen, for getting me that long needed Hawaiian vacation."

  15. Nicholas Schiavone from Nicholas P. Schiavone, LLC, February 28, 2015 at 3:54 p.m.

    Open message to Tim Daly.
    I found the comments Tim addressed to me (Nick) to be thoughtful & respectful. I have known Tim for some time and view him as a person & professional of integrity, caring & practical wisdom. Because of my intense business travel schedule, I am unable to formulate the extended response now that he and the topic deserve. No doubt the diary is finished as far as certain forms of measurement are concerned. My concern, at core, is only that we don't substitute a hammer for a screwdriver, if that's what it takes to do the right things and to do things right (Peter Drucker). Moreover, I completely reject Nielsen's Magic Wand solution to Panel Household Transformations & National Sample Expansion. Interpolation is good. Extrapolation is bad. Nielsen must not extrapolate currency measures.
    Onwards & upwards.
    Sincerely,
    Nick
    Nicholas P. Schiavone, LLC

  16. Ed Papazian from Media Dynamics Inc, March 1, 2015 at 9:42 a.m.

    I await Nick's comments with interest. As for the issue of diaries being an inferior measurement instrument, that's been clear for a long time----especially for low rated channels and certain forms of program content---movies, for example. In addition to the issue of mixing methodologies and statistically estimating viewer data instead of measuring it, I state, again, that no matter how this plays out, sample size is, and will continue to be, a major problem for local TV ratings as nobody seems willing to pay for the much, much, much larger samples that are required.

  17. Nicholas Schiavone from Nicholas P. Schiavone, LLC, March 4, 2015 at 5:18 p.m.

    Home by another way...

    Dear John Grono & Tony Jarvis,

    I, too, am saddened by the studied ignorance and collective misunderstanding that this conversational tributary displays.

    My knowledge and understanding of "JIC's" and what the
    "industry deserves" transcend your assumptions and expectations.

    Because my learning and experience are a matter of Public Record and not PR, allow me to set the record straight for those who may not know the modern history of the US TV Research Business.

    As Chairman of CONTAM from 1989 to 1999, I oversaw the last industry-wide effort to develop a JIC for the United States and Canada. The project was called JICTAM and it was intended to synergistically merge the best elements of a JIC and CONTAM. [CONTAM was a Congressionally-mandated methodological research body launched in the early 60's by the three major broadcast networks in existence at the time.] It was my responsibility to invest substantial NBC/GE Capital Funds in the development of a JIC and to lead and to manage the primary research efforts of CONTAM.

    Simply stated, CONTAM had a two-fold mission: to understand and to improve TV audience measurement. (It is the ultimate irony that so many of Nielsen's so-called innovative technical advancements proposed in the past 5 years were so well modeled and demonstrated by CONTAM in the 1990's using the SMART Laboratory of SRI.) It was also my job as CONTAM Chairman to exercise fiduciary responsibility with tripartite legal guidance on behalf of ABC, CBS, Fox and NBC to make certain that their major financial investments in methodological research and JIC development were consistent with US Law and Federal Regulatory Statutes.

    As the distinguished Management Consultant Peter Drucker would say: CONTAM endeavored constantly "to do the right things and do things right." Ultimately, it was also my job in 1999 to bring all these good and necessary efforts to an end because of the faint-heartedness of Agencies, the indifference of Advertisers and the obduracy of Nielsen. But truth to be told, the aggrieved Networks also wimped out.

    (To be continued in next post due to MediaPost space requirements. Thank you.)

    Nicholas P. Schiavone
    Formerly Chief Research Officer of NBC/GE

  18. Nicholas Schiavone from Nicholas P. Schiavone, LLC, March 4, 2015 at 5:21 p.m.

    Home By Another Way (NPS Comments Continued) ...

    In sum, I do not wonder about the benefits if a JIC.

    I've spent time in London being inspired and envious of BARB. I also spent time in Toronto, Canada and Cologne, Germany looking for "a better way" with seemingly more caring and knowledgeable research business professionals.


    In the end, what pained me more than anything was to re-learn the meaning of an aphorism that dates from 1175: "You can lead a horse to water, but you can't make it drink." Yes, people, like horses, will only do what they have a mind to do. "So it goes." (Kurt Vonnegut)

    Alas, I have no regrets because I tried, smart and strong. Now, it is someone else's responsibility - or perhaps no one's? For the past 15 years, all I have heard is talk and all I have seen is nonsense played out on a global scale.

    But if you all succeed in bringing a good JIC and a better measurement to bear in the USA, I shall publically salute you and send you each a bottle of the finest French Champagne (authenticity counts) in celebration of true collective wisdom and real working knowledge.

    Thank you very much for your time and attention.

    Sincerely,
    Nicholas

    Nicholas P. Schiavone, LLC
    Former Chief Research Officer of NBC/GE

Next story loading loading..