The Coalition for Innovative Media Measurement (CIMM), a division of the Advertising Research Foundation, this morning said it is launching new research to "better understand how time is spent across every platform available on TVs today."
The initiative, dubbed the "Passive TV Measurement Study," will explicitly look at how American consumers spend time with linear TV as well as OTT (over-the-top internet-connected) devices, smart TV apps and video game consoles.
CIMM said the study will be conducted in partnership with HyphaMetrics, and will utilize its "multi-layered methodology," which includes active and passive measurement in combination with machine learning in order to understand consumer media behavior when they are present in a room.
"The methodology can be used to assign person-level demographics to machine-level TV exposure datasets, such as those from smart TV and Set-Top-Box (STB) Data," CIMM said in a statement, adding: "While many in the TV industry are moving to the use of scaled granular TV datasets, calibration panels such as those utilized by HyphaMetrics can enable adjustment for data missing in these datasets."
The pilot test, which will be conducted over the next few months, will utilize a panel of 100 households.
“The industry has long sought to know, in real-time, who is watching what and in what format are they watching,” CIMM CEO and Managing Director Jane Clarke noted, adding: “With HyphaMetrics, we hope to establish the validity between the meter-detected person presence and the ‘in-the-moment’ source of truth from the phone survey to identify what TVs were on, what was being watched and which household members were in the room with TVs on.”
Not sure exactly what they indend to do but this could be a very interesting project if it is done in a normal exposure setting for each medium---not a lab situation---and especially, if a meaningful indicator of attentiveness is included.
A 'phone survey becomes the 'source of truth'? Really?
One big problem with doing a study where a sample of people has its sets metered and is called by phone at the same time---I assume---to ask who was watching and what they were watching---concerns defining the nature of the content in a way that gets at the truth.
This is especially so when commercials are on the screen. For example, say my TV set is on and a commercial break is in progress. Suddenly the phone rings and I pick it up. I am asked if I was just watching TV and, if so, what was watched? Do I say I was watching the commercials---even if true--and often it isn't---or do I assume that the researcher meant the program and say yes, I watched whatever show I had just been giving my attention to---perhaps three or four minutes ago. Or the researcher, cognizant of exactly what was on the screen, might ask if I was watching the commercials. If so, will I give an honest answer or will I try to impress the caller with my coolness by saying no---even if I did look at a few of the commercials?
I agree with John that the telephone is no longer the gold standard for this kind of research. Why not use all three methods---a meter for set usage, TVision's "eye cameras" for visual attentiveness and a coincidental phone survey and see what you get?
Coincidental calls (affectionally called "coins" by interviewers) have a very focused objective - validating "who is watching at the moent the phone rang". Calls are completed within 2-3 minutes to identify which TVs were on and who is watching them in the moment. The data are then matched by timestamp to meter data - creating a 2 x 2 table: Meter data - Person1 watching Yes / No by Coin call data - Person1 watching Yes / No. The greater the matches to Yes / Yes and No / No quadrents the higher the validation or our approach.
Why choose this over other methods? Answer: to maximize acceptance of the results by the CIMM / industry. The elegance of using the "tried and true" telephone coincidental is for the very reason that it is so well known. Thus, following standard protocols allows for comparision to other audience measurement approaches.
Ed - appreciate your POV and insights on all things measurement. Per the comment above for John, the use case for this conincidental is focused on validating who is watching which TV. Because our meters have the ability to definitively capture a broad range of sources of video hitting the glass, we are asking how they are using the TV (e.g., watching from the STB, streaming device, game console, etc.). Thus we're not relying on a self-report of granular activity (such as what program or commercial was on = hard to answer and ever changing), instead it is an easier to answer source of video hitting the glass.
Thanks, Chuck. But I still have a question. When you call and someone answers who claims to have just been watching TV, how exactly is your question phrased. If the call happens during a commercial break do you ask whether the respondent was watching the commercials? Or do you simply ask if the respondent was "watching" the program just before the call, without referrence to whether it was program content or ads?This is the point I was making. What does the respondent think you are referring to when you pose the question---the program---very likely---or the ads---not so likely? I am also assuming that if the respondent is asked about the commercials this may produce a biased answer---probably one that understates the case.
One way to get at this if you are wedded to the coincidental phone call method---which is fine with me---is to ask the respondent who claims m to be a very recent viewer---like a few minutes ago---what he/she was doing just then. For example had the viewer left the room---about 15% do this per minute per the camera studies. Or whether the viewer was fully attentive to the content or engaged in other activities and if so what activities, etc.
Coincidentals are (or should be) mandatory as a validation procedure that delivers compliance performance of each participating household and household member.
In AU performing coincidentals has been a mandatory requirement since metering started in the early '90s with the compliance performance being part of the regular audit. They are done as 'rolling' coincidentals (you can't call 8,000+ homes every day) to not only monitor compliance but to also 'coach' the household members. Daily usage analysis is also conducted to help identify irregular viewing patterns which could initiate a call to the household to verify usage, household composition (e.g. we notice that the 18 year old male hasn't watched TV for a fortnight - has he gone on a holiday, has he gone to college and moved out of the family household etc.)
So does CIMM's 'initiative' indicate that Nielsen in the US is not conducting coincidentals that are independently audited?
John, I'm not aware that a system such as you describe is in place here---if I'm wrong I hope that someone at Nielsen will clarify.
Thanks Ed. I hope that someone can provide that information - Nielsen, CIMM, networks ... someone!