Sarandos Says NBC's Netflix Show Data Is 'Remarkably Inaccurate'

PASADENA, CALIF. -- Ted Sarandos, chief content officer of Netflix, says efforts by NBC to measure original Netflix TV shows reveal “remarkably inaccurate data.” 

Speaking at the Television Critics Association meeting on Sunday, Sarandos said NBC's efforts to measure original TV shows on Netflix's subscription video-on-demand service "doesn’t reflect any sense of reality of anything that we keep track of."

Earlier in the week at TCA, NBCUniversal President, Research and Media Development Alan Wurtzel revealed in a presentation addressing 18-to-49s viewing of some Netflix original shows -- by way of third-party technology company Symphony -- that traditional TV still dominates overall viewing. 

Netflix has a long-standing policy of not releasing any data concerning its viewers or viewing of its shows.

advertisement

advertisement

“I don't know why anyone would be spending so much energy and time," Sarandos said. "Given what I believe is really remarkably inaccurate data, I hope no one's paying for it.” He added: “18- to-49-year-old viewing is so insignificant to us that I can't even tell you how many 18-49-year-old members we have. We don't track it.”

Explaining Netflix's current strategy of not revealing data about its viewing, Sarandos said: “I really don't want to get into a position where we are reporting. Because I do think that once we give a number for a show, then every show will be benchmarked off of that show. Even though they were built sometimes for very specific audiences.”

Netflix targets different audiences for shows -- not always looking for the biggest numbers. “We may build a show for 2 million people, and we may build a show for 30 million people.”

Sarandos says Netflix doesn’t want to get into the trap of what linear advertising-supported TV networks need to survive.

“If we turn it into a weekly arms race by doing box scores every live plus three or live plus seven, I think it's going to have the same result that it has on television. Which, I think, has been remarkably negative in terms of the quality of shows.”

Sarandos did offer some guidance. With regard to "Orange is the New Black," he said it has been "our largest show."

6 comments about "Sarandos Says NBC's Netflix Show Data Is 'Remarkably Inaccurate'".
Check to receive email when comments are posted.
  1. Ed Papazian from Media Dynamics Inc, January 18, 2016 at 9:31 a.m.

    What? Eighteen to forty-nine viewing is so "insignificant" to Netflix that they don't even measure it. Astounding!

  2. Leonard Zachary from T___n__, January 18, 2016 at 2:28 p.m.

    Ed you read it the wrong way- Netflix has a ton of 18-49 years old but tracking viewership by demographic( a la Nielsen) is just not their priority, it's the number of paying subscribers Globally. The priority of the anyalytics is to generate new pitch ideas for content- yes thats called looking forward as in Innovation not backwards.

    What linear advertising-supported TV networks needs to survive is innovative distribution to deliver a new experience and real time quality analytics.

  3. John Grono from GAP Research, January 18, 2016 at 6:50 p.m.

    What's the odds that Netflix is talking streams and NBC is talking average minute audience?

    As a recent MediaPost article on January 7th showed, a Yahoo stream of an NFL game garnered 15.2 million visitors (with a typical NFL broadcast getting an audience of 19 million), but when re-calculated based on average minute audience it fell to 1.6 million.

    Here in Australia, unpublished data on streams vs audience is also showing a pretty typical reduction of around a factor on 10 (early days though).

    Moral of the story ... streams <> audience.

  4. Ed Papazian from Media Dynamics, January 19, 2016 at 8:43 a.m.

    John, that raises an interesting question about "streaming" metrics. When a user streams some content, is this counted on a "total audience" basis, irregardless of the amount of time or the length of the video? Or, putting iot differenty, how many streams are cut off or discontinued before the entire video has been "seen"?

  5. John Grono from GAP Research, January 19, 2016 at 5:43 p.m.

    Ed, unfortunately the answer is "it depends".

    When video streaming first started and Internet speeds were much slower, a 'stream' was any segment of video.   So, if it was a 3-minute music clip with a pre-roll and a post-roll, the count woule be three (pre-roll + clip + post-roll).

    Similarly, if it was an episode of a 30-minute programme, it might have been segmented into three parts each with an interstitial ad, so the count would be seven (pre-roll + segment 1 + interstitial 1 + segment 2 + interstital 2 + segment 3 + post-roll).

    When ad-serving became dominant the content server wouldn'y know about the third-part ads served (i.e. no stream identifier known to that server) so things got better.

    In essence a 'stream' generally meant a 'stream start'.

    With faster speeds, better buffering and outsourced ad-serving it (normally) is much closer to what we would call a programme, though some still count a segment as a stream.   One of the biggest issues is identifying the stream content due to use of bitlys (is that its plural?) which can't easily be deciphered beyond the content server (e.g. YouTube).

    But the issue still remains that someone may start to stream a programme (say a nominal 60 minute programme) and stop after three minutes.   In the world of linear TV that would count as .05 of a person.   In the stream-counting world it would count as a full person.   The content server has no knowledge of (i) who is watching - beyond an account name (ii) whether indeed anyone is watching - or is there more than one person watching (iii) do they watch the entire content (iv) is the stream in an active browser and the active tab in that active browser.   This is less of an issue with video than it is with text content when it is common to have multiple tabs open but due to video's sound it isn't common.

    The point is that stream counts DO have a very important role in determining the gross level of traffic to a piece of content.   A sample/panel cannot measure all content, nor can it measure all of the streaming of that content.   But a panel CAN provide important heuristics such as viewing duration, presence at the computer, co-viewing, viewing sessions if non-contiguous viewing, repeat viewing, duplicate viewing across devices.

    I see the future as a combination of both (validated) streaming counts, with (validated) panel heuristics with (validated) processing algorithms to produce robust streamed audience data.   I also see panels from which we get online streamed video and linear TV viewing data and then the ability to provided de-duplicated data.

    I hope this helps.

  6. Ed Papazian from Media Dynamics Inc, January 19, 2016 at 6:11 p.m.

    Thanks, John. It's about as I expected, which means that before we worry about audience metrics across platforms we might come up with a more meaningful and comparable audience metric within platforms. I'm starting to wonder now about my estimates of Netflix user usage---based, in part, on streaming data that Netflix released on a worldwide basis a few years ago. Perhaps, I've overestimated the average number of minutes spent with Netflix content per day by its subscribers. Hmmm?

Next story loading loading..