Mainframe Move Crashes Nielsen Ratings System: Million Dollar Snafu Delays All Data

by Joe Mandese , January 8, 2009

TV ratings giant Nielsen is getting 2009 off on an awkward foot, disclosing a major technical snafu to clients Wednesday that has caused at least several days of delays for virtually all of its major TV ratings services. The glitch, which was caused by the relocation of the mainframe computing system used to process all its national and local TV ratings, occurred despite a $1 million premium Nielsen paid to the technology provider hired to handle the transition.

Details of the gaffe were outlined in a letter sent by Nielsen Executive Vice President-Global Business Services Mitchell Habib to clients, as well as at least half a dozen separate communiqués from individual Nielsen service units on Wednesday.

Habib described the event as a "significant system disruption," but added that Nielsen technicians have been working aggressively to fix the problem and that they believe it had been resolved as of Wednesday morning.

The problem, which occurred when Nielsen relocated its mainframe computing system to a new facility in Plano, Texas from its an older one in Oldsmar, FL, triggered a "series of technical issues" that made it impossible for Nielsen to retrieve data necessary to process its ratings.

Ratings affected by the glitch included:

* Local Overnight Data
* Fast Affiliates Data
* Fast Nationals
* National Broadcast and Cable Overnight and Weekly Data
* Live+7 Data

Habib said Nielsen planned extensively for the transition and hired an "industry-leading technology partner" that was paid a "premium of $1 million over the standard fee to have them focus additional resources on this project to ensure a high quality and timely transition. We took these extra steps in order to avoid any impact on clients."

A Nielsen spokesman said that while the core problems have been resolved, it make take until Jan. 12th to process the remaining delayed data.

"I've never seen a day like this," said one Nielsen client after receiving his ninth in a series of related ratings delay notices.

The client, who requested anonymity, said his main concern was that Nielsen had not advised clients of the relocation - and the potential risk of dislocation that might occur from it - beforehand, so that they could have been prepared for a snafu should it occur.

"It's fine for Nielsen to invoke the term of 'continuous improvement' in these matters, but to its clients, it looks more like continuous chaos," he said.

A Nielsen spokesman said it did not issue a broad client communication about the mainframe system relocation because it believed it had taken the steps necessary to ensure it would be "seamless to client."

research, tv

2 comments about "Mainframe Move Crashes Nielsen Ratings System: Million Dollar Snafu Delays All Data".

Check to receive email when comments are posted.

Rex Taylor, January 8, 2009 at 11:05 p.m.
How does it happen? Ask the NBC affil in Okla. City. They had 2+ years of digital numbers dumped into a catch-all bucket and not credited to them. So when the error was finally corrected a #3 went to #1 in several timeslots.
It's a computer controlled by humans...thus errors and very costly errors will occur.
Reply

John Doe, January 9, 2009 at 12:12 p.m.

Mitchell Habib, the all powerful overlord of The Nielsen Company, has cut out all of the old time American datacenter staff and replaced them with novice Indian immigrants. Regarding the mainframe, Tata has no mainframe skilled staff and now all old Nielsen mainframe staff are gone. By deciding to move to a new mainframe somewhere else a HUGE risk was taken. Now we have inexperienced Indians leading a third party vendor to a miserable failure. Funny that one million dollars was paid to ensure it went smoothly, but we probably saved much less than that by getting rid of our old American staff experts. Where's the big savings for the outsourcing now?