The analysts are huddled around a computer, pointing at shapes and motions on the screen, which represent the movement and activities of people on the ground in some remote part of the world who are being observed distantly, in real-time and without their knowledge.
One of the technicians fiddles with the system's filters, dialing up the thermal imaging, because people and manmade machines generate heat that can be analyzed as moving shapes and patterns of color that are signals for certain kinds of human behavior.
“Basically, we're looking at the degree of angularity of things like roads and paths. If one of the shapes has very uniform angles, we know it is probably manmade and people are probably using it,” Paul Donato says, gesturing in the air with his index finger as if to depict a solid line. “Basically, we're looking for signatures that tell us something meaningful about what people are doing on the ground.”
Donato is the chief research officer of Nielsen, and he sat down with me on Friday for an exclusive, in-depth conversation about some of the most cutting-edge technologies and data Nielsen has begun working with that could transform the way the world's biggest consumer research organization measures the behavior of consumers. The interview followed months of me pestering Donato for an “on-the-record” interview about things I've heard him say off-the-record during various industry meetings over the past six months -- including some surprising military intelligence-like techniques, such as real-time satellite imaging, monitoring phone records and an ambitious project to utilize artificial intelligence, Big Data and computer processing to develop even more powerful “signatures” of consumer behavior.
The timing of the interview with Friday's public address by President Obama about his plans to reform the controversial ways the National Security Agency has been using similar techniques to gather data on people was purely coincidental. But the President's address and the subsequent reaction by people in the marketing and media industry, as well as public advocates, illustrates how close the worlds of military and marketing intelligence have become.
In fact, following one recent presentation, Donato said he was surprised to be approached by someone in the audience whom he described as an “officer” from the National Geospatial Intelligence Agency, which serves both the NSA and the U.S. Department of Defense, with the same kind of data Nielsen is now using to measure consumer behavior around the world.
“I was talking about the commercial applications of this data and he came up to me and said they had never thought about how valuable that would be for intelligence,” Donato recalled, noting that one of the things the military and security agencies routinely analyze is data that could predict “strategic unrest.”
“They look at things like crop failures to understand how that could lead to unrest that could affect security,” Donato explained. “But he realized that there are other ways of looking at this data to understand what's going on with burgeoning commerce in an area. That's what we're doing. We're using it to understand supply chains.”
Donato said that as far as he knows, Nielsen does not and has not worked for the military or any national security agencies or agents. And while the techniques Nielsen has begun developing could have applications for them, Donato said the goal is to give military-grade intelligence to the world's biggest marketers and media companies. What follows are some examples of how it is working.
In some developing markets -- including countries like Kenya, Ghana, Lebanon, Indonesia and the Philippines, Donato said Nielsen has begun working with wireless carriers to monitor and analyze phone records in real-time. He emphasizes that Nielsen is only doing this in nations that have “invited” Nielsen to work with the phone companies to help develop better marketplace data, and that all of it is ”behind firewalls” and none of it includes personally identifiable data about individual people.
“At 8 a.m., we see a spike in cell phone activity coming together in a place of interest, so we start analyzing it,” Donato explained. “We know it may be commercial activity -- maybe it's an open air market where people are gathering to do business. We don’t know, but we start looking into it.
“The next thing we do is hit the satellite, and we determine it's a fuel depot,” he said, referring to Nielsen's geospatial satellite imaging technology. “We then follow those same mobile phones everywhere they move and everywhere they stop. And every 30 minutes they stop at a convenience store, so we know they're unloading fuel."
While these scenes seem like they could come out of a Hollywood movie, they are being applied in the real world for some very practical, if seemingly mundane reasons, Donato said -- like “understanding supply chains” and “a consumer’s path to purchase.”
But like military intelligence, Donato said the data is only as good as the analysts interpreting it, so he said Nielsen has also recruited an elite and somewhat eclectic team of “data scientists,” including people with military intelligence backgrounds, as well as some other seemingly less obvious areas of expertise.
During one session, Donato said he was surprised to hear one of his top analysts draw an analogy between the satellite and mobile data to the way some animals forage for food. It turned out that the analyst was also an expert in animal science and was applying some of his logic from his PHD on how wolves forage to the way human activity can be viewed from space.
Donato acknowledged how serendipitous that might sound, but he said there is a fair amount of creativity and human judgement involved in analyzing and interpreting data -- whether it comes from TV meters, supermarket scanners, telephone records or satellite images.
“It turns out that foraging theory is a valuable way of analyzing this data, because in the developing world it's still a fundamental way of defining people's trading areas,” Donato said, adding ”in terms of how far they can walk in a week, and what they can carry in a week, and the patterns associated with that.”
Donato said Nielsen is currently utilizing these techniques in six developing nations and plans to have another six in its database within six months. And while the techniques are pragmatic solutions for markets where no other data may exist, he said he sees the potential for “reverse innovation” -- taking some of the methods created in the developing world and applying them to the most developed and sophisticated industrialized markets, including the U.S.
Toward that end, Donato said all of these data-gathering pieces fit into an even bigger Nielsen initiative -- what you might call Donato's grand scheme -- which is to find ways of taking all the “unstructured data” available in the world and figuring out ways of linking it to the most structured data Nielsen has. While that is a mission almost every Big Data company and user has, Donato explained that Nielsen is somewhat unique in its approach, because the structured data it has -- the actual measurements of consumer media and marketing behavior -- also happen to be the trading currency for most of the developed marketing world.
Donato declined to disclose what “unstructured data” companies Nielsen is working with, but he said it is working with at least two of the biggest, and that the goal is to find ways of linking Nielsen's measurements to what he describes as “data exhaust” -- or the data available in the real world that could potentially be made meaningful by linking it to one of Nielsen's trading currencies.
Those exhausts could include data locked in consumer retail receipts and loyalty card programs that are signals for future purchase intent, or they could be the torrent of unstructured and largely unfiltered data being generated directly by consumers themselves through social media -- comments, posts, tweets, blogs. Or even other forms of public unstructured data, such as news, journals or scientific papers. The trick, explained Donato, is finding ways to hook Nielsen's structured data to the morass of unstructured data to find those meaningful “signatures” that are important insights for marketers and media companies.
“It's a very early science, but we're trying a number of techniques to be able to put a science behind using social data,” he said. “You have to be able to tease it out, because right now, it's a very recursive system: Does social cause TV, or does TV cause social?”
Ideally, Donato said machine-learning systems utilizing sophisticated artificial intelligence will simply learn what's important from Nielsen's structured media measurements, and then use it to sort and parse the most meaningful signatures that could represent important insights from the rest of the world. And by world, Donato literally means the whole planet -- not just the part that can be viewed from space, but the parts that are increasingly meta tagged as part of the progression toward the so-called “Internet of things.”
The problem will always be the same one Nielsen has wrestled with ever since it began measuring consumer behavior -- understanding which data is most important.
“In a world where everything is data, only so much of it will be relevant,” Donato said, suddenly turning a little Zen on me. “Yes, it's true that if you throw a stone into the water it will cause a butterfly to be knocked off its perch somewhere, but so what? The key is go get the right signature that has relevance and meaning to you.”
On that note, Donato inexplicably shifted the conversation from butterflies, marketing and Big Data to rock and roll. Donato, who lives on a boat moored in New Jersey's Liberty Landing, casually mentioned he would be traveling abroad this week and was lending his boat to friends who are part of up-and-coming rock band Typhoon to stay on while they are in town touring the tri-state area.
There was a time when "inventions" went from the military to the public (think microwave and freeze-drying). That seems to be reversed, at least in this example.
I've heard Donato speak several times and always leave amazed. This guy has one of the coolest jobs in the industry! Thanks for the article.