Taming Big Data

The zettabyte is here.  Sun Microsystems became the first company to go to market with a zettabyte file system Nov. 20.  You don’t really need to know that, but what you do need to know: it’s an ominous sign that “big data” has no horizon, and it’s only getting bigger.  It could be more intimidating than the national deficit, but big data means big opportunity for digital marketers, if you know how to make sense of it.

We know that customer data, ad data and cloud computing data has sent what we can store into the zettasphere.  Today, a terabyte is tiny.  Big data is what happens when the earth’s population is flipping through web pages, sending social posts, texting and viewing other content.  All that activity generates “big data.”

According to McKinsey Quarterly: “Over the last few years, the volume of data has exploded. In 15 of the US economy’s 17 sectors, companies with more than 1,000 employees store over 235 terabytes of data—more data than is contained in the US Library of Congress. Reams of data still flow from financial transactions and customer interactions but also cascade in at unparalleled rates from new devices and multiple points along the value chain. Just think about what could be happening at your own company right now: sensors embedded in process machinery may be collecting operations data, while marketers scan social media or use location data from smartphones to understand teens’ buying quirks. Data exchanges may be networking your supply chain partners, and employees could be swapping best practices on corporate wikis.”

Again, intimidating.  But digital marketers and agencies need a strategy for Big Data. Here’s three ways to get started:

1. Get intimate with data. At PepsiCo, I worked in a group sometimes known as the “Big Problems” group.  One of the big problems was capacity in the restaurant division, Pizza Hut (now a division of Yum! Brands). 

Pizza ovens can only cook a finite number of pizzas during peak periods (dinner time) therefore limiting revenue opportunity.  We needed data, so naturally we went to the head of IT and clearly laid out the big picture strategy to grab data, isolating which of the thousands of locations had capacity bottlenecks at peak periods.  Big picture, big strategy -- we just needed data.

The answer we got was:  big consultant, big dollars, long timeline.

Then one day in the company gym, we cornered an IT manager and, over StairMasters and sweat rags, we asked little questions instead of big questions. 

Questions like:  What’s the smallest time interval of the summary data from the Point of Sale system?  What data are we capturing at point of sale? 

Then we took our sweaty selves to the R & D lab, physically measuring the geometry and capacity by minute for each oven type.  Then we asked our StairMaster buddy in IT how long it would take him to write the code to extract those little pieces of data.  Answer:  Three days.

When it came to Big Data, we had it bass-ackwards.  We logically started with big strategy and the big guy.  But how we solved the problem was getting intimate with the data; asking little questions, in little increment

In three days, we got our data, applied the capacity algorithms, discovered the specific bottleneck stores, and uncapped revenue opportunity in those stores.  No sweat.

2. Simplify. The problem with big data is that it’s getting bigger. Both CFOs and CMOs are drowning in data, yet thirsting for knowledge.  When presented with so much, humans simply can’t process and make sense of it all.  And not because we’re distracted by text messages, Facebook, or the like. 

Recent compilation of research shows humans can only process about 3-4 things at a time (Farrington, January 2011 PIQ).  So the trick is to simplify.

The easiest way to simply is to use percentages or ratios -- essentially taking two numbers and matching them together.  Not exactly earth shattering, especially when looking at so much data.

In our world of Internet adverting attribution, we bind three numbers into one ratio, which takes three of those four things the average human can process, reducing it to a single number.

And binding three powerful and relevant data points into one simple number, means even the CFO’s or CMO’s child can understand it.  Simplification takes the scary out of big data.

3. Stop collecting garbage. You’d be surprised by how many companies collect garbage data and make multimillion-dollar decisions from it.

With Internet advertising, the unfortunate and hurtful situation is this.  All the online ad tracking systems used by most advertisers are severely handicapped -- much of what they collect is garbage.

Here’s an illustration. Imagine a $100 Internet purchase from Lands’ End.  A reader on a newspaper website sees a display ad for Lands’ End.  The reader thinks about sheepskin boots and uses a different browser tab to surf Lands’ End.  After a fall weekend, the reader decides to order those boots by typing “Lands’ End” into Google, making a purchase online.  100% of the ad credit goes to the search term “Lands’ End,” which was merely the endpoint before checkout, not what got the reader thinking.  Since 100% credit ignores every ad except the very last ad, Lands’ End gets a wrong interpretation of what happened.  

Why?   Because that’s the way it started when online ad tracking systems were developed in the late 1990s, and that’s the way it’s always been done.  And is there a solution?  Yes, it’s called an attribution model that captures every online media source from the first ad stimulus where sales originate down to the very last ad. 

In a full-funnel attribution model, 100% of value, say from Lands’ End, is attributed among Originators, Assists, and Converters.  All three pieces of this pie equal 100%.  Each ad designated worthy of attribution credit is assigned revenue and divided to the cost of the ad, ending with a single number: [attributed revenue of the ad] divided by [total cost of the ad].  A powerful number, yet a simple ratio.

This chain of data is going to get longer, and understanding each data point is going to becoming increasingly important in informing how media is bought and sold online.  That’s Big Data.

When it comes to zettabytes, it can be a zoo.  Big Data gets big results, but only when tamed. 

Tags: metrics
Recommend (6) Print RSS
1 comment about "Taming Big Data ".
  1. Tilly Pick from Development Practice 360, LLC. , January 20, 2012 at 5:16 p.m.
    Amen! I'd add that it's easy to get distracted by the big data and the big strategy. Why? Because it's human nature to want to feel important and want to add value. Being involved with big data and big strategy does that more readily than bugging co-workers on StairMaster machines. In fact, my bet is that you had no idea where the StairMaster conversationn was going to lead you. I love that example!