• by Jeff Weidauer , February 15, 2012

In the next few months, Facebook will welcome its billionth member. That’s more than one-sixth of the world’s population -- all sharing thoughts, concerns and what they had for dinner with their online “friends.”

Facebook is a perfect example of what is being called “big data,” which is a broad and somewhat subjective term that refers to data sets too large to be manipulated and managed by standard data tools. Data sets that qualify are measured not in gigabytes or terabytes, but in petabytes and zetabytes.

All of those postings on Facebook, Twitter and other social networks, along with history from the various search engines, are saved, catalogued and can be mined for insights. Facebook and Google make money by offering narrowly targeted ad space to marketers, which makes the ad both more effective and seem less intrusive to the target audience.

Retail marketers have been exploring the underlying premise of big data for more than 20 years through the use of loyalty cards, with mixed success at best. The quantities of data gathered -- roughly one gigabyte per store per week on average -- were more than most retailers were prepared to handle internally. While they continued to gather information, loyalty card programs quickly became discount tools that offered no real value to the shopper.

With social media and a nearly universal reliance on the Web for information, the data available for analysis via loyalty programs, social media, search engines, etc., is expanding exponentially. Therein lies the challenge for marketers. Not only will standard tools such as relational databases not suffice to glean insights, but few marketers have the training or experience to deal with them effectively.

Big data has the potential to make everything we do as marketers more effective and more measurable -- and it will change what marketing is and how it’s perceived. True one-to-one marketing is becoming an option, both from the insights we can derive from big data, and the near-ubiquity of mobile phones.

As consumers become more cynical, and get more tools that allow them to avoid advertising, big data will do more than just bridge the gaps. It will close the chasm between marketer and buyer, providing relevant, timely and compelling offers based on history, interests, and psychographics, all gathered from consumers who willingly offer up personal facts and a peek into their minds if they receive relevant value in return.

Several hurdles remain, but the first players to use big data will reap the greatest benefits, and it won’t be long before data analysis skills will be part of everyday business just as IT is. While many of the technical requirements have yet to be resolved, what is clear are some of the ways big data will help retail marketers to drive sales, lower costs, and improve overall shopper engagement:

• Cross-selling—using customer data to offer relevant cross-sell opportunities. This means mass personalization—going beyond the segment level and actually drilling down to individual needs. Amazon is doing this already, using customer data to generate a “you might also want” prompt for each product purchased or visited online. It claims to drive 30 percent of sales via this method.

• Integrating all the touchpoints across the path to purchase. This is using big data to tie offline, online, in-store and out-of-store promotions together, making for a more consistent and compelling customer offer.

• Location-based marketing. Geo-fences and in-store tracking are becoming more common, along with knowing when a shopper is near a store. All of this allows the marketer to direct real-time, highly relevant messages to persuade the customer to enter the store, or an aisle, that the shopper might not have otherwise.

• Optimizing assortment. Instead of the idea of “stock-keeping unit (SKU) rationalization,” or removing underperforming SKUs from inventory based on prior sales data alone, big data will allow retailers to look at other relevant data points (which items drive other sales; what is competitive availability, etc.) to make more informed decisions.

• Real-time intelligence. Big data will allow all businesses to operate using real-time data that can be acted upon quickly, rather than using day- or week-old reports that are backward looking and may not accurately reflect the present.

Big data is not yet fully-baked as a marketing tool. But it’s coming, and quickly, and we need to prepare now to make use of it when it arrives. The impact will be unlike anything we’ve experienced before.