We are living in the age of data. As Google’s chairman, Eric Schmidt, famously stated in 2010, “There was 5 exabytes [1 trillion megabytes] of information created between the dawn of civilization through 2003, but that much information is now created every 2 days, and the pace is increasing…”
“Big data”—the current buzz-phrase in the marketing world—involves taking large, complex data sets—from web and e-commerce sites, social media, emails, text messages, mobile devices, point-of-sale and other electronic records—and combining them with powerful technologies that allow the rapid and dynamic analysis of the data.
Traditional data analysis required the use of predefined data sets and use of historical data to try to prove hypotheses of relationships between variables; big data analysis—which can allow quick or even real-time reporting—allows the correlation of seemingly unrelated and unstructured data and variables to specific results. Companies and their IT and marketing teams can leverage existing and new data sources and harness that information to predict and forecast with some accuracy consumer behavior. This information can also be used for target marketing and promotions, optimizing programs and even redesigning products and retail space. This can create significant improvement to the customer experience.
As an example of both the power of big data and the PR concerns that can arise, consider the story of one large retailer, which wanted to have its consumers come in for more than just discrete purchases. Their goal was to become more of a “one-stop-shop” for multiple items, but they recognized that shopping behavior is relatively habitual. They saw pregnancy as a small window when women and couples alter their shopping habits: their needs, and the stores at which they fulfill those needs, change with childbirth. In analyzing the purchases of women who signed up for the retailer’s baby registry, the retailer found that a number of purchase patterns emerged at different points during pregnancy.
The retailer was able to identify about 25 products that could be used together to provide a relatively accurate scoring methodology to predict pregnancyrelated buying factors. By using big data analysis, the retailer could apply that filter to their current data sets and determine the likelihood that a particular shopper was pregnant and even an approximate due date, allowing them to target promotional activities and coupons to that individual at each stage of pregnancy and after the presumed birth of the child.
Big data is becoming a part of normal business operations—but many companies have not fully considered the legal impact of their use of big data, ranging from internal practices and privacy policies, to information technology procedures and vendor/client agreements. Unlike the traditional view of data, which involves use of data for the primary purpose for which it was collected, big data analysis often involves the use of data for a secondary purpose that wasn’t even contemplated at the time the data was collected.
Companies are beginning to undertake big data reviews. This process involves looking not only at what data has been and is being collected, but also where the collection and storage takes place, if and how it is secured, how it can be used and who owns it. Given the relatively recent development of big data analysis, documentation and agreements are often silent or ambiguous on many of these important issues.
While big data provides some challenges, particularly at early stages of understanding the scope of the information collected and considering the implications of its use, it can provide enormous opportunities. Addressing potential issues early on can make the process work more smoothly, and can allow the realization of significant benefits to companies and, most important, their customers.