"Big Data" is trending. The digital, interconnected, interdependent world can produce unlimited amounts of structured and unstructured data sets. The ability to harness huge amounts of data as a resource to produce business focus, efficiencies and profit is fueling a groundswell of interest and the growth of industries designed to exploit the opportunities created by the evolution of data-centric technologies.
In an automated world, governments, individuals, businesses and other institutions are generating more data more quickly than ever before. Consider how information is communicated, generated and stored – think computer email, smartphones, tablets, GPS, security cameras, automated toll payment systems, public cloud based storage, and weather sensors, to name but a few examples. To shed some light on just how "big" Big Data is and will be, the volume of data is expected to expand by orders of magnitude, from 130 exabytes in 2005 to 40,000 exabytes (40 trillion gigabytes) in 2020.1 This data includes everything that can be stored electronically, including items such as transaction information and financial records (e.g., shopping receipts), weather records, email and text archives, survey data, social network activity records and internet search data.
The utility of Big Data is not in its collection, however, but rather in the ability of collectors to analyze and process the data to retrieve usable information. Success lies in the ability to create the right analytics – that is to ask the right questions to derive the most useful, accurate and beneficial answers (note to parents: encourage your children to consider a career in data science). As important as the volume of data is, what really sets Big Data analysis apart from traditional data analysis is its capacity to handle variety. Big Data analysis will offer the ability to piece together seemingly disparate information from unrelated sources to enable us to view problems and potential solutions in a way never before possible. For example, combining an individual's GPS location information with the same individual's online shopping habits (different types of information obtained from different sources) may help retailers identify where to open new stores. A process that currently takes up to several years of market analysis may one day be compressed into a simple search run on data already on hand from continuous and passive collection.
Big Data could have tremendous value in areas such as retail and marketing based on the large-scale analysis of consumer behavior. The Wall Street Journal reported that "[r]oughly half of senior finance executives in the U.S. said they wanted to beat their competitors by mastering Big Data," so it may be surprising to learn that "less than a quarter said they will focus on implementing that technology over the next year."2
One reason for this hesitation is the traditional cost and risk of being a "first/early adopter" of an evolving technology – high implementation costs for developing tools (or procuring them from vendors that do not have long, proven track records) to convert raw data into useful information. Some executives cite the high cost for low immediate and unknown future value that is endemic to investing in any new technological improvements (remember how much a computer cost back in 1980?).3 Additionally, companies fear the potential negative reaction of the public to the seemingly ‘big brother'-style recording and analysis of its behaviors.4 Business is apprehensive of fully exploiting the potential of Big Data analytics before the public is comfortable with how it is applied.
That said, some companies are already recognizing the potential benefits and believe Big Data analysis should be implemented as a substitute for older forms of data analysis.5 Big Data is set to be a major factor in the future of knowledge development. If done right, the application of Big Data analytics could be essential to an enterprise’s economic viability and growth.