Big Data is a popular term today that references the huge volumes of business and consumer data being collected and stored by organizations, which cannot be effectively data mined due to the limitations of commonly used software tools that capture, manage, or process the data.
While first diagnosed in the science, government, healthcare and military industries, the vast volumes of consumer data being produced through social technologies has landed this reality – and this problem – on the desks of CMOs globally.
Not only is data being produced at lighting speeds, the devices used to produce, broadcast, measure, store and share that data are on the rise, which then fuels further content generation. The cycle is creating a Big Data cyclone that organizations will continue to struggle with.
The world’s technological per capita capacity to store information has doubled every 3 years since the 1980s. Today, 2.5 quintillion bytes of data is produced daily. To seize the opportunity, firms like Oracle, IBM, Microsoft, and SAP have spent $15+ billion on software firms that only specialize in data management and analytics.
Greater Insights or Just More Noise?
Data scientists argue that successfully and creatively understanding Big Data produces operational efficiencies and greater margins for business. Big Data is relative and can take different shapes in different organizations, ranging from a few dozen terabytes to many petabytes of data in a single data set. Some will seek to use it simply to aid in consumer and product forecasting where others will use Big Data collection and analysis to conduct controlled experiments to make better management decisions.
In either case, budgets are opening up for Storage Area Networks (SANs) where CIOs can stockpile data. But to what end? Storing data is one thing, leveraging the lessons locked within it is another. And the bigger and more varied the data sets are, the greater the difficulty a business will have in extrapolating nuggets of wisdom.
Is the money spent on storing Big Data today really driving business value? Even if we could successfully analyze and comprehend Big Data, we would have to invest in advance software to create unlimited analytical models that find the connections within the data. The hope is that these would lead to insights about customers and business processes. However, these are just more data points. An analyst(s) would still be required to determine which of those models is best suited for the marketing objectives, the business culture, current management styles, employee skill set, etc.
Is Big Data a Red Herring?
In our age of relationships and consensus building is big data not just a Red Herring? Does it not shift the focus of a business’ decision making and strategic planning to patterned historical transactions instead of monitoring future trend currents? What products will capture the imagination of your audience or how your brand strategy will be perceived in the future cannot be predicted by calculations or algorithms spewing out past transactional matrices.
If we’re to believe the current consensus on social business, enterprises are evolving through the power of crowdsourcing, public opinion and relationship building. Product innovation is being propelled by predicting the intangible not analyzing data sheets. There’s a real threat in making Big Data analysis a business strategy.
- Would analysing Big Data have led to the innovation of the iPod? How many products come out of R&D departments based on industry research, consumer panels and test scores only to fail?
- Would Big Data have predicted consumers turning on Blackberry? Would it have predicted the change in mindset of CIOs over use of corporate email via smart phones and other devices? I argue that even “small data” could have (and probably did) warn RIM that consumer preferences were changing but it did not stop the implosion this once-giant is now experiencing.
So my question to you is: is the current fixation on Big Data collection an operational reaction to the volume of data being created or truly a strategic paradigm for business? Will it simply get in the way of better judgment? Make executives lazy in their innovation?
Sam Fiorella