Big Data Promise and Problem Formulation
Big Data is getting multiple conflicting viewpoints of late. Consider the following examples.
1) Big data is at the peak of inflated expectations, about to fall into trough of expectations http://blogs.gartner.com/svetlana-sicular/big-data-is-falling-into-the-trough-of-disillusionment/
2) Businesses, struggling to find employees to handle big data, the term assigned to gathering and analyzing massive quantities of information. http://www.computerworld.com/s/article/9232237/Big_data_worker_shortage_demands_job_candidates_with_diverse_backgrounds
3) IT industry would see creation of 4.4 million jobs globally by 2015 to support big data, but only one-third of the jobs will be filled because there is not enough talent, according to IT research and advisory firm Gartner.
The twin-edged opinions on the Big Data Usage and Landscape are to be looked into the context of the Gartner hype cycle. This is predictable and in synch with the earlier technology waves.
The reasons and limitations for the trough of disappointment are typically driven by the lack of a good use-case in the individual business case. As Gartner research director calls out in her article (link # 1 above), “Formulating a right question is always hard, but with big data, it is an order of magnitude harder”
Without a right use case and a right problem formulation, Big Data is likely to generate some more data, making it a race for nothing. The challenges in problem formulation are multi-sided:
- The wizards of Big Data that know the technology well have limited or no understanding of the business
- The peer-pressured adoption curve, which starts with the technology as the horse and the destination
- Business being in the learning phase of “Data Conscious Decision Making” – where the inter-relationship between data insights and business actions is yet to be understood.
- There is an extreme dearth of good, published business cases of adoption of Big Data, beyond the traditional internet log analysis and reverse indexing kind.
Bridging the gap, here is a wonderful document that lists the Business Use Cases of Big Data in the government sector, from TechAmerica Foundation’s Big Data Report. Must Read document in understanding the applications. Cryptic they are, but can fire up the imagination if one understands the data space.
Do you have a good use case of big data, do share them here.
Leave a Comment so far
Leave a comment