My issues with Big Data: Distraction

While reading the economist on the plane from Switzerland every other ad was somehow Big Data related. After a while it would not have surprised me if Lexus advertised for a car powered by Big Data. This made me think about the enormous amounts of effort and resources that goes into this concept. With Cloud being a household name, Big Data is perceived as the next thing to drive growth. Venture capital is funding Big Data start-ups while existing companies are re-branding or extending their product lines into the Big Data space. With money and time being scarce resources they have to be reallocated from somewhere else.



Looking at this from a information management perspective there are still big unsolved challenges and untapped opportunities that deserve all the attention they can get. Here are some examples:

  • Data quality is an issue in almost every organization. Big Data will only make this worse when attention is shifted to integrating vast amounts of noisy data with data of already poor quality.
  • Organizations are still introvert in their reporting and analysis. External data sources to benchmark and enrich internal information are underutilized. Big Data may be one such source, but there are far more mature and less costly alternatives from market research providers, governmental agencies and so forth. Some are calling this Wide Data.
  • Companies are not effectively utilizing their existing data let alone Big Data. Every consulting company worth their salt has some kind of BI or Information Management strategy offering. The logical reason is that there must be a big market for helping companies become more mature in this space.
  • Big Data is a solution looking for a problem. A lot of effort is going into finding this problem both among providers and customers. Good to know there are other avenues to follow.

On the positive side, Big Data is associated with Business Intelligence and related fields. Some of the effort put into it will surely trickle down into better offerings for the good old “small data” solutions. I just hope we do not get too distracted from the main purpose of our field: Help customers make better decisions.

Disagree? Feel free to discuss!