Big Data Growing rapidly

Despite some confusing the Big data is growing rapidly. It has been growing as key business tool and the vendors are taking it seriously.

71 percent of mid-sized and large companies have plans for, or are currently involved with, Big Data initiatives. A recent Survey reveals keen interest for vendors in Big Data, it also shows that the sector isn’t fully mature. Big Data remains an emerging market sector.

For instance, the role of vendors and the relative status of vendors is still every much up for grabs. When participants were asked which vendors they were working with (or planned to work with) to address Big Data analytics, a large chunk – 43 percent – said “none.” Surprisingly, only one vendor was selected by more than 10 percent of respondents. Part of what’s holding back businesses is the big confusion surrounding Big Data.

While IT professionals realize they need to get on board with Big Data, many are concerned about issues like project and management costs, along with issues involved with scaling infrastructure and overcoming data silos and application integration. Additionally, firms that are already leveraging Big Data – the early adopters – face an array of challenges.

These include the myriad complexities of software and data integration, data curation and storage, and the difficulty in making analytics accessible for users throughout the organization. This last concern is of particular importance.

After all, the value of Big Data is enhancing decision throughout the business, not only in the C suite. This promise of decentralized knowledge is the key competitive advantage of Big Data. Ideally Big Data analytics enables personnel from the sales staff to the IT department to upper management to make smarter, faster decisions based on insightful analysis of a timely data set. Will this be easy?

Certainly not – particularly in light of the oceans of data that business now creates, from unstructured social media posts to data gleaned from always-on mobile apps. Survey respondents expect their data volumes to grow by 45 percent over the next two years – a stunning upward trajectory. Even more overwhelming, 10 percent of respondents forecast that their data volumes will double (or more) in that time period. And this survey response is supported by research from IDC, which forecast that by 2020, the world will generate 50 times today’s information. Oddly, IDC also predicts that, given the evolution of Big Data tools, the IT staff needed to manage the tsunami of data will grow by less than 1.5 times. An optimistic staffing forecast, perhaps.

How Big Data Will Grow So Big To be sure, the challenges of Big Data are numerous, including the need to scale to accommodate the sheer scope of data. Not surprisingly, the most popular survey responses to a question about how firms will scale for Big Data are “establish easy to use tools” and “increasing network bandwidth.” It’s interesting to note that “building analytics internally” also scored high among respondents. This may reflect that (as noted above) many respondents have yet to settle on a Big Data vendor and so still expect to rely more heavily on internal resources. In the years ahead, it’s reasonable to assume one other survey response about scaling will change: “migrating to cloud based storage.” While a mere 23 percent of respondents chose this, massive data volume will surely push this number higher in the years ahead.