3 Top Trends in Big Data & 3 Things Holding Them Back

Post by Ashley Stirrup, CMO of Talend

Big Data as a set of technologies and as a business strategy is maturing. The upside to this maturation is more advanced tools, smoother deployments, and new business opportunities. The downside is the rise of new challenges that require smarter strategies if companies want to be truly successful in achieving their digital transformation goals.

A company’s digital transformation should start with a clear view of trends and obstacles that might derail that campaign so it can better map a route to the business outcomes it seeks. With that in mind, here are the top 3 big data trends to keep an eye on, as well as the top 3 obstacles that are likely to stand between your company and success in the digital age.

Top 3 Trends:

• Getting real about machine learning. First the mantra was that machine learning — and by extension artificial intelligence (AI) – would soon take over the world, or at least most human jobs. However, the reality is coming into better focus now and organizations are finding that machine learning works best as an assistant to humans rather than a replacement. Human plus machine learning is the winning combo. With that in mind, the trend now is for companies to strategize how best to supplement human talent with a machine helper.
• Shift from data hunter/gatherer to data manufacturer. Until recently, companies have been keenly focused on mining the data they own, and finding and collecting data that other organizations own. But now there is a conscious shift to deliberately and strategically create data that is needed in order to sell new products and services and meet business objectives. For example, Accolade, an on-demand healthcare concierge for employers, health plans and health systems, provides guidance and personalized services to its customers by collecting data on a patient’s lifestyle and health condition, to orient patients to the right insurance provider options. Soon, companies will go much further—directing existing sensors to collect specific types of customer data on-demand.
• Optimizing customer experience in new ways. One of the few remaining low-hanging fruits in big data customer analysis is in improving the customer experience. Interestingly, it’s just now trending as a use case but it’s doing so with sophisticated aplomb: using natural language processing coupled with existing analytics such as sentiment analysis on social media.

Top 3 Obstacles:

• Data governance woes. While data governance is and always has been a top concern, the very concept of data governance is evolving to meet the need for more granular controls in order to remain in compliance with forthcoming regulations like GDPR and others. Companies need to not only control who can access what data and when, but they also need to know where the data came from (chain of custody), who possesses it or controls it, whether the data has been modified, which data this data set replaced, and other information pertaining to governing reliability, security, and accountability. Have this level of detailed insight is rather difficult to obtain, which is why governance is a primary obstacle to big data success.
• Multi–cloud management mishaps. Managing and keeping track of multi-cloud environments is already quite taxing. Companies can expect that problem to grow as more data, app and processing capabilities move to the cloud. While at first blush, the world of multi-cloud can appear to be nothing more than a managerial headache for IT leaders, it offers a myriad of both opportunities and challenges that need to be carefully considered when architecting a global enterprise approach to cloud management. 
• Self-service snags. Self-service capabilities are all the rage today in the ongoing effort to decouple IT from data, and put users in charge of it instead. Unfortunately, in most cases we swapped a bottle-neck in IT for a bottleneck in the platform. The obstacle here is a matter of scale: how to make data available to hundreds if not thousands of users simultaneously. Decoupling data from IT and moving to user self-service models is just the first step in transforming a company into a truly data-driven organization. The next one is to decouple data from business-as-usual so that data becomes the business engine.

The way I see it, the use cases for big data are only limited by human imagination. There are a few we can already clearly see coming—for example, imagine the customer buying experience evolving to the point of sophistication where grandparents that purchase a fire truck for their grandson’s 6th birthday then receives new product offers that map to the grandson’s age with each passing year. Imagine anticipatory analytics that power automation to prepare everything for your next meeting, from gathering the digital files you’ll need ahead of time, to ordering lunch that meets the tastes and health requirements of every individual attending the meeting.

The world of big data has already evolved at a race pace over the last four years, but the best and most exciting parts are still to come. What’s important to realize is the real ROI from any big data deployment results from the processes a company puts in place to utilize data and having the discipline to continuously improve those processes and approach to becoming more data driven. Keeping an eye on the future while implementing the tools needed to address current trends and navigate past immediate obstacles is the best way for any company to traverse their digital transformation journey.