Predictions 2020: No Stopping “Kubernetifying” of the Analytics Stack

Alluxio Founder, Chairman and CTO Haoyuan (H.Y.) Li  Reveals Top Data Predictions 2020 for Hybrid Cloud, Machine Learning, Kubernetes, Talent gap, Hadoop compute and the Chinese Economy

Alluxio’s Founder, Chairman and CTO Haoyuan (H.Y.) Li forecasts seven major developments in the Cloud, AI, DevOps, Data Analytics and Storage in 2020. HY thinks Containers and Kubernetes, with a push from AI, will make the biggest impact on the IT, Cloud and Big Data Analytics businesses.

With a majority of IT-based businesses looking to latch on to the AI and Open Cloud systems to boost their agility and security, it’s time to replace obsolete technologies with the faster, smarter and controllable ones.

Here’s what HY Li forecasts for the IT, Cloud and AIops players.

“Kubernetifying” the Analytics Stack

While Containers and Kubernetes work exceptionally well for stateless applications like web servers and self-contained databases, we haven’t seen a ton of container usage when it comes to Advanced Analytics and AI. In 2020, we’ll see a shift to AI and Analytic workloads becoming more mainstream in the Kubernetes land.

“Kubernetifying” the analytics stack will mean solving for data sharing and elasticity by moving data from remote data silos into K8s clusters for tighter data locality.

Talent Gap Will Inhibit Data Technology Adoption

Building the stacks that enable data technology into practice is hard, and this will only become more obvious in 2020.

As companies discuss the importance of data in their organizations, they’ll need to hire the data, AI and Cloud engineers to architect it. But there aren’t enough engineers who have expertise in these technologies to do that. This “super-power” skill is the ability to understand data, structured and unstructured, and pick the right approach to analyze it.

Until the knowledge gap closes, we’ll continue to see a shortage of these types of engineers – many companies will come up short on their promises of ‘data-everywhere’.

Hadoop Storage (HDFS) Is Dead… Hadoop Compute (Spark) Lives Strong

There is a lot of talk about Hadoop being dead…but the Hadoop ecosystem has rising stars. Compute frameworks like Spark and Presto extract more value from data and have been adopted into the broader computing ecosystem.

The Hadoop Distributed File System (HDFS) is dead because of its complexity and cost and because compute fundamentally cannot scale elastically if it stays tied to HDFS. For real-time insights, users need immediate and elastic compute capacity that’s available in the cloud. Data in HDFS will move to the most optimal and cost-efficient system, be it Cloud storage or On-prem object storage. HDFS will die but Hadoop compute will live on and live strong.

The Reality of Hybrid Cloud Adoption

We’ve been hearing people talk about the hybrid cloud for the past three years now; and, for the most part, that’s all it’s been – talk.

2020 is the year it gets real. We are seeing large enterprises refusing to add capacity on-prem to their Hadoop deployments and instead invest in the public Cloud. But they are still not willing to move their core enterprise data to the Cloud. Data will stay on-prem and compute will be burst to the Cloud, particularly for peak demands and unpredictable workloads.

Technologies that provide optimal approaches to achieve this will drive the rise of the Hybrid Cloud.

One Machine Learning Framework to Rule Them All

Machine Learning with models has reached a turning point, with companies of all sizes and at all stages moving towards operationalizing their model training efforts. While there are several popular frameworks for model training, a leading technology hasn’t yet emerged.

Just like Apache Spark is considered a leader for data transformation jobs and Presto is emerging as the leading tech for interactive querying, 2020 will be the year we’ll see a frontrunner dominate the broader model training space with pyTorch or Tensorflow as leading contenders.

AI and Analytics Teams Will Merge Into One as the New Foundation of the Data Organization

Yesterday’s Hadoop platform teams are today’s AI/Analytics teams. Over time, a multitude of ways to get insights on data have emerged. AI is the next step to structured data analytics. What used to be statistical models has converged with computer science to become AI and ML.

So data, analytics, and AI teams need to collaborate to derive value from the same data they all use.

And, this will be done by building the right data stack – storage silos and computes, deployed on-prem, in the Cloud, or in both, will be the norm. In 2020 we’ll see more organizations building dedicated teams around this data stack.

The Rise of the Chinese Cloud Industry versus the US Economy

China is moving to the Cloud on a scale much larger than the US and will leapfrog from on-prem to massive Cloud deployments for advanced workloads.

Over the past 5 years, while enterprises in the US have been moving in leaps and bounds to public clouds, enterprises in China have been investing mostly in on-prem infrastructure primarily for data-driven platform infrastructure. 2020 will be the inflection point where these changes.

China will leapfrog into the Cloud at a scale much larger than the US by adopting the public Cloud for new use cases, bursting in the cloud for peak loads and over time move existing workloads. Public cloud leaders in China will see dramatic growth that might outpace the growth of the current Cloud giants.

(To share your predictions and recent product developments in the Cloud and ITOps market, please write to us at news@martechseries.com)