New Research Finds Data Automation Adoption to Climb From 3.5% to 88.5% Over the Next 12 Months

Adoption Rising as the Need for Data Products Grows Faster Than Team Size and a Staggering 95% of Data Teams Are Still at or Over Capacity

Ascend.io, the Data Automation Cloud, announced results from its third annual research study, The DataAware Pulse Survey, about the work capacity and priorities of data teams. Findings from more than 500 U.S.-based data scientists, data engineers, data analysts, enterprise architects, and—new this year—chief data officers (CDOs) reveal that despite 81% of respondents indicating that their team’s overall productivity has improved in the last 12 months, 95% of teams are still at or over capacity—just a 1% decrease from the 2021 study. The study also found that data automation is emerging as the most promising path to increase data team capacity and productivity, with a majority (85%) planning to implement automation technologies in the next year even though only 3.5% of the same respondents reported currently have automation technologies in place.

Ascend.io, the Data Automation Cloud, announced results from its third annual research study, The DataAware Pulse Survey, about the work capacity and priorities of data teams.

Data Initiatives Are Ballooning Beyond Team Capacity

Nearly all data teams (93%) anticipate the number of data pipelines in their organization to increase between now and the end of the year—with 57% projecting an increase of 50% or greater. Amid the rising number of data pipelines across their organization, nearly three in four respondents (72%) indicated that the need for data products is growing faster than their team size. This was especially true among data engineer respondents, of which 82% stated that the need for data products was increasing at a faster rate than their team size.

Recommended AI News: nerdytec Launches Sonic the Hedgehog 2 Couch Master in Collaboration With Paramount Pictures

“Data team productivity remains the single biggest threat to the success of data projects and workloads,” said Sean Knapp, CEO and founder of Ascend.io. “In fact, data team capacity has only marginally improved year over year, yet the demands on these teams continue to grow exponentially—far beyond what teams can feasibly keep up with.”

Team Backlogs Have Emerged Across the Data Lifecycle

One major roadblock to data team productivity remains fast access to data. When asked how much time they spend trying to gain access to the data they need to do their job, respondents said they spend an astounding 18.91 hours on average per week. Data scientists spend the most time trying to gain access to data each week at 24.6 hours, followed by data engineers at 19.1 hours.

However, data access is not the only roadblock. When it comes to the other top bottlenecks for team productivity, 66% cited team size or hiring constraints as their biggest productivity roadblock, followed by technology limitations (42%).

When asked which activities or tasks in their organization’s data ecosystem are the most backlogged, respondents were split. However, data scientists, data engineers, data analysts, and enterprise architects all agree to disagree—they each were all more likely to identify their own function as the most backlogged or resource-demanding compared to their peers.

• Data scientists are 3.3 times more likely to say data science is the most bottlenecked
• Data engineers are 2 times more likely to indicate data engineering
• Data analysts are 1.9 times more likely to say data analysis
• Enterprise architects are 1.5 times more likely to indicate data architecture

Recommended AI News: Innovative New AdTech Provider Interact Makes a Splash With Charity: Water Partnership

Data Teams Look to Automation, Flex Code, and Data Mesh to Increase Productivity

As data teams look for ways to overcome bandwidth limitations, many data professionals are turning to automation to improve data workload efficiency and productivity. In fact, while only 3.5% currently use them, 85% of respondents indicated that their team will likely implement data automation technologies in the next 12 months.

As data teams assess new solutions, many are considering low-code tools and data mesh frameworks to unlock greater team efficiency and business value. Respondents indicated a strong interest in low-code tools that provided greater flexibility (i.e., flex code), with the majority (81%) saying they would be more inclined to use a no-code or low-code tool if it offered the ability to use their preferred programming languages, up from 73% in 2021.

Respondents also cited a strong interest in data mesh frameworks, with 76% planning to implement a data mesh in the next 12 to 24 months. The majority (86%) of data teams believe a data mesh ​​will enable their business to make the most of their existing data architectures and resources. A striking 90% of CDOs agree that a data mesh will enable the business to make the most of their data investments.

“The numbers don’t lie—data teams must find a way to dramatically accelerate their productivity, and the overwhelming majority are looking to automation as the answer,” said Knapp. “Data leaders are increasingly finding that leveraging automation in conjunction with flex code and data mesh technologies significantly increases productivity and amplifies the impact of some of their most talented resources.”

Recommended AI News: Ellucian to Expand Strategic Partnership with PowerSchool

[To share your insights with us, please write to sghosh@martechseries.com]