Metaplane Releases CI/CD Tools to Help Data Teams Prevent Data Quality Issues

Metaplane launched Data Test Previews, a tool which prevents data quality issues before they’re introduced by automatically testing data engineers’ proposed changes. This marks Metaplane’s continued push into building powerful automated CI/CD (Continuous Integration/Continuous Delivery) tools for data engineers. When used alongside the company’s Data Impact Previews, which was launched in late 2022, data teams can have confidence that merging code won’t break downstream dependencies or the data itself.

 Latest Aithority Insights : Got It AI Announces AutoFlows, a Breakthrough Autonomous Conversational AI

Traditionally, if data engineers wanted to know if their changes would cause data quality issues, they’d have to spend hours or days writing and running manual scripts on development branches of their data. Once done, they’d have to manually compare the resulting metadata to understand how the proposed changes differed from production data. For large changes or cases where there are many downstream dependencies, this process could be an enormous drain on the data team’s time and effort.

With Data Test Previews, Metaplane runs a suite of data tests against a data team’s pull request and their production warehouse. It then compares the results and alerts users about any large changes to their production data’s mean, uniqueness, nullness, cardinality, and row count. These automatic tests are configurable; users can choose which tests run, as well as fine-tune the testing thresholds to fit their use cases and needs. Finally, users can share both the pull request and a Data Test Preview report with stakeholders about the changes being made and the impact of those changes.

AI News: An Investment Into Artificial Intelligence as Daktela Buys Coworkers.ai

With Data Impact Previews, Metaplane parses column-level lineage in their users warehouses and BI tools, and proactively alerts users about which downstream dashboards may be impacted by their proposed changes. And if any changes do happen to trigger a data incident, Metaplane helps users identify the root cause by highlighting the pull requests where the changes were made.

Metaplane believes that data engineers deserve tooling and automation for their development lifecycles that are just as functional and sophisticated as the tools that exist for software engineers. The company plans to continue expanding that toolset for its customers.

AiThority Interview : AiThority Interview with Hardy Myers, SVP of Business Development & Strategy at Cognigy

 [To share your insights with us, please write to sghosh@martechseries.com]