Datadog Releases Data Streams Monitoring to Assess Streaming Data Pipeline Performance

Datadog, the monitoring and security platform for cloud applications,announced the general availability of Data Streams Monitoring, which makes it simple for organizations to track and manage the performance of applications that rely on messaging systems like RabbitMQ. Data Streams Monitoring automatically visualizes all interdependencies and key health metrics across all streaming data pipelines to help organizations prevent and troubleshoot latency and downtime.

Latest Insights: AiThority Interview with Vova Kyrychenko, CTO at Xenoss

Traditional solutions often provide only a limited view of streaming data pipelines and struggle to consistently identify the root causes of issues, such as drops in throughput or failing services. They may also lack context from upstream and downstream dependencies. This means internal teams must devote more time to manual tasks and troubleshooting in order to gather relevant information, analyze it contextually and determine the necessary collaborative efforts with other teams.

Streaming pipelines lie at the heart of numerous mission-critical applications, including banking and trading technologies, automotive systems, payment platforms and video infrastructures. Teams responsible for managing these streaming pipelines require precise measurement of end-to-end request latencies as they are frequently governed by service-level agreements (SLAs) and regulatory restrictions.

As part of Datadog’s APM platform, Datadog Data Streams Monitoring automatically visualizes interdependencies and key health metrics across all streaming data pipelines. With Data Streams Monitoring, engineers can measure latency between any two points across an entire streaming data pipeline, locate faulty queues or services and mitigate floods of backed-up messages so they can avoid critical downtime.

“Building streaming data pipelines to power our digital trading platform requires visibility into throughput and latency across the entire system, including all queues and services,” said Darren Furr, Solutions Architect at MarketAxess. “With Data Streams Monitoring, our team is able to proactively discover performance bottlenecks and optimize stream processing to ensure maximum throughput and low latency delivery of data to our customers.”

Latest Insights: AiThority Interview with Luke Damian, Chief Growth Officer for Applause

“Kafka has become a key component in powering our data processing infrastructure and gaming data streams, which are built using highly sophisticated architectures,” said SeungYong Oh, VP of Engineering at Devsisters. “Datadog Data Streams Monitoring allows us to easily map the topology of our Kafka pipelines across our globally spread clusters, monitor end-to-end latency and identify sources of incidents across our entire pipeline in one place. Having Data Streams Monitoring integrate seamlessly with APM helps us ensure that millions of gamers around the world always get the best possible experience.”

“There is a gap in visibility between SREs and application developers today, which limits their understanding of how queues are impacting mission-critical services and end-to-end latency of high-value user experiences like the resolution of banking transactions, the generation of shopping recommendations or payment processing,” said Omri Sass, Group Product Manager at Datadog. “While existing solutions can only observe individual queues and services, Data Streams Monitoring provides visibility into an organization’s entire ecosystem to make troubleshooting quicker and easier.”

Latest Insights: AiThority Interview with Ahmad Al Khatib, CEO and Founder at Qudo

[To share your insights with us, please write to sghosh@martechseries.com]