AiThority Interview with Gagan Singh, VP of Product Marketing at Elastic

Gagan Singh, VP of Product Marketing chats about the new Elastic AI Assistant and how it uses generative AI to help engineers, the importance of real-time data analysis in observability, and more on automation and machine learning-driven insights. 

——-

Please tell us about your role at Elastic.

I’m a vice president of product marketing at Elastic where I lead the product marketing for Elastic Observability.

Can you elaborate on how the Elastic AI Assistant for Observability uses generative AI to improve problem resolution for engineers?

The Elastic AI Assistant for Observability leverages generative AI to empower SREs and operations teams, helping them streamline investigations, uncover actionable insights, and accelerate problem resolution. By integrating advanced AI capabilities with Elastic’s search and analytics foundation, the AI Assistant delivers tangible benefits to engineers managing complex systems. Here are some real-world examples:

1. Access to comprehensive data without breaking the bank
2. Faster and more accurate data analysis
3. More relevant and contextual insights for smarter decision-making
4. Embedded AI workflows
5. Reduced downtime and enhanced system reliability

Also Read: AiThority Interview with Adriano Koshiyama, Co-founder and Co-CEO of Holistic AI

How can Elastic’s solutions address challenges organizations are facing in observability today?

There is a long list of challenges organizations face when it comes to observability, but the four major ones that we see are exploding data volumes, the rising costs that come with those growing data volumes, slow, manual insights, and tool sprawl and siloed data.

Elastic is uniquely equipped to address these challenges. We consolidate observability data into a single platform, eliminating tool sprawl and unifying previously siloed data. Elastic also has the ability to store and manage a large corpus of data using different techniques that save organizations money. Additionally, our platform utilizes powerful AI capabilities that allow organizations to pinpoint anomalies quickly.

By combining innovative data management, advanced analytics, and unified visibility, Elastic empowers organizations to address the most pressing challenges in observability, driving faster resolutions, reducing costs, and improving operational resilience.

How crucial is real-time data analysis in observability, and what measures can ensure quick access to insights for users?

Real-time data analysis is quite critical due to the adverse business and reputational impact of poor user experience and downtime. To enable effective real-time observability, organizations should take the following measures:

Monitor both operational and business metrics

Observability should extend beyond system performance to include business-critical metrics. This holistic approach ensures that all KPIs are accounted for, making it easier to identify misalignments and resolve issues before they escalate.

Define and align SLOs, SLIs, and error budgets

Clearly defined service-level objectives (SLOs) and service-level indicators (SLIs) establish accountability and priorities. Error budgets act as guardrails, helping teams monitor systems and dependencies proactively to prevent breaches.

Leverage AI and ML for the “unknown unknowns”

With the exponential growth in data, manual analysis becomes impractical. AI and ML capabilities are critical for anomaly detection and predictive analytics, enabling organizations to identify and address issues before they impact customers. These tools also help uncover “unknown unknowns,” offering deeper insights that manual processes often miss.

Invest in real-time analytics

Powerful analytics capabilities should be integrated into observability platforms to ensure fast querying and real-time visibility. This allows teams to access actionable insights instantly, reducing response times and minimizing downtime.

By combining real-time data analysis, advanced AI/ML capabilities, and a focus on both operational and business metrics, organizations can ensure quick access to insights, maintain system reliability, and deliver exceptional user experiences.

In what ways does Elastic prioritize user experience in the design of its observability tools to cater to both technical and non-technical users?

Elastic prioritizes user experience by deeply understanding the needs of both technical and non-technical users. When developing new product capabilities, the Elastic design team works closely with the product team to understand the persona, the problem they are solving, and their challenges. We employ a strong User-Centered Design approach, conducting user research through interviews and usability testing. This informs our design decisions, ensuring the tools are intuitive and effective for everyone.

To cater to diverse user needs, we aim to design “hybrid” interfaces. These interfaces provide UI-driven flows for common tasks, simplifying complex operations for less technical users. At the same time, they offer advanced options and granular control for experienced users who require deeper customization and flexibility. This approach ensures the platform empowers and caters to users of all skill levels.

Also Read: AiThority Interview with Balakrishna D.R. (Bali) – Global Head for AI and Automation, Infosys

How can automation and machine learning-driven insights help engineers speed up issue resolution in complex, distributed systems?

Automation and machine learning-driven insights are transformative for engineers managing complex, distributed systems, where the sheer scale and interdependencies often make manual monitoring and troubleshooting impractical. Here’s how they help accelerate issue resolution:

Proactive issue detection

AI and ML observability solutions can identify anomalies early—before they become service-impacting.

Contextual understanding of systems

Complex systems are composed of numerous interdependent entities with upstream and downstream relationships. ML models analyze these relationships to pinpoint how an anomaly in one component might ripple through the system, impacting overall performance.

Automated correlation and root cause analysis

ML algorithms excel at correlating data from disparate sources, such as logs, metrics, and traces, to uncover patterns that humans might miss. This automated analysis significantly reduces the time spent manually sifting through data and speeds up root cause identification.

Dynamic baselines and adaptive insights

ML models dynamically learn the normal behavior of a system, adapting to changes over time. This capability ensures that anomaly detection remains accurate even as the system evolves, reducing false positives and enabling teams to focus on genuine issues.

Actionable recommendations

Advanced ML-driven observability tools don’t just detect problems; they provide actionable insights and recommendations. For example, they can suggest specific configurations, optimizations, or steps to mitigate issues, further accelerating resolution times.

Can you highlight emerging technologies that IT leaders and decision-makers should be paying attention to as observability continues to evolve?

As observability continues to evolve, several emerging technologies are shaping the future of how organizations monitor and manage their complex systems. IT leaders and decision-makers should continue focusing on generative AI and AI-driven observability tools, as we’ve already seen how revolutionary AI-powered solutions can be. I also predict that observability will continue moving toward unified data platforms and edge observability, so leaders should be paying attention to these applications and solutions as well.

Finally, OpenTelemetry is becoming the standard for telemetry data collection across metrics, logs, and traces. It promotes vendor-neutral observability, enabling organizations to instrument their systems consistently and efficiently while maintaining flexibility in choosing observability platforms. IT leaders should keep an eye on this framework as observability continues to evolve.

By staying ahead of these trends, IT leaders can position their organizations to navigate complexity, optimize performance, and deliver seamless customer experiences in an increasingly dynamic technological landscape.