Upwind Unveils Novel Approach to High-Speed, Precise Detection of Malicious AI Prompts with Nvidia AI

Upwind, the runtime-first cloud security platform leader unveiled the results of research from RSAC Conference demonstrating that malicious Large Language Model (LLM) prompts can be detected with approximately 95% precision, while maintaining sub-millisecond inference for real-time traffic with Nvidia technology. In the evaluation, advanced LLM reasoning was applied only to a small subset of high-risk requests, avoiding the latency and cost overhead that has made many AI security approaches impractical at scale.

Also Read: AiThority Interview with Glenn Jocher, Founder & CEO, Ultralytics

Upwind has demonstrates that malicious Large Language Model (LLM) prompts can be detected with approximately 95% precision, while maintaining sub-millisecond inference for real-time traffic with Nvidia technology.

As enterprises move generative AI into production, with Gartner predicting that more than 80% will use generative AI APIs, models, or deployed enabled applications in production this year, application security is undergoing a fundamental shift. The interface itself, natural language, has become the attack surface. Unlike traditional exploits that target code vulnerabilities or malformed packets, LLM threats are embedded in language, manipulating meaning and intent. As these models move into enterprise workflows, they introduce new threat categories including prompt injection, jailbreaks, data exfiltration and social engineering. Traditional security controls are poorly suited to these threats.

“LLMs don’t just process input, they interpret intent,” said Moshe Hassan, VP Research & Innovation, at Upwind. “That changes the security model entirely. Organizations aren’t just trying to block bad code anymore, they have to stop attempts that twist language and manipulate systems. Our research with Nvidia shows you can do that effectively in live production environments, without slowing things down or driving up costs.”

A Three-Stage Architecture Built for Production

Rather than relying on a single heavyweight model or static rules, Upwind engineered a layered detection system designed around challenges including, latency, cost, false-positive tolerance and explainability.

The system operates in three stages:

Stage 1: LLM Traffic Identification
A lightweight classifier filters traffic to determine whether a request is even LLM-bound. This stage ran in under a millisecond and achieved 99.88% accuracy, ensuring that semantic analysis is applied only when necessary.

Stage 2: Semantic Threat Detection
Once a request was identified as heading to an LLM, the next challenge was determining whether it was malicious. The team analyzed these requests using the Nvidia nv-embedcode-7b-v1 model, deployed through NVIDIA NIM microservices. After testing multiple models, nv-embedcode-7b-v1 proved most effective at distinguishing normal prompts from malicious prompts, including indirect jailbreaks and prompt injections, while running on infrastructure fast enough for real-time protection. This stage achieved 94.53% detection accuracy, while maintaining inference times well under 0.1 milliseconds, demonstrating that high-quality AI security can operate at production speed and scale.

Stage 3: Selective LLM Validation
As part of a progressive, multi-stage workflow, high-risk or uncertain cases are escalated to NVIDIA Nemotron-3-Nano-30B model for a more reliable determination. NVIDIA NeMo Guardrails is also integrated to apply predefined rules and structured output formats, ensuring responses remain consistent and aligned with security policies. This selective escalation improves accuracy and decision confidence while keeping the system efficient.

From Detection to Actionable Security

Detection alone isn’t enough in modern cloud environments, where a flagged prompt is just one piece of a much larger puzzle. By embedding LLM threat detection directly into Upwind’s runtime and cloud visibility platform, malicious prompts are surfaced not as isolated model outputs, but as actionable security events within the broader cloud ecosystem.

Also Read: ​​The Infrastructure War Behind the AI Boom

[To share your insights with us, please write to psen@itechseries.com]