deepset Unveils Reference Architecture for Self-Improving, Sovereign AI Agents on NVIDIA Infrastructure

New reference architecture combines open models, modular agent orchestration, and governed execution on infrastructure organizations control.

deepset, the company behind the open source Haystack AI orchestration framework and Haystack Enterprise Platform, unveiled a new open reference architecture developed with NVIDIA open models and software for self-improving AI agents designed to run in locally controlled and air-gapped environments. The architecture is designed for organizations that need AI systems to adapt continuously while keeping sensitive data and workloads under their control.

Developed as part of an ongoing technical collaboration between the two companies, the architecture combines open models and NVIDIA technologies for model serving, optimization and governed execution with Haystack’s modular agent orchestration.

Cybersecurity is the first proving ground. At NVIDIA GTC Berlin, the deepset will demonstrate the architecture through a cybersecurity agent that analyzes confidential source code for vulnerabilities and uses feedback from prior runs to refine its performance over time.

From “Deploy and Forget” to Continuous Improvement

At the center of the architecture are two Haystack agents that form a continuous feedback loop. The first performs the domain task. In the GTC demonstration, the agent analyzes confidential source code and vulnerability information within a locally controlled environment, using open models rather than sending sensitive code to external APIs.

A second Haystack agent analyzes traces and results from those runs and proposes changes to the agent harness. Each proposed change is evaluated against predefined objectives and the current version before it can be promoted, with approval, audit and rollback controls keeping people in charge of what reaches production.

The architecture combines four building blocks: open models for reasoning, a modular orchestration layer that connects those models to an organization’s data and tools, an optimization environment where improvements can be evaluated, and security controls governing what agents are allowed to do.

Haystack provides the open source orchestration layer and agent harness, managing how models interact with context, retrieval, tools and workflows and making those components independently adaptable. running.The solution uses NVIDIA Nemotron open models served through NVIDIA NIM microservices, NVIDIA NeMo Gym as the optimization environment, and NVIDIA NemoClaw open blueprints, which run the agent inside NVIDIA OpenShell — a secure runtime that enforces policy-based controls over files, networks, credentials, and tools.  Nemotron 3 is a family of high-efficiency, open multimodal models with leading accuracy to deliver fastest time to completion for long-running multi-agent systems.

Also Read: AiThority Interview with Gou Rao, co-founder and CEO at NeuBird AI

“Powerful open models create the foundation for sovereign AI, but realizing their full potential requires engineering the complete system around them,” said Milos Rusic, CEO and co-founder of deepset. “By combining NVIDIA’s open models and optimization technologies with Haystack’s agent orchestration, organizations in Europe and beyond can build specialized agents that evolve with their environment while operating within governed infrastructure they control, and under human oversight.”

“Organizations need agents that work effectively while retaining control over their data, infrastructure and operations,” said Pat Lee, vice president of strategic partnerships at NVIDIA. deepset is providing a reference architecture organizations can deploy themselves, with the controls regulated industries require.”

Sovereign by Architecture

For governments and regulated organizations in Europe and elsewhere, sovereignty requires more than running a model locally. Organizations also need control over what data and context an agent can access, which models and tools it uses, and the infrastructure on which the complete system operates.

The reference architecture is designed to preserve that control across the stack. Open models can run on locally controlled NVIDIA-accelerated infrastructure, including NVIDIA DGX Spark; Haystack keeps the agent harness modular across models, tools, and data sources; and OpenShell provides isolation and policy controls around agent execution.

Haystack’s modular architecture allows organizations to adapt individual parts of the system as their requirements change, reducing dependence on a single model provider or closed application stack.