Rafay and Fortanix Help Neoclouds, Sovereign AI Clouds and Enterprises Deliver Confidential AI as a Service

Joint solution lets operators run customers’ AI workloads in attested, hardware-isolated environments where sensitive data and models stay encrypted even while in use.

Rafay Systems and Fortanix® Inc. announced a joint solution that lets neoclouds, sovereign AI clouds and enterprises deliver confidential AI as a Service. The Rafay Platform now orchestrates workloads that run on CNCF® Confidential Containers (CoCo) with Fortanix Confidential AI, extending the companies’ existing integration for confidential virtual machines.

With the joint solution, operators can offer AI workloads that run inside hardware-based Trusted Execution Environments (TEEs), where encryption keys are released only after Fortanix verifies the environment and AI workload through cryptographic attestation. End users launch those workloads from the Rafay Developer Hub without filing a support ticket.

In regulated industries such as financial services, healthcare and the public sector, many of the mission critical AI use cases involve sensitive data such as customer records, patient information and financial transactions, and model providers need to protect the proprietary models that process it. These workloads often run on GPU infrastructure operated by a 3rd party: a neocloud, a sovereign cloud or a shared internal AI factory. Encryption protects data at rest and in transit, but without confidential computing, data and model weights must be decrypted for processing, leaving them exposed in CPU and GPU memory to anyone with privileged access to the infrastructure. Research from Fortanix and Atomik Research found that while 93% of AI leaders understand traditional encryption does not protect data while AI is processing it, only half can fully verify that protection is in place during inference.

Also Read: AiThority Interview with Tim Jurka, VP Engineering at LinkedIn

“Confidential computing enables model owners to deploy proprietary AI models on third-party infrastructure with reduced risk of exposing valuable intellectual property, while creating new revenue opportunities for cloud operators to offer premium, security-focused AI inference services.” said Mohan Atreya, chief product officer at Rafay. “Fortanix verifies the environment through cryptographic attestation and controls the secure key release. Rafay turns that into a governed, multi-tenant service, so a neocloud or sovereign cloud can offer confidential AI as a SKU, and its customers can run sensitive workloads on infrastructure they don’t operate. With Confidential Containers, that model now reaches the Kubernetes pod, where AI teams already ship their workloads.”

How the Joint Solution Works

CoCo, a CNCF Incubating project, applies confidential computing to the Kubernetes pod. Built on Kata Containers, CoCo runs each pod in its own lightweight confidential virtual machine, with memory encrypted by the CPU and, on supported hardware, GPUs included in the trust boundary. Developers deploy workloads with standard Kubernetes workflows.

Fortanix Confidential Computing Manager (CCM) performs composite attestation across the confidential computing environment, verifying the integrity of CPU and GPU trusted execution environments, firmware, and software components before authorizing the release of encryption keys. Fortanix Data Security Manager (DSM), backed by a FIPS 140-3 Level 3 HSM certification, releases encryption keys only to environments CCM has attested. If attestation fails, encryption keys remain protected, preventing the workload from accessing sensitive data or executing successfully.

The Rafay Platform turns that infrastructure into a service operators can deliver to their own customers:

• Cluster automation: Rafay blueprints install the Fortanix CoCo add-ons and keep them in sync on every confidential cluster.
• Multi-tenancy: Operators onboard GPU infrastructure once, then give each customer an isolated tenant.
• Self-service SKUs: Administrators package a confidential workload as a SKU in Rafay SKU Studio, and end users launch it on demand from the Rafay Developer Hub.
• Governance: Policy, access controls and quotas apply across tenants and clusters.

“Every AI Factory faces a trust dilemma. Model owners want their weights hidden from operators and customers; enterprises want their data hidden from model owners and operators; and operators must prove they can see neither, even via another tenant or compromised software,” said Anuj Jaiswal, Chief Product Officer at Fortanix. “Fortanix and Rafay resolve that dilemma in one click. Rafay orchestrates the deployment of proprietary models and AI applications across on-premises, sovereign and cloud infrastructures. Fortanix Confidential AI releases keys only to attested workloads, so model owners, enterprises, and operators, each keep control of what they own, every time it runs.”