Protect AI Open Sources Three Tools to Help Organizations Secure AI/ML Environments from Threats

Protect AI, the leading artificial intelligence (AI) and machine learning (ML) security company announced a set of open-source software (OSS) tools designed to help organizations protect their AI and ML environments from security threats. The company is leading security for AI/ML by developing and maintaining three OSS tools — NB Defense, ModelScan and Rebuff — that detect vulnerabilities in ML systems and are freely available via Apache 2.0 licenses to Data Scientists, ML Engineers, and AppSec professionals.

Recommended AI News: Cloudflare Equips Organizations with the Zero Trust Security They Need to Safely Use Generative AI

.@ProtectAICorp open source tools help organizations secure AI/ML environments from threats. NB Defense, ModelScan, and Rebuff detect vulnerabilities in Jupyter Notebooks, Machine Learning Models and LLM user prompts

OSS has become one of the most important components for helping companies innovate quickly and maintain a competitive advantage. It underpins much of the software used by organizations in their applications, particularly for AI and ML applications. While OSS offers clear benefits, it also poses inherent security risks. Although widespread efforts have been made to secure the software supply chain, the focus on AI/ML security has been overlooked. Protect AI is committed to helping build a safer AI-powered world, and in doing so has taken significant steps to securing the AI/ML supply chain.

In addition to the recent announcement of Protect AI’s Huntr, the world’s first AI/ML bug bounty platform focused on fixing AI/ML vulnerabilities in OSS, the company is also actively contributing to this effort by developing, maintaining, and releasing first of a kind OSS tools focused on AI/ML security. These tools include, NB Defense for Jupyter notebook security, ModelScan for model artifacts, and Rebuff for LLM Prompt Injection Attacks. All three can be used as standalone tools, or can be integrated within the Protect AI Platform which provides visibility, auditability, and security into ML Systems. The Protect AI Platform provides an industry first look into the ML attack surface by creating a ML Bill of Materials (MLBOM), that helps organizations detect unique ML security threats and remediate vulnerabilities.

“Most organizations don’t know where to start when it comes to securing their ML Systems and AI Applications,” said Ian Swanson, CEO of Protect AI. “By making NB Defense, Rebuff, and ModelScan available to anyone as permissive open-source projects, our goal is to raise awareness for the need to make AI safer and provide tools organizations can start using immediately to protect their AI/ML applications.”

Recommended AI News: Cloudflare’s R2 Is the Infrastructure Powering Leading AI Companies

NB Defense – Jupyter Notebooks Security

Jupyter Notebooks are an interactive web application for creating and sharing computational documents, and are the starting point for model experimentation for most data scientists. Notebooks enable code to quickly be written and executed, can leverage a vast ecosystem of ML-centric open-source projects, make it easy to explore data or models interactively, and provide capabilities to share work with peers. Creating a threat vector for malicious actors, notebooks can often be found in live environments with access to sensitive data. With no commercial security offering in the market that can scan a notebook for threats, Protect AI built NB Defense as the first security solution for Jupyter Notebooks.

NB Defense is a JupyterLab Extension, as well as a CLI tool, that scans notebooks and/or projects looking for problems. It detects leaked credentials, personally identifiable information (PII) disclosure, licensing issues, and security vulnerabilities. NB Defense improves the security posture of data science practices and helps protect ML data and assets. Visit this link to get started with NB Defense.

ML models are shared over the internet, between teams and are used to make critical decisions. Yet they are not scanned for code vulnerabilities. The process of exporting a model is called serialization, and involves packaging it into specific files for use by others. In a Model Serialization Attack, malicious code is added to the contents of a model during serialization — a modern version of the Trojan Horse. These create vulnerabilities that can be used to execute multiple types of attacks. First is Credential Theft, that allows for writing and reading data to other systems in an environment. Second, Inference Data Theft that infiltrates requests to the model. Third is Model Poisoning, which alters the results of the model itself, and finally, Privilege Escalation Attack which loads the model to attack other assets like training data.

Recommended AI News: Cloudflare Releasing New Tools to Help Manage Gen AI Security Risks

[To share your insights with us, please write to sghosh@martechseries.com]