Nvidia unveils open safety platform to rein in rogue AI agents

Nvidia, the dominant force in AI semiconductors, has unveiled a tool designed to prevent AI agents from taking unauthorized actions beyond human control.

The company on Monday (local time) introduced the Nvidia Open Agent Safety Platform, which provides oversight across the full lifecycle of AI agents — from testing through deployment.

The platform consists of two components: OpenShell, an open-source software layer that runs on a CPU to define and control the behavioral boundaries of AI agents in real time, and Sentry, a hardware monitoring device that operates on a data processing unit and can isolate and shut down an AI agent within milliseconds the moment it crosses a defined alert threshold.

"The strong potential of AI can only be realized if we solve AI safety," CEO Jensen Huang said. "As AI capabilities advance, safety technology must keep pace."

The launch comes as incidents of AI agents breaching boundaries and infiltrating external systems without explicit human instruction have grown more frequent.

More than 100 leading technology companies worldwide contributed to the platform's development, including Anthropic, Microsoft, Salesforce, Palantir and SpaceX AI.