Nvidia launches dedicated AI safety team to tackle autonomous agent risks

Late last month Nvidia listed multiple positions to form a dedicated AI safety and security engineering division. Roles range from a founding technical leader to engineers specializing in security research, evaluation and senior management. The unit’s mandate is to evaluate autonomous AI agents before they reach the market, probe their operational boundaries and develop AI-powered software that can automatically detect and patch code vulnerabilities.

Nvidia’s job descriptions stress a commitment to open-weight models, transparency and extensive scientific scrutiny as pillars of American AI leadership and cybersecurity. The initiative dovetails with the company’s broader advocacy for open-source AI, contrasting with closed-system approaches from rivals. In a recent letter to U.S. officials, CEO Jensen Huang urged support for open AI architectures, arguing they bolster software resilience. Nvidia also joined the Open Secure AI Alliance, a coalition of about 120 firms, including Microsoft, Palantir, SpaceX and Hugging Face, aimed at building open-source defenses for machine-learning systems.