Synthèse
To safely adopt advanced AI capabilities without introducing unpredictable failures into deployment pipelines, organizations should ground their approach in established industry standards. While guidelines like the NIST AI Risk Management Framework (RMF) and the OWASP Top 10 for LLMs provide comprehensive baselines for identifying risks, operationalizing these controls requires a structural blueprint. Frameworks like Google’s Secure AI Framework (SAIF) and Google’s approach to secure AI Agents p
To safely adopt advanced AI capabilities without introducing unpredictable failures into deployment pipelines, organizations should ground their approach in established industry standards. While guidelines like the NIST AI Risk Management Framework (RMF) and the OWASP Top 10 for LLMs provide comprehensive baselines for identifying risks, operationalizing these controls requires a structural blueprint. Frameworks like Google’s Secure AI Framework (SAIF) and Google’s approach to secure AI Agents provide a practical path forward, demanding that organizations extend existing deterministic controls directly into the AI execution environment. When deploying AI agents, security teams should navigate specific operational and structural risks: Pre-agent data security and Defense-in-Depth: Agents should not be able to access personally identifiable information (PII), protected health information (PHI), or other sensitive data. Organizations should enforce data security before the prompt reaches the model.
This includes strictly using non-production environments populated with synthetic data for testing. For production, security teams should deploy a hybrid defense-in-depth model. This includes Layer 1 deterministic policy engines acting as chokepoints, alongside Layer 2 reasoning-based defenses like specialized guard models (such as Model Armor or similar provider-agnostic guardrails) to filter out sensitive data and block malicious prompt injections before they reach the agent layer. Crucially for vulnerability discovery, security teams should treat the codebase itself as an untrusted input. Threat actors can embed indirect prompt injections within source code comments or third-party dependencies (e.
g. , hidden instructions telling the agent to ignore vulnerabilities or exfiltrate environment variables), making input sanitation a requirement even for internal scanning.
Source originale
nist.gov