Pillar · AI Risk Management

AI Risk

The risk is not missing out. It is adopting AI faster than you can govern it, then having to answer for it to an auditor, a board, or the public. In 2026 that means governing agents that act, the identities and credentials they hold, and the control planes that grant their authority, with a risk register that changes as fast as the systems it describes.

Reviewed October 5, 2026 · Standards in force

Glowing neural network sphere inside a dark server room with a few nodes drifting out of alignment in red, representing AI model drift and agentic AI risk governance. AI-generated illustrative image. — Glowing neural network sphere inside a dark server room with a few nodes drifting out of alignment in red. AI-generated illustrative image.

The buyer's problem

An annual AI assessment describes a system that no longer exists by the time the report is signed. Models are swapped, prompts change, agents gain tools and connectors, and vendors ship features weekly. On October 2, AWS fixed a CVSS 10.0 flaw that could hand any network client super-admin control of the Loom agent control plane, including stored credentials and IAM role policies, and GitLab fixed a 9.9 sandbox escape in its AI Gateway. The control has to be continuous: every AI system and agent on a live register, every grant of authority scoped and revocable, and every deviation detected while it can still be stopped.

What to ask before you deploy AI

Is it on the live register? Every model, copilot and agent needs an owner, a purpose, a data scope and a risk tier that updates when any of those change, not once a year.
What can the agent do, not just say? List every tool, connector and credential it holds. Agentic risk is measured in actions and permissions, including the power to send, write, buy or delete.
How do we stop prompt injection? Treat every retrieved document, email and web page as untrusted input. Separate instructions from data, filter tool calls, and require human approval for high-impact actions.
Would we notice drift? Monitor output quality, refusal rates, cost and tool-call patterns against a baseline, with alerts that page a person, so a model change or a poisoned source surfaces in hours.
Can we answer for it? Keep the evidence an auditor will ask for: risk decisions, test results, approvals and logs mapped to NIST AI RMF functions and ISO/IEC 42001 clauses.
What can the model remember? Ask whether conversation memory, retrieval caches or fine-tuning can retain source code, secrets or regulated data, how long, and how it is purged. Keep repositories with credentials out of AI assistants until secret scanning runs first.
Where is the non-human identity boundary? Give each agent its own identity with an owner, a task scope and an expiry. Never let an agent inherit a human administrator's session, and front every agent control plane with an identity provider.
Can data leave without approval? Enforce egress allowlists for agents and AI gateways, block unknown destinations, and alert on uploads of code or regulated data to unapproved AI services.
Could we reconstruct what an agent did? Log every tool call, data access and outbound request an agent makes under its own identity, retain it for the audit period and test that you can replay one day of activity.
Can one email steer our agent? For every agent that reads email or documents, test whether hidden or obfuscated instructions can make it run code or send data, and make sure blocking happens before execution, not after.

The 2026 standards this page is aligned to

NIST AI RMF 1.0 (revision in progress) and the Generative AI Profile (NIST-AI-600-1) Voluntary. NIST says AI RMF 1.0 is being revised under the White House AI Action Plan; as of October 5, 2026 no AI RMF 2.0 has been published. On April 7, 2026 NIST released a concept note for a critical infrastructure profile.
NIST ↗
ISO/IEC 42001:2023 The certifiable AI management-system standard. Buyers increasingly ask AI vendors for certification or a dated roadmap to it, the same pattern SOC 2 followed for SaaS.
ISO ↗
EU AI Act Enforcement by the AI Office and national authorities began August 2, 2026, including Article 50 transparency duties. The AI Omnibus moved stand-alone high-risk obligations to December 2, 2027.
European Commission ↗
OWASP Top 10 for Agentic Applications 2026 The peer-reviewed list of agent risks, led by ASI01 Agent Goal Hijack. It is the shared vocabulary for agentic red-team scope.
OWASP GenAI Security Project ↗
Careful Adoption of Agentic AI Services Joint guidance from CISA, NSA and Five Eyes partners, published May 1, 2026: start with low-risk tasks, limit agent autonomy and access, and keep human oversight and monitoring.
NCSC New Zealand ↗
NIST Cyber AI Profile (IR 8596, draft) A Cybersecurity Framework 2.0 community profile for AI. The initial preliminary draft closed comments January 30, 2026; NIST planned a next draft for summer 2026.
NIST CSRC ↗
Microsoft Digital Defense Report 2026 Released October 1, 2026. Frames AI security around agent identity, appropriate access, authentication between agents, attribution and the ability to revoke access.
Microsoft ↗
White House Accord on Super Intelligence (voluntary) Announced September 29, 2026 with Executive Order 14434. Signatories commit to four layers: internal controls, an internal oversight team, external auditors and an independent committee. Not binding on deployers.
Nextgov/FCW ↗

From annual assessment to continuous AI risk management

B3 analysis. Each row maps a control to the standard an auditor is most likely to cite.

Control area	Static model (retire)	Continuous model (2026)	Framework anchor
Risk register	Spreadsheet refreshed at the annual review	Live inventory of models, agents and connectors; risk re-scored on every change	NIST AI RMF Govern and Map; ISO/IEC 42001 clause 6
Agent governance	Policy says “human in the loop”	Scoped identities, least-privilege tools, approval gates for high-impact actions, a tested kill switch	OWASP ASI02–ASI03; Five Eyes agentic guidance
Prompt-injection defense	Vendor assurance letter	Untrusted-content isolation, tool-call allowlists, output filtering, recurring red-team prompts	OWASP ASI01; NIST AI RMF Measure
Drift detection	Accuracy test at go-live	Baselines for quality, refusals, cost and tool calls with paging alerts and rollback	NIST AI RMF Manage; ISO/IEC 42001 clause 9
Evidence	Screenshots assembled before the audit	Logs, approvals and test results collected automatically and retained	ISO/IEC 42001 clause 7.5; EU AI Act record-keeping
Model memory and code leakage	Trust the vendor's privacy page	Memory and retention settings documented per tool, secret scanning before code reaches an assistant, purge tested	OWASP LLM02 Sensitive Information Disclosure; ISO/IEC 42001 Annex A data controls
Non-human identity boundaries	Agents run on shared service accounts	One identity per agent, scoped tools, expiring tokens, identity provider in front of every control plane	Microsoft DDR 2026 agent identity; SOC 2 CC6.1–CC6.3
Agent drift detection	Spot checks after complaints	Automated baselines for tool-call mix, destinations and volume, with auto-pause on deviation	NIST AI RMF Measure and Manage; OWASP Top 10 for Agentic Applications
AI incident evidence	Chat transcripts saved ad hoc	Action-level logs per agent identity, retained and replayable, with a named owner for AI incident decisions	White House accord internal controls; NIST AI RMF Govern 1.5 and Manage 4
Agent execution boundary	Agent warns the user after acting	Untrusted content cannot trigger code execution or outbound actions without policy approval; connected accounts scoped to the task	NIST AI RMF Manage 2; OWASP LLM01 prompt injection

This is one of the six domains in the readiness assessment. See where your organization actually stands.

Take the readiness assessment

Current signal · October 5

One email was enough to hijack an AI agent.

Salt Labs showed that an obfuscated payload in a single email ran code in the Manus agent's cloud sandbox before its warning appeared, reaching the email, storage and code accounts the user had connected. The issue was reported through Meta's bug bounty and fixed; there is no CVE. Separately, Korean investigators suspect AI agents probed seven lenders for weak points.

Test now: Send one benign, obfuscated test instruction to an inbox your agent reads. If it acts before anything stops it, restrict its execution rights first.

Read the October 5 brief →

Watch

Public talks, with a buyer-side read.

Curated public-sector and regulated-technology speeches, each with buyer-side commentary.

Loading…