AI Vendor Security Checklist for Procurement Teams B3 buyer worksheet | Reviewed October 17, 2026 https://b3cyber.com/guides/ai-vendor-security-checklist Vendor / service: ____________________ Buyer / business owner: ____________________ Review date: ____________________ Use case / scope: ____________________ EVIDENCE SCALE 0 = claim only; 1 = dated evidence reviewed; 2 = buyer-witnessed test passed. B3 meeting tool, not a certification. Do not average away a failed critical control. Choose approval blockers before testing. Phase 1: Pre-commitment / discovery [ ] What business action are we authorizing? Write a one-sentence use case, its data classes, intended users and prohibited actions. Distinguish drafting advice from sending messages, changing records or approving payments. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Which AI tools already touch our data? Reconcile approved purchases with identity-provider application grants, browser extensions, connected SaaS features and staff-reported tools. Investigate unexplained access through your established privacy and IT processes. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] What does no training actually cover? Request a data-flow schedule covering model improvement, evaluation, abuse monitoring, support and subprocessors. Record retention and deletion exceptions explicitly; do not assume a settings toggle changes the contract. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Which requirements actually apply to this purchase? Identify jurisdiction, sector, contract and deployment context with the appropriate reviewers. Federal teams should check current OMB guidance, agency implementation and the solicitation instead of relying on a generic compliance badge. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ CHECK YOURSELF: Can we name the person allowed to accept the risk, and the person who can stop the service? Phase 2: Technical & operational proof [ ] Will external content change the agent’s authority? In an authorized sandbox with synthetic data, place conflicting instructions in a test document or message. Verify the agent cannot expand its tools, send data elsewhere or bypass the required approval. Preserve the input and action log. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Does the server enforce identity and scope? Have the security team test rejection of invalid, expired, revoked and wrong-audience credentials within the agreed test scope. Verify signatures, issuer, audience and object-level authorization independently of anything the model says. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Can we reproduce a consequential answer? Retain permitted source references, retrieval context, model/version, tool calls and reviewer decision, with sensitive values minimized. Do not require private model reasoning; require evidence of inputs, outputs and actions. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] What happens when the answer is wrong? Run a fixed set of normal, ambiguous, outdated-source and unanswerable cases. Set pass criteria by consequence before the test. Record the denominator, failures and human escalation behavior, not just an average score. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ CHECK YOURSELF: Did our team choose the difficult cases, or did we approve only the vendor’s prepared demo? Phase 3: Incident, failure & exit scenarios [ ] Does the stop procedure stop the work? Pause a sandbox job, revoke connector access and inspect queued tasks and destination logs. Establish which in-flight actions can still complete and how to contain them. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Can we recover from a wrong action? Practice identifying affected records, notifying the business owner and reversing a permitted test change. Define which actions cannot be reversed and therefore require stronger approval before execution. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Will model changes trigger revalidation? Agree notice, regression tests, rollout approval and rollback or suspension options for model, prompt, retrieval and connector changes. A version pin is useful only if the provider actually supports it. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ [ ] Can we leave without abandoning the evidence? Export approved prompts, configurations, evaluations and audit records in documented formats. Test what remains usable elsewhere, then exercise connector revocation and the agreed data-deletion process. Evidence score (0/1/2): ___ Test date: ___ Owner: ___ Evidence location / observed result: ____________________ Pass condition / approval blocker: ____________________ Gap / action / due date: ____________________ CHECK YOURSELF: If the vendor’s dashboard is unavailable, can we still disable its access at the systems it connects to? DECISION [ ] Approved for stated scope [ ] Conditional [ ] Blocked Decision owner and date: ____________________ Accepted exceptions and expiry: ____________________ Retest trigger and date: ____________________ NEXT ACTIONS Next 7 days: Inventory the AI vendors and connectors with access to sensitive information. Pick one proposed or active use case and document its allowed data, actions, owner and stop path. Next 30 days: Run the buyer-selected evaluation and stop-control exercise in a bounded test environment. Review data terms and access scope with security, procurement and the business owner together. Next 90 days: Make acceptance evidence, incident duties, change notice and exit rights part of the purchase or renewal. Schedule retesting when the model, connector, data class or business use changes.