How to Evaluate AI Vendors on Data Privacy (with a Scorecard)

Evaluating AI vendors on data privacy requires going beyond marketing claims and compliance badges. You need to verify where your data is stored, whether it trains models, who can access it, and what happens when something goes wrong. This guide provides a weighted scorecard with specific questions, red flags, and green flags so your team can make defensible vendor decisions.

Why Most AI Vendor Privacy Evaluations Fall Short

Most organizations evaluate AI vendors the same way they evaluate any SaaS tool: check for a SOC 2 badge, confirm the vendor will sign a BAA, and move on. That approach was already insufficient for traditional software. For AI, it is dangerously inadequate.

The reason is straightforward. AI systems introduce privacy risks that conventional software does not. Your data may be used to train models that serve other customers. Prompts and outputs may be logged and retained indefinitely. Third-party model providers may process your data without your knowledge. And the vendor's privacy policy may grant permissions that contradict what their sales team told you.

The numbers back this up. According to IBM's 2025 Cost of a Data Breach Report, the average healthcare data breach now costs $7.42 million. The Verizon 2025 Data Breach Investigations Report found that 30% of all data breaches involved third-party vendors, double the 15% reported the prior year. Meanwhile, IBM found that only 35% of healthcare organizations can even track their AI usage, meaning the majority have no visibility into where sensitive data flows once it enters an AI system.

A SOC 2 badge on a vendor's website is not a privacy strategy. It is the starting point.

10 Questions Every Organization Should Ask AI Vendors

Before you deploy the scorecard below, use these questions in your initial vendor conversations. The answers (or lack thereof) will tell you more than any data sheet.

1. Where is my data stored, and can I choose the region? - Acceptable answers name specific cloud regions. Vague answers like "our secure cloud infrastructure" are not sufficient.
2. Is any of my data - including prompts, outputs, or metadata - used to train or fine-tune models? - This must be a clear, contractual "no" with no exceptions, not just a toggle in settings.
3. Who at your company can access my data, and under what conditions? - Look for named roles, documented access policies, and audit trails. "Our engineers may access data for troubleshooting" is a red flag.
4. Do you use any third-party model providers or subprocessors? - If yes, you need the full list and confirmation that your data protections extend to every link in the chain.
5. What is your breach notification timeline, and is it contractual? - HIPAA requires notification within 60 days, but best practice is 72 hours or less. If the timeline is only in a privacy policy (not the contract), it can change without notice.
6. Can I get a SOC 2 Type II report, and when was the last audit period? - Type I is a snapshot. Type II covers operational effectiveness over 6-12 months. Insist on Type II.
7. Will you sign a Business Associate Agreement? - For healthcare, this is non-negotiable. But also ask what the BAA actually covers - does it include the AI processing layer, or just the storage layer?
8. What happens to my data when I terminate the contract? - You need a contractual commitment to deletion within a defined window, with certification that deletion is complete, including backups.
9. Do you maintain immutable audit logs of all data access and model interactions? - For regulated industries, audit logs are not optional. They need to be tamper-evident and available to your compliance team.
10. Can I deploy this on my own infrastructure? - Self-hosted or on-premises deployment eliminates entire categories of third-party risk. Not every vendor offers it, but it is the strongest privacy posture available.

The AI Vendor Privacy Scorecard

Use this scorecard during vendor evaluation. Score each category on a 1-5 scale based on the green and red flags listed. Weight the final score by priority. A vendor that scores well on low-weight items but poorly on high-weight items should not pass evaluation.

Category	Key Question	Red Flags	Green Flags	Weight
Data Residency	Where is data stored and processed? Can I choose the region?	No region selection; data may cross borders; vague "global infrastructure" language	Named regions (e.g., US-East, EU-West); contractual data residency guarantees; single-tenant or on-prem option	High
Training Data Usage	Is my data used to train, fine-tune, or improve models?	Opt-out (not opt-in) model training; vague "improve our services" language; no contractual prohibition	Contractual zero-training guarantee; no data retention beyond session; independent audit verification	High
Access Controls	Who can access my data internally, and how is access governed?	Broad internal access; no role-based access control (RBAC); no access logging; "engineers may access for support"	Named-role RBAC; just-in-time access with approval workflows; all access logged and auditable	High
Breach Notification	What is the notification timeline, and is it in the contract?	No defined timeline; notification only "as required by law"; timeline buried in privacy policy (changeable without notice)	72-hour contractual notification; named point of contact; incident response plan shared during evaluation	High
Compliance Certifications	What certifications do you hold, and can I review the reports?	SOC 2 Type I only; expired or unavailable reports; certifications that exclude AI processing components	SOC 2 Type II (current); HITRUST; ISO 27001; reports available under NDA; audit scope explicitly covers AI workloads	High
Subprocessor Disclosure	Do you use third-party model providers or data processors?	No subprocessor list available; "we may use third-party providers"; no notification of subprocessor changes	Published subprocessor list; advance notice of changes; contractual right to object; DPA flows down to all subprocessors	Medium
Deletion Rights	What happens to my data on contract termination?	No deletion guarantee; "data retained per our policies"; no timeline for deletion; backups excluded from deletion	Contractual deletion within 30 days; deletion certification provided; includes all backups and derived data	Medium
Audit Logs	Are all data access events and model interactions logged?	No customer-accessible logs; logs retained less than 12 months; no tamper-evidence	Immutable, customer-accessible audit logs; 12+ month retention; real-time export to your SIEM	Medium
Encryption	Is data encrypted at rest and in transit? Who holds the keys?	Vendor-managed keys only; no encryption at rest; TLS 1.2 only (not 1.3)	AES-256 at rest; TLS 1.3 in transit; customer-managed keys (BYOK/HYOK); envelope encryption	Medium
Deployment Model	Can I run this on my own infrastructure?	SaaS-only with no isolation; shared tenancy with no segmentation; no VPC or private link options	Self-hosted/on-prem option; single-tenant cloud; VPC peering or private endpoints; air-gapped deployment supported	High

How to Read Vendor Privacy Policies

Privacy policies are legal documents written to protect the vendor, not you. Here is what to look for and what language should trigger deeper scrutiny.

Dangerous Wording

• "We may use your data to improve our services" - This is the single most common way vendors grant themselves permission to train on your data. If this phrase appears anywhere in the policy, your data is likely being used for model improvement unless you can get a contractual carve-out.
• "Aggregated and anonymized data" - Vendors often claim they only use "anonymized" data for training or analytics. But research has repeatedly shown that anonymization of AI training data is difficult to verify and easy to reverse, especially with large language models that can memorize patterns.
• "We may share data with our affiliates and service providers" - This can mean your data flows to entities you have never evaluated. Ask for the full list.
• "This policy may be updated at any time" - If the privacy policy can change without notice and your contract incorporates the policy by reference, the vendor can unilaterally weaken your privacy protections.

What to Demand Instead

• A Data Processing Agreement (DPA) that supersedes the privacy policy
• Contractual language that explicitly prohibits training on customer data
• A fixed subprocessor list with advance notice of changes
• Breach notification terms in the contract itself, not just the privacy policy

SOC 2 Type I vs. Type II: What It Means for AI Vendors

SOC 2 is the most commonly cited compliance framework for SaaS and AI vendors. But not all SOC 2 reports are equal, and the distinction matters significantly for AI procurement.

SOC 2 Type I evaluates whether a vendor has designed appropriate security controls at a single point in time. It is a snapshot. It confirms that the right policies exist on paper as of the audit date.

SOC 2 Type II evaluates whether those controls actually work over a sustained period, typically 6 to 12 months. It tests operational effectiveness, not just design. An auditor reviews evidence that controls were consistently applied during the audit window.

For AI vendors, this distinction is critical. An AI vendor with a Type I report has demonstrated that their security controls looked good on one specific day. A Type I report does not tell you whether the vendor consistently encrypts data, restricts access, or monitors for anomalies over time.

When evaluating AI vendors, insist on a current SOC 2 Type II report and verify three things:

1. The audit period is recent (within the last 12 months)
2. The scope of the audit covers the AI processing environment, not just the billing or marketing infrastructure
3. Any exceptions or qualified opinions in the report are reviewed by your security team

BAA Availability: What It Covers vs. What It Does Not

For healthcare organizations, a Business Associate Agreement is a legal requirement under HIPAA whenever a vendor processes protected health information (PHI). But signing a BAA does not automatically make an AI vendor safe to use.

What a BAA typically covers:

• The vendor's obligation to safeguard PHI
• Permitted uses and disclosures of PHI
• Breach notification requirements
• Requirements to return or destroy PHI on termination

What a BAA typically does not cover:

• Whether PHI is used in model training (unless explicitly addressed)
• Subprocessor handling of PHI - many BAAs only bind the primary vendor
• Prompt and output logging - if the vendor logs AI interactions containing PHI, this may fall outside the BAA's scope
• De-identified or "anonymized" data derived from PHI, which vendors may argue falls outside HIPAA protections entirely

When negotiating a BAA with an AI vendor, push for explicit language covering the AI inference layer, all third-party model providers in the chain, and the handling of any data derived from PHI during processing.

Red Flags That Should Immediately Disqualify a Vendor

Some findings during evaluation should end the conversation. These are not areas for negotiation or "roadmap" promises.

1. Refusal to provide a SOC 2 Type II report - If a vendor cannot produce a current Type II report and their product handles sensitive data, the evaluation is over.
2. No contractual prohibition on training with customer data - Verbal assurances are worthless. If the contract or DPA does not explicitly prohibit it, assume your data trains their models.
3. Inability to name their subprocessors - If a vendor cannot tell you exactly who processes your data, they either do not know or do not want you to know. Neither is acceptable.
4. No BAA availability for healthcare use cases - Any AI vendor targeting healthcare that will not sign a BAA is either not HIPAA-compliant or not serious about the market.
5. Privacy policy grants unilateral right to change terms - If the vendor can change how your data is used without your consent, you have no meaningful privacy protections.
6. "We take security seriously" with no specifics - This phrase, without supporting documentation, certifications, and technical controls, is the hallmark of security theater.
7. No customer-accessible audit logs - In regulated industries, the inability to independently verify who accessed your data and when is a compliance failure.

How Self-Hosted AI Changes the Equation

Many of the scorecard categories above exist because your data leaves your control when you use a cloud-hosted AI vendor. Self-hosted AI platforms fundamentally change this calculus.

When the AI runs on your infrastructure, data residency is determined by you. Training data usage is a non-issue because the models run locally. Access controls integrate with your existing identity provider. Subprocessor risk drops to zero for the inference layer.

This is the approach taken by platforms like Compass AI, which deploys entirely within an organization's own environment. On the scorecard above, a self-hosted deployment inherently scores at the top of every high-weight category - not because the vendor claims to protect your data, but because your data never leaves your perimeter. It is not the right fit for every organization (you need the infrastructure and team to manage it), but for regulated industries where data sovereignty is non-negotiable, it eliminates the hardest vendor risk questions entirely.

Whether you choose a cloud vendor or a self-hosted solution, the scorecard applies. The difference is that self-hosted deployments shift accountability from "trust the vendor" to "verify your own controls" - a trade-off most CISOs in healthcare, legal, and financial services are happy to make.

Frequently Asked Questions

What is the most important question to ask an AI vendor about data privacy?

Ask whether your data - including prompts, outputs, and metadata - is used to train or improve models. This is the single highest-risk area because it is often hidden in privacy policy language like "improve our services." Demand a contractual prohibition, not just a verbal assurance or settings toggle.

Is SOC 2 certification enough to trust an AI vendor with sensitive data?

No. SOC 2 Type I only confirms controls exist on paper at a point in time. You need a SOC 2 Type II report covering at least six months of operational effectiveness. Even then, verify the audit scope includes the AI processing layer, not just ancillary systems like billing or marketing infrastructure.

Does a BAA make an AI vendor HIPAA-compliant?

A BAA is necessary but not sufficient. It establishes contractual obligations for handling PHI, but it does not guarantee the vendor's technical controls are adequate. Most BAAs also do not cover AI-specific risks like model training on PHI, prompt logging, or subprocessor handling of data during inference.

How can I verify an AI vendor is not training on my data?

Require a contractual zero-training clause in the DPA or master agreement. Review the privacy policy for language like "improve our services" or "aggregated data" provisions. Ask for the SOC 2 Type II report and look for controls around data isolation. For maximum assurance, choose a self-hosted solution where models run on your own infrastructure.

What is the average cost of a data breach involving an AI vendor?

IBM's 2025 Cost of a Data Breach Report found the global average breach cost is $4.44 million, with healthcare breaches averaging $7.42 million. Breaches involving shadow AI - unauthorized or untracked AI tools - added an average of $670,000 in additional costs. Third-party vendor involvement, including AI vendors, contributed to 30% of all breaches according to Verizon's 2025 DBIR.