Ten questions to ask any AI vendor before you sign

Every AI vendor's demo is good; that is what demos are for. What separates a vendor you can trust with member or client data from one you cannot is how they answer ten plain questions in writing. Send them before the second call. The answers, and the speed of them, tell you most of what you need to know.

The ten

1. Is our data used to train or improve your models, or anyone else's? The only acceptable answer is no, in the contract, not the FAQ. 2. Where is our data stored and processed, and who are your subprocessors? You want a list, not a link. 3. How long do you retain prompts, uploads and outputs, and can we set that to zero? 4. What do you log, and can we export the log? If you cannot read what the tool did, you cannot govern it. 5. When your product informs a decision about a person, what explanation will you give that person? From January 2027 this is a legal question in Colorado and California. 6. When was your last bias test, who ran it, and can we see the summary? 7. What is the human-review path inside the product, and how is it recorded? 8. What happens to our data, our prompts and our configuration if we leave? You want export in a usable format and deletion in writing. 9. How will you notify us of a security incident, and within what time? 10. What in your pricing changes with usage, and what did it cost your customers last year?

How to read the answers

Short, specific, contractual answers are a good sign. Long answers that restate the marketing are not. A vendor who says “we take security seriously” instead of naming a retention period has told you the retention period is not zero. A vendor who cannot name subprocessors has not looked. A vendor who will not put the training answer in the contract should not hold your data.

What we add for unions and funds

For a benefit fund, add: does the product touch protected health information, and is the vendor willing to sign a business associate agreement? For a local or a training center, add: can the tool be deployed inside our environment, and can we turn training material into an assistant without it leaving? For anything that touches bargaining-unit members, add: what notice and review language will the vendor support so that we can meet our own commitments?

We run this list as a written questionnaire with a scorecard in every Privacy & Security Review. Use it yourself first; it is meant to be given away.