Every AI vendor will tell you their platform is secure. The word does almost no work. What follows is the set of questions that produce different answers from different vendors, which is the only useful test of a claim.
Where the data physically is
Ask where documents are stored, where they are processed, and whether those are the same place. Ask which sub-processors see them in transit. Ask what happens when a model provider is in one jurisdiction and your regulator is in another.
The answer that matters is not a region name on a slide. It is whether you can choose: a private cloud in your own account, an isolated tenant, or your own data centre. If deployment is one-size, then the residency conversation is already over and you lost it.
What is sent, not just where it goes
There is a large difference between a system that sends a model your document and one that sends a model the paragraph that answers the question. Retrieval-based systems store and index your files, then pass only the relevant passages into the prompt. The rest of the document never leaves your storage.
This is worth confirming rather than assuming, because it changes the exposure of every third-party model call from the whole file to a few hundred words.
Training, in writing
"We don't train on your data" should appear in the contract, not the marketing site, and it should cover the vendor and every model provider they route to. A platform that orchestrates models rather than building them is in a structurally better position here — it has no commercial interest in your corpus — but the position needs to be written down.
Ask for the clause. A vendor who cannot show you where the commitment lives in the agreement is describing an intention.
Identity, and what happens when someone leaves
Single sign-on is table stakes; the real question is deprovisioning. When someone leaves on Friday, is their access gone on Friday, and does that include the chat history they created, the files they uploaded, and any API credential issued in their name? Systems that treat AI access as a separate account list rather than a role in your directory tend to fail this quietly for months.
The audit trail, and who can read it
You want a record of what was asked, which model answered, and which documents were retrieved. You also want to know who can read that record, because a full transcript log is itself sensitive: it contains, in plain text, whatever anyone typed.
- Is the log complete, or sampled?
- How long is it retained, and can you set that?
- Who inside the vendor can read it, and under what process?
- Can you export it into the tooling your security team already uses?
Controls that match how work is organised
Departments have different appetites. Legal may want a restricted model list; engineering may want everything. If the platform's permission model is a single global setting, you will end up governing to the strictest team and frustrating everyone else, or to the loosest and satisfying no auditor.
The two questions people forget
First: what does exit look like? Can you export your documents, your prompt library and your history in a usable format, without a professional services engagement? A platform that is hard to leave is a platform whose pricing will eventually reflect that.
Second: what happens when a model is retired? Providers deprecate models on their own schedule. If your workflows are pinned to a specific model name, that is a migration you did not plan. Ask how the vendor absorbs it.
Who answers when it is wrong
Every AI platform will eventually produce an output that is confidently incorrect and reaches a customer. The question worth asking during procurement is procedural, not technical: when that happens, what can you reconstruct?
You want to be able to answer four things. What was asked. What material was retrieved. Which model produced the answer. Who then approved it. A platform that can supply the first three turns an incident into an investigation. A platform that cannot turns it into a conversation about whether AI should have been used at all, which is a much worse conversation to have in front of a regulator.
Cost, which is a security question too
Spend control gets filed under finance and belongs here as well. Uncapped per-seat usage against frontier models is how a pilot becomes an unbudgeted line item, and the usual response — restrict who has access — narrows adoption exactly where you wanted it broad.
Ask whether the platform can show usage per department and per user, whether quotas can be set, and whether a cheaper model can serve the routine tasks while the expensive one is reserved for work that leaves the building. A system that treats all queries as equivalent will be expensive in proportion to how successful your rollout is.
Certification is evidence, not a synonym
A SOC 2 Type II report or an ISO 27001 certificate is a useful signal: someone independent looked. It is not the same as the answers above, and it does not describe how your documents move through the system. Ask for the report, read the scope section, and then ask the nine questions anyway.