Don't Buy the Demo. Buy the Documentation.
The demo is theater. It's the vendor's best prompts, best data, best day, performed by their best presenter. It tells you almost nothing about what the tool does on your worst Tuesday.
The documentation tells you the truth. So before you buy, ask for the paperwork: How was the model evaluated, and on what? What are its known limitations? Where does customer data go, and how long does it stay? What security testing has been done? What happens when the model is updated?
Good vendors have answers ready, because serious customers always ask. They'll hand you model documentation, security whitepapers, data flow diagrams, sometimes independent audit reports. It won't all be perfect, but it will be honest.
Vendors with nothing to show will stall, deflect, or send marketing material relabeled as documentation. That reaction is itself an answer — just not the one they intended.
This habit comes straight from my federal grants days: never fund the pitch, fund the paperwork. The proposal is a story. The budget narrative, the audit trail, the supporting docs — that's the reality. AI vendors work the same way.
I explain frameworks; I don't give legal counsel.
Related consulting: Business systems diagnostics · All services
I explain frameworks; I don't give legal counsel.
