Questions to ask before buying.
ABOUT CAPABILITY
What specifically does it do? What was it tested on, and by whom? What does it get wrong, and how often? What happens when it is uncertain?
ABOUT YOUR CONTEXT
Was it validated on data like yours? Does it cover your jurisdiction? Does it handle your terminology?
THAT SECOND QUESTION
A tool validated elsewhere may perform differently here. Ask.
ABOUT ACCOUNTABILITY
Who is responsible when it is wrong? What does the contract say about liability? Does using it satisfy your regulator?
ABOUT DATA
Where does our data go? Is it retained? Used for training? Does this satisfy our confidentiality obligations?
ABOUT FAILURE
How would we know it had gone wrong? How do we stop it? What happens if the provider ceases?
THE MOST USEFUL REQUEST
"Show me where it fails."
A vendor who can answer understands their product.
BEFORE COMMITTING
Test on your own work, including difficult cases.
