What a Site Visit Taught Me About AI

Grant monitoring wasn't just paperwork. The most revealing part was the site visit — when monitors showed up, walked around, and talked to the staff doing the actual work. The reports always looked fine. The conversations revealed what was really happening: the workaround nobody documented, the step everyone skipped, the problem everyone knew about but never wrote down.

AI oversight needs site visits.

Not literally — but the habit. Don't just review the dashboard and the accuracy metrics. Talk to the people who use the system every day. Ask what they actually do with it, what they don't trust about it, what workarounds they've invented. The frontline users know where the AI helps, where it hallucinates, and where they've quietly stopped checking its work.

Every monitoring visit I joined taught me the same lesson: the truth lives with the people doing the work, not in the reports about the work.

Schedule your own version. Ask the users. Listen like a monitor. The system you think you deployed and the system people actually use are often two different things.

I explain frameworks; I don't give legal counsel.