AI quality bugs

“The AI recommended stopping exactly the thing the data says is still needed.”

An assistant produces an actionable recommendation that a datum in the same record directly contradicts. The output is confident and well-formed; the danger is that it reads like a conclusion when it never consulted the disqualifying value.

AI recommendation contradicts the recordblack beltSafety testing of AI recommendations against ground truth

Choose your app

The same mistake, planted in different products. Pick the domain you want to hunt it in.

QualityChart

St. Quality Health · Healthcare · EHR

qualitychart.stquality.test

1 planted defectaccounthunt here →

TapTrust Federal

TapTrust Federal Credit Union · Banking · Online + teller

TapTrust Federal doesn't run an AI surface (an assistant/copilot over the app's own data) yet — contribute the infra surface and this problem becomes seedable here.

QualityMart

QualityMart Inc. · Retail · Big-box e-commerce

QualityMart doesn't run an AI surface (an assistant/copilot over the app's own data) yet — contribute the infra surface and this problem becomes seedable here.