AI explanations actually improve human decision-making and found a surprising result: they can sometimes make it worse. Researchers from Harvard Business School, MIT, and the University of Washington studied how people evaluated early-stage innovation proposals with the help of AI recommendations. They found that evaluators were sometimes persuaded to reject promising ideas that independent human experts considered strong, while also accepting weak ideas when the AI recommended them.
The most striking finding was that adding an explanation to an AI recommendation increased the tendency to defer to the machine. When the AI provided a narrative rationale for its decision, people were more likely to follow an incorrect recommendation than when they were shown the recommendation without an explanation. Instead of encouraging critical thinking, the explanation appeared to give the AI's judgment an additional layer of credibility, making people less willing to challenge it.
This creates a particular problem for innovation screening, where decisions involve uncertainty and there is no immediately verifiable right answer. Companies routinely have to decide which projects to fund, develop or abandon, meaning both false positives and false negatives are costly. The researchers point to examples such as Google Glass and Amazon's Fire Phone as failed projects that illustrate the risk of pursuing bad ideas, while prematurely abandoning potentially valuable technologies can be equally damaging. AI that confidently reinforces either decision could therefore amplify human errors rather than eliminate them.
The broader lesson is that explainable AI is not automatically better AI. An explanation can make a system easier to understand, but it can also make humans overly confident in an incorrect recommendation. Effective human-AI collaboration may therefore require deliberately preserving independent human judgment, such as asking people to form their own assessment before seeing the AI recommendation or using AI as a source of competing evidence rather than as an authority. The goal should not be to make humans agree with AI more efficiently, but to make the combined human-AI decision better than either one working alone.
