Leo Breiman is my hero. And I have a confession: I had already seen Rashomon, Kurosawa’s 1950 film, because my partner made me watch it one evening, long before I discovered that Breiman had borrowed its name. Had I known, I would have paid much more attention.
In the film, four witnesses tell the same story in four different ways, and all of them sound believable. In 2001 Breiman borrowed the name for something that happens with data: different models can be equally good and still disagree on what matters.
I tried it with a simple simulation. Imagine two brain measures that are almost identical, like the volume of the left and right hippocampus in Alzheimer’s disease. I built 100 models on slightly different samples of the same patients. They were all equally good, right about 72% of the time. But 29 said the left hippocampus mattered most, and 71 said the right one.
With only one model, I could have written a paper about the left hippocampus. Or about the right one. By chance.
The pill: when a model “explains” a disease, ask whether other equally good models would tell the same story. One explanation is one witness, not the truth.
Leave a Reply