Highly recommend this book. I would consider myself experienced with many statistical methods but this book was still chock full of brilliant examples that let me look at things with fresh eyes. It was helpful also in giving me language to explain technical concepts to less technical folks.
yeah. the "base rate fallacy" example sheds some light on the pros/cons of mammogram results. one interesting thing about that section of the book is that it says medical doctors fall prey to the fallacy more often than not.
i always kind of wonder when i see a medical doctor explaining statistics to a general audience: do they really have this right?
I also found this passage about mammographers from "Moonwalking with Einstein" interesting:
For most mammographers, practicing medicine is not deliberate practice, according to Ericsson. It’s more like putting into a tin cup than working with a coach. That’s because mammographers usually only find out if they missed a tumor months or years later, if at all, at which point they’ve probably forgotten the details of the case and can no longer learn from their successes and mistakes.
One field of medicine in which this is definitively not the case is surgery. Unlike mammographers, surgeons tend to get better with time. What makes surgeons different from mammographers, according to Ericsson, is that the outcome of most surgeries is usually immediately apparent—the patient either gets better or doesn’t—which means that surgeons are constantly receiving feedback on their performance.
Many of us in the software community know that if the feedback loop isn't fast enough, it doesn't work.
I wonder if you made mammographers spend a day a week analyzing mammograms from 5 years ago and then showing them the outcomes if they would get more accurate?
Well, that’s pretty much like training a neural network, except that the integration of the matrix coefficients happens not from the immediate output data, but from data which was output by the network millions of epochs ago.
Giving them the information could certainly help, and it is a good first step.
Mammographers would still be missing the motivation surgeons get when they see a patient die hours or days later. The emotional kick in the pants is harder to provide.
> i always kind of wonder when i see a medical doctor explaining statistics to a general audience: do they really have this right?
My wife is in her last year of medical school. I wonder the same question frequently when talking with her peers and seniors. They certainly get taught the basics of interpreting statistics, but I don't really see the "average" doc discussing things in a critical manner.
The system seems deeply focused on the results without critically thinking about the context and appropriateness of the studies.
I can answer that clearly for you. 99+% of docs don't have the slightest clue about how statistics are done, or even about what they mean (am MD with special interest in stats).
