You are thinking about a Bayesian analysis of your state of knowledge when you don’t know if you have cancer. But that’s not the best way of looking at it when only some outcomes lead to actions that result in cost or benefit. It’s better to consider the cost/benefit in the 4 cases.
If you do have cancer, then (a) not taking a test or (b) taking a test and getting a false negative both leave you in the same state of ignorance, so there is no cost. When you are looking at a cost-benefit analysis for the 4 possible scenarios, the cost-benefit of a false negative is zero (or it’s just the cost of taking the test).
Cost/benefit of 4 scenarios:
(1) Have cancer, positive test => benefit of early diagnosis
(2) Have cancer, negative test => zero
(3) No cancer, positive test => cost of unneccesary biopsy
(4) No cancer, negative test => zero
So when the cost of the test is small, only (1) and (3) are relevant, the true positives and the false positives.
An argument against simplifying by reducing (2) and (4) to approximately zero might be that there’s a psychological boost - happiness - from a negative test result. But if we’re talking about a routine screening where you have a high prior expectation of a negative result anyway, it seems unlikely that this is significant compared to the more tangible cost/benefit in (1) and (3).
Because the (in fact misleading) metric of sensitivity has been blasted so many times that the general public has been mistaught that sensitivity is THE number that matters and THE number to expect to hear. The nature of reporting emphasizes how bad it is to miss a case, to the exclusion of really ANY consideration of the costs incurred to detect an additional case. Missed cases are easy to sensationalize; the many getting unnecessary testing and/or treatment with pain, risks, and costs, not so clickable.
The intuitive play is to people wanting to know they don’t have cancer, and not even think about how often the test would given them a positive when they in fact do not have the condition, nor to think about the consequences of that.
It exploits the cognitive illusion being discussed.
Expanding the nature of screening beyond cancer, it depends. Just like in the OP’s video, we have to be clear what question we are interested in answering and why.
Sometimes the false negative rate is the result we do and should care most about. For example - Imagine you were considering visiting an elderly high risk relative during Covid’s peak. You did a test. Likely you were wanting to be sure you did not have Covid. The true negative rate was actually what mattered the most, and best would have been to be able to state, given local rates (and of course whether or not you had a known exposure and when), what the odds were that you still were infected despite a negative result.
Absolutely, and that’s a great example. In a cancer screening, it’s usually the case that only positive outcomes result in any action, so the cost/benefit of negatives is negligible. Conversely, in your example it’s only the negative result that leads to major consequences.
But it’s still notable in your example that it’s not the sensitivity that captures what we care about, although it’s an input into our calculation. True positive: little benefit if you’re young and healthy, since there’s little we would do to treat - that’s not why you’re taking the test. False positive: little cost if the the cost of sequestering is not that great. It’s the negatives that really matter. As you say, the ratio of false negatives to true negatives tells us the probability that we might kill grandma.
Nope, as I’ve explained. In a typical cancer screening situation the sensitivity per se is irrelevant, except as an input that allows us to calculate the rate of true positives vs false positives. The negative test outcomes have no consequences (aside from the cost of the test).
Well perhaps now you’re getting it. Because the 80% sensitivity in your preferred headline was just as irrelevant per se as the 100% sensitivity in the reductio ad absurdum.
Yes, and as Riemann said, it’s analogous to the headline that advertises an 80% detection rate. Even with a hint to read on for more details, it is misleading (IMO). In my example, any headline aside from “New cancer test does no better than chance” would be misleading. And for the original example, one that doesn’t in some way hint that even with a positive result, you very likely don’t have cancer, would also be misleading.
I don’t like a headline with any simplistic summary of a complex situation.
My preferred headline that references the 80% is only preferred over your version and the original that uses 7.5%. I would much prefer that none of the above were used.
Huh. In the context of this thread your conclusion is very interesting.
Let’s recap.
The video of the OP illustrated that information can be presented in ways that exploit our “intuition”, our heuristics, that result in cognitive illusions. These often lead us as a general population to have pervasive false beliefs.
You would like to have most of us better able to not have those false beliefs.
The last study I shared provided solid evidence that while greater math and reading literacy helps reduce susceptibility to these cognitive illusions how information is presented matters much.
The specific cognitive illusion involved in misunderstanding the meaning of sensitivity and specificity is one that even those with high math and reading literacy are subject to.
Standard media presentations, most headlines, exploit and reinforce that cognitive illusion to sometimes pernicious effect. Worries about having disease missed sells; more accurate complicated considerations regarding costs v benefits overall are not as appealing to read for most.
So we have two headline choices: one that continues the standard exploitation of the cognitive illusion, that facilitates the common intuition that avoiding missed cases is all that matters; and one that instead presents in a way that appeals to the intuitive conclusion less, that exploits the cognitive illusion less, that facilitates an understanding of both benefits and costs of screening. Which would be “sensationalist” because people have been entrained to getting information in the way that exploits the cognitive illusion, and, as in the video of the OP, are not well equipped for analyses of how many of one harm is worth how many of a different sort of harm.
And you prefer the former, because that exploitation is standard operating procedure.