Reliable Error · notes on psychology

Judgment under uncertainty

Most judgment errors are substitutions. A question that is hard to answer gets quietly replaced by one that is easy, the easy question is answered competently, and the answer is returned with the confidence the hard question deserved.

Framed that way, the standard list of biases stops looking like a catalogue of separate defects and starts looking like one mechanism with several faces. The substitutions are efficient — they are right often enough to be worth making — which is precisely why they are not experienced as shortcuts.

HeuristicHard questionEasy question substituted
Availability How common is this? How easily do examples come to mind?
Representativeness How likely is this category? How well does this case match the stereotype?
Anchoring What is the right number? How far should I move from the number I was just given?
Affect What are the risks and benefits? How do I feel about it?

Anchoring survives knowing about it

Present people with an arbitrary number before asking for an estimate, and the estimate moves toward the number. This holds when participants know the anchor was generated at random, when they are warned about the effect, and when they are paid for accuracy. It shrinks under those conditions but does not disappear.

The practical consequence is not "be aware of anchoring" — awareness is close to useless as a defence. It is procedural: produce your own estimate before hearing anyone else's. In a group, that means independent written estimates before discussion, because the first number spoken becomes everyone's anchor, and a group that has converged cannot un-converge.

Base rates, worked through

Base-rate neglect is the most consequential item on the list, because the errors it produces are not small — they are frequently off by an order of magnitude. It is worth doing once with numbers rather than in the abstract.

Take a condition present in 1% of a population, and a test that correctly flags 90% of those who have it while also flagging 9% of those who do not. Someone tests positive. How likely is it that they have the condition?

The intuitive answer clusters around 90%, because that is the number the problem made salient. The correct answer requires counting a thousand people:

Have itDo notTotal
In 1,000 people109901,000
Test positive98998
Test negative1901902
9 truly have it 89 do not 98 positive results Chance a positive result is correct: about 9%
The 90% figure describes the test's behaviour among people who have the condition. The question asked about people who tested positive — a different group, dominated by the 990 who were always going to supply most of the false alarms.

Nine of the 98 positives are correct. The answer is roughly 9%, not 90%. The test is not bad; the base rate is simply low enough that false alarms drawn from a large healthy majority outnumber true findings by ten to one.

The general form. When a condition is rare, a positive result from a good test still usually means the condition is absent. This is not a curiosity of medical screening — it applies to fraud detection, security alerting, hiring filters, and any monitoring system watching for something uncommon. If you have ever wondered why an alerting system that is "99% accurate" produces mostly false alarms, this is the entire reason.

One reliable fix: express the problem in natural frequencies — counts out of a concrete population — rather than percentages or conditional probabilities. Performance on problems of this type improves markedly under that framing, for experts as well as for novices.

Calibration, and why it needs feedback

Being well calibrated means your confidence matches your hit rate: of the things you are 70% sure about, roughly 70% turn out to be true. Most people are overconfident, and the overconfidence is largest exactly where it is most costly — on hard questions, in unfamiliar domains, and on long-range predictions.

Where calibration is good, it is almost always because the domain supplies fast, unambiguous feedback. Weather forecasters are the standard example of a well-calibrated group, and they are well calibrated because they make explicit probabilistic forecasts and find out the answer the next day, thousands of times over a career. Experience without that loop does not produce calibration; it mostly produces confidence.


About these notes →