Availability Bias After a Dramatic Callback

Why this matters

One flooded basement, one furious customer, one insurance claim, and the owner can think of nothing else for a week. That is human, and it is also when the most expensive bad decisions get made: a control bolted onto every job to prevent something that happens to a fraction of one percent of them, a tech written off over a single bad day, a whole category of work quietly avoided. The dramatic case feels like the typical case because it is the one you can see when you close your eyes. This card is about how to weigh that vivid memory against the record, so the response fits the actual risk.

What the availability heuristic is

Amos Tversky and Daniel Kahneman described it in 1973: people judge how often something happens by how easily examples come to mind. Most of the time that works, because frequent things are usually easy to recall. It fails when something is easy to recall for reasons other than frequency.

The things that make an event easy to recall in a shop:

• It was recent. Last Tuesday beats last March.
• It was vivid. Water on a finished floor, a scorched panel, a customer shouting on the phone.
• It was expensive or embarrassing. A claim, a refund, a bad review with your name in it.
• You were personally involved. You took the call or drove out at midnight.
• It was retold. Every time it comes up in the shop it gets easier to recall.

A related study by Sarah Lichtenstein and colleagues in 1978 found people badly overestimated deaths from dramatic, widely reported causes and underestimated quiet common ones. The pattern is the same at a smaller scale: dramatic shop failures get overcounted, dull ones undercounted.

What a dramatic callback distorts

The obvious effect is on policy, and making a rule after one bad job is its own card. See related: We Made a Rule After One Bad Job. The less obvious effects are on the estimates that feed every later decision:

• How often this failure happens. One flood and the owner starts believing water heater installs leak "all the time."
• Who causes it. The tech on that job becomes, in memory, the leak tech, and his other 80 clean installs fade.
• Which jobs are risky. The owner starts pricing in a risk premium or steering away from a job type that has a fine record.
• Which customers are trouble. One angry customer from a neighborhood becomes "those customers."

Each of those is a frequency estimate, and each one is now being made from one data point that happened to be loud.

The memory and the record, side by side

Here is what that gap looks like on paper, for an 11-person plumbing shop the week after a water heater install leaked overnight into a finished basement:

WHAT THE OWNER REMEMBERS
  Water heater, finished basement
  Fitting let go overnight, carpet and drywall out
  Insurance claim, angry call, two-star review

WHAT THE JOB RECORD SAYS (last 18 months)
  Water heater installs ............ 412
  Leak callbacks ...................   3
    compression fitting, new batch   1  (damage)
    T&P discharge, high pressure     1  (no damage)
    drain valve weep                 1  (no damage)
  Installs with no callback ........ 409

The memory has one case in it. The record has 412, three of which leaked, for three different reasons, and only one of which did any damage.

Four questions that weigh the vivid case properly

1. How often, counted? Pull the numerator and the denominator from the records before forming a view. Three leaks in 412 installs is about 0.7 percent, roughly one install in 137. That is the frequency, not the feeling.

2. How bad is it when it happens? Severity is a legitimate reason to act on a rare event, so this question is not a way of dismissing the incident. A failure that is rare and cheap deserves little; a failure that is rare and catastrophic can justify a control even at very low frequency.

3. Is it new information? This is the question owners skip. Did the incident reveal a mechanism you did not know about, or did a known, already-managed risk simply land on someone this time? A new failure mode is real news and deserves a response aimed at it. A known risk landing at its usual rate is not news, however loud it is.

4. What does the proposed response cost across every job it touches? A control applied to all jobs costs its time on every one of them, including the 409 that were fine. Price it over a year.

Worked example: the flooded basement

The owner's first reaction. Every water heater install now gets a second tech to come by and sign off, plus a 30-minute leak watch with photos before the installer leaves.

Question 1, the count. 412 installs in 18 months, 3 leak callbacks, 1 with damage. That is about 275 installs a year (412 divided by 1.5), and about 2 leak callbacks a year.

Question 2, the severity. The damaged case was costly: an insurance claim, a review and a lost customer. Water damage in a finished space is serious, but it is property damage, not a life-safety event. Severity justifies a response; it does not by itself justify one of any size.

Question 3, new information. The tech pulled the failed fitting. Its threads were poorly cut, and it came from a box opened the week before, from a new supplier. That is a new mechanism. The other two leaks were a relief valve discharging on high street pressure and a drain valve weeping, each a different cause the shop already knew how to prevent. So the dramatic case does not represent the three; it is its own story.

Question 4, the cost of the first reaction. At about 275 installs a year:

• 30-minute leak watch: about 137 hours a year (275 times 0.5)
• second-tech sign-off at about 45 minutes each including the drive: about 206 hours a year (275 times 0.75)
• together about 343 hours a year, roughly eight and a half weeks of one tech's full-time hours

That is a large standing cost aimed at a failure that occurs about twice a year, and most of it lands on installs that were never going to leak.

The response that fits. The shop pulls the remaining fittings from that supplier's batch and returns them. It adds a short, specific check aimed at the actual mechanism: before leaving, the installer runs the heater up to pressure, wipes every new water joint dry, holds a dry paper towel at each for a minute and photographs it on the ticket. (Gas connections on a gas heater keep their own leak test with a leak-detection solution or a gas detector; a paper towel tells you nothing about gas.) That takes about 10 minutes per install, about 46 hours a year (275 times 10 minutes, divided by 60). For the relief valve case, the install checklist gets a street-pressure reading, with a pressure-reducing valve quoted when it reads high.

So the response went from about 343 hours a year aimed at everything to about 46 hours aimed at the mechanism, plus a one-time supplier fix. It took the incident seriously and sized the answer to the record.

When one case should change everything

The four questions can say "act hard on a single case," and they should when:

• The failure is a life-safety event. A gas leak after an appliance connection, carbon monoxide from a venting error, an energized enclosure. Here severity dominates frequency, because the cost of the rare event has no ceiling. One case justifies a protocol change across every job of that kind, and you make it the same day.
• The record shows a trend. Three leaks in 18 months is a rate; three leaks in the last six weeks after none for a year is a change. A clustered, recent run is evidence something shifted (a new supplier, a new tech, a new code requirement), and the response should find what.
• The incident exposed a hole in your controls rather than a one-off. If the investigation finds nobody was checking joints at all, the dramatic case did reveal a systemic gap, and a broad response is right.

The opposite error: the risks nobody remembers

Availability also works in reverse. Risks that never produce a dramatic moment stay underweighted however large they are:

• underpriced jobs that each lose a little, adding up to a bad quarter
• an estimate follow-up that nobody does, costing a share of close rate every week
• receivables quietly aging past 90 days
• a tech whose callbacks are slightly high every month, never dramatic

None of these will keep you awake, and together they usually cost more than the flooded basement. The cure is the same as for the vivid case: a count, on a schedule, that does not depend on what you happen to remember.

Habits that keep the record in charge

• Contain the hazard immediately; change policy after 48 hours. Anything touching safety is acted on at once. Shop-wide policy changes wait two days and are made with the counts on the table.
• Pull the denominator before the meeting. Whoever brings an incident to the team also brings the job count for that type of work over the last year.
• Review the quiet record monthly. Callback rate, margin by job type and aging of receivables, looked at whether or not anything dramatic happened.
• Separate the incident from the person. Pull the tech's full record before deciding what one job says about him.

References

• Tversky, A., and Kahneman, D. (1973), "Availability: A Heuristic for Judging Frequency and Probability," Cognitive Psychology
• Lichtenstein, S., Slovic, P., Fischhoff, B., Layman, M., and Combs, B. (1978), "Judged Frequency of Lethal Events," Journal of Experimental Psychology: Human Learning and Memory
• See related: We Made a Rule After One Bad Job; Hindsight Bias in the Callback Review; Base Rates: The Number You Should Start From; The Fundamental Attribution Error With Customers and Crew