Outcome Bias: Judging a Decision by How It Turned Out

Why this matters

A shop owner who skips the third truck and then gets the hottest summer in a decade will hear about it all season: from customers who waited, from techs who worked six-day weeks, and mostly from themselves. A shop owner who takes a big job on a handshake and gets paid on time will tell the story at the supply house as proof of good instincts. Both are judging the decision by the result. Sometimes that is fair. Often it teaches exactly the wrong lesson, because good decisions lose some of the time and bad ones win some of the time. Learn from outcomes alone and you will abandon sound habits after one bad break and repeat reckless ones after one lucky one. This card gives you a short gate for separating the two, and runs two real-shaped decisions through it.

The term

Outcome bias is judging the quality of a decision by its result rather than by what was known and how it was reasoned at the time. The psychologists Jonathan Baron and John Hershey demonstrated it in 1988: people rated the identical decision, made with identical information, as better or worse depending only on whether they were told it worked out. The professional poker player and writer Annie Duke later gave it a shop-floor name, resulting, in her book Thinking in Bets.

It sits next to two neighbors worth keeping separate:

• Hindsight bias is believing, after the fact, that the outcome was more predictable than it was ("I knew that builder was trouble"). It corrupts your memory of what you knew. See related: Hindsight Bias in the Callback Review.
• Regression to the mean explains why an extreme result tends to be followed by a less extreme one, which makes an unusual outcome a poor guide to the next one. See related: Regression to the Mean and the Tech Who Had a Great Month.

Outcome bias is the judgment step: deciding the call was good or bad because of how it landed.

The four boxes

Every decision lands in one of four places, and only two of them teach the lesson the outcome seems to teach:

The trap is the diagonal. The bottom-left box feels exactly like the top-left, and the top-right feels exactly like the bottom-right. Without a deliberate review, you file the lucky win as skill and the bad break as a mistake.

Why small shops are more exposed

Small numbers. A shop with 40 large jobs a year cannot tell skill from luck on any one of them. One commercial job going bad can be the story of the whole year.

Single big decisions. Most of an owner's important calls (a truck, a hire, a location, a contract) happen once or twice. There is no long run to average out the luck, so each outcome feels like a verdict.

The story gets told. A win becomes a supply-house story and a loss becomes a cautionary tale for the crew. Both harden into shop policy faster than anyone checks whether the decision behind them was sound. See related: We Made a Rule After One Bad Job.

The review gate: four questions

Run any decision worth reviewing through these, in this order, writing the answer to each before moving to the next.

1. What did we actually know when we decided? Not what you know now. Go to whatever was written at the time: the journal entry, the estimate, the texts. If nothing was written, say so, and treat your memory as contaminated by the outcome.
2. What could we have learned at a cost that fit the decision? A two-call reference check before handing a third of your capacity to a new builder fits. A consultant's market study before buying one truck does not. A gap here is a process failure even when the outcome was good.
3. Was the reasoning sound given what we knew? Did the numbers support the call, were the risks named, was the downside survivable?
4. Was the outcome inside the range we would have expected? If you had made this exact decision ten times, how many would have gone like this one? An outcome that lands in the tail of a range you anticipated is luck, good or bad. One outside any range you considered points back to question 1 or 2.

A decision is sound when questions 1 to 3 hold up. The outcome is evidence only through question 4.

Two decisions through the same gate

Decision A: no third truck. In March, a two-truck HVAC shop decided against adding a third truck and tech for summer.

1. Known then: the last three summers had averaged about one week each of overflow, meaning calls the two trucks could not reach inside two days. A third truck and tech would have been busy for about four months and largely idle for the other eight, while carrying payroll and payments all year.
2. Cheap to learn: the shop had the three summers of call data and used it. Long-range weather outlooks in March say little about a single summer's heat. Nothing obvious was skipped.
3. Reasoning: sound. Paying for eight slow months to cover about one week of overflow did not pencil out on any of the three prior summers.
4. Outcome: a record summer. About six weeks of overflow, and the office turned away or lost about 45 calls. Measured against the three-summer history of about one week, six weeks is far out in the tail, but heat waves are a known feature of the trade, so it was inside the range of what could happen, just at the far end.

Verdict: sound decision, bad outcome. The wrong lesson is "always buy the truck." One extreme summer is one data point, and the next summer is more likely to look like the prior three than like this one. The right lesson is narrower: the shop had no plan for the tail. A standing overflow arrangement with another licensed shop, or a short-term rental lined up in advance, would have caught much of the six weeks for a fraction of a year-round truck. That is the cheap hedge the top-right box asks for.

Decision B: the builder contract. The same owner, the year before, agreed on the spot to a four-month new-construction package for a builder new to the area, committing about a third of the shop's install capacity.

1. Known then: the builder's name, the job count, and a friendly meeting. Nothing about payment history.
2. Cheap to learn: a lot. Two calls to trades that had worked for the builder, a question to the supply house about how the builder's accounts paid, and a read of how the contract handled payment timing and lien rights. Perhaps two hours. None of it was done.
3. Reasoning: a third of install capacity for four months tied to one unknown payer, with no fallback if payments stalled. If even one in four unvetted builders pays slowly, and you should put your own number on that rather than borrow this illustrative one, the downside was a cash squeeze in the shop's busiest season.
4. Outcome: the builder paid every draw on time and offered more work.

Verdict: poor decision, good outcome. The owner's takeaway at the time was "trust your read of people." The gate's takeaway is that the process skipped two cheap hours on a decision that put a third of capacity at risk, and the outcome does not change that. The right change is a standing rule: no capacity commitment above a set share of the schedule to a new payer without references and a supply-house check. Keep the builder, because the evidence is now good. Fix the process, because the next unvetted builder will not be this one.

Set side by side, the decision that felt wrong was the sounder of the two, and the decision that felt right was the one that needed fixing. That reversal is the whole reason to run the gate.

What changes the verdict

A decision that was sound given the information can still be a process failure. If a cheap check would have changed the call, question 2 fails even when questions 1 and 3 hold. Decision B is the example: the reasoning was coherent on what the owner knew, and what he knew was thin by choice.

An outcome far outside any range you considered is information about the model, not just luck. If the shop in Decision A had seen overflow go from about one week to six weeks two summers running, that is no longer a tail event. It is a sign the market has shifted, and the March reasoning needs new inputs.

Repeated decisions are different. For something you decide dozens of times a year, such as which calls to accept or which jobs to bid, the outcomes do average out, and a run of bad results over enough cases is a fair verdict on the process. The gate matters most for the decisions you make once.

The opposite failure

Outcome bias has a mirror image that looks like wisdom: calling every bad result bad luck. An owner who answers each loss with "the decision was sound" is protecting their ego with the language of this card.

The tell is that question 4 never gets a straight answer. If you cannot say, before the outcome is known, what range of results you expected, you cannot claim afterward that the result fell inside it. That is the practical case for writing the expected range down when you decide, and the decision journal is where it goes. See related: Keeping a Decision Journal.

A second tell: the "bad luck" explanations cluster on the same kind of decision. Three bad breaks in a row on fixed-price jobs is not three bad breaks. It is a pattern question 4 should be asked about across all three at once.

Running it in your own shop

Pick one decision from the last year that went well and one that went badly, and run both through the four questions, in writing, with the person who was closest to each. Expect at least one of the two to land in a diagonal box. If neither does, you have either been unusually lucky in your sample or unusually generous in your answers to question 2.

Then change one thing: for the next major decision, write down the expected range of outcomes before deciding. It takes two minutes and it is the only thing that makes question 4 answerable honestly later.

References

• Baron, J. and Hershey, J. C., "Outcome Bias in Decision Evaluation," Journal of Personality and Social Psychology (1988)
• Duke, A., Thinking in Bets (2018)
• See related: Hindsight Bias in the Callback Review
• See related: The Job We Lost Money On Was the Right Call
• See related: Keeping a Decision Journal