Regression to the Mean and the Tech Who Had a Great Month

Why this matters

A tech has the best month anyone can remember. You say so at the huddle, and the next month he comes back to earth. Another tech has a terrible month, you sit him down, and the next month he is fine. Run that loop for a few years and you will conclude, honestly and from experience, that praise makes people coast and pressure makes them perform. You will be wrong, and you will have built a management style on an illusion. Extreme months drift back toward normal on their own. Knowing that changes how you read every leaderboard, every bonus, every "the training worked."

What regression to the mean actually is

Any monthly number a tech produces (tickets sold, callbacks, billed hours, reviews) is part skill and part luck: which calls dispatch handed him, which customers happened to need a replacement, whether two big jobs landed in the same month. Skill is stable month to month. Luck is not.

When you pick out the most extreme month, you are picking the month where luck ran hardest in one direction. Next month the skill is still there and the luck is a fresh draw, which on average is ordinary. So the next number lands closer to that person's normal. That is regression to the mean: extreme measurements tend to be followed by less extreme ones, purely because the extreme was partly luck. Francis Galton described it in the 1880s with the heights of parents and children. Nothing has to push the number back. Selection did all the work.

Daniel Kahneman tells the version every manager should hear: flight instructors in the Israeli air force insisted that praising a great landing made the next one worse, and yelling at a bad landing made the next one better. Their observation was accurate. Their explanation was not; the landings were regressing whatever the instructors said.

A shop example with the numbers carried through

Six techs at an HVAC shop, tracked on add-on tickets sold per month (a maintenance plan, a repair beyond the service call, an accessory). The 12-month averages below come from the prior year. All figures are illustrative.

At the April 1 huddle the owner publicly praised Dana and Ray, the March top two, and wrote up Mo and Luis, the bottom two, with a sales refresher.

April results, by group:

• Praised (Dana, Ray): 41 + 30 = 71 in March, 31 + 25 = 56 in April. Down 15.
• Written up (Mo, Luis): 18 + 13 = 31 in March, 25 + 21 = 46 in April. Up 15.
• Left alone (Kim, Pat): 26 + 23 = 49 in March, 24 + 22 = 46 in April. Down 3.

The owner's reading: praise backfires, the write-up worked, and he should be harder on the crew. The real reading is in the first column. Every one of the four extreme techs moved back toward their own 12-month average. Dana went from 14 above her average to 4 above. Luis went from 11 below his to 3 below. The two techs who had ordinary Marches had ordinary Aprils. The write-up may have helped Mo and Luis a little, but this data cannot show it, because they would have bounced back anyway.

How far back to expect it to fall

The size of the bounce depends on how much of a monthly number is skill. You can estimate that from your own records with a crude rule: take every month where a tech was well above their own average, and see how far above average the following month was.

Say that across the six techs over the prior year there were 14 months that ran more than 5 tickets above the tech's own average. In the month after each, the tech averaged about 3 above their own average, against an average of about 9 above in the high months themselves. So roughly a third of an extreme carried into the next month and two thirds was luck. Call that carry-over fraction about 0.3.

Then a reasonable forecast for next month is: the tech's own average, plus 0.3 times how far this month was from it.

• Dana: 27 + 0.3 x 14 = about 31. She sold 31.
• Ray: 22 + 0.3 x 8 = about 24. He sold 25.
• Mo: 26 - 0.3 x 8 = about 24. He sold 25.
• Luis: 24 - 0.3 x 11 = about 21. He sold 21.

Your shop's carry-over will be different, and it differs by measure. Callback rates on a small number of installs are mostly luck month to month (a low carry-over); billed hours for a tech who runs a full schedule are mostly skill and dispatch (a high one). The lower the carry-over, the harder a single month regresses and the less a single month should move your opinion of anyone.

Which average they regress toward

A tech regresses toward their own long-run level, not the shop's. Dana's average is 27 against a crew average of about 24.5, and she regresses toward 27. That matters in two places:

• A genuinely strong tech's ordinary month is not a slump. If Dana sells 27, she has not "dropped off" from 41; she has had a typical month for Dana.
• A new hire has no own average yet. Three months of data is mostly luck, so a new tech's best month should be read as regressing toward the shop's average for new techs, not toward their own short record. A spectacular first month predicts a solid second month, not a star.

Where it bites in a shop

• Monthly contests. The winner of a one-month sales contest is, more often than not, the tech whose luck ran hottest. Next month's results will make the prize look like it demotivated them. A quarterly contest, or one scored on improvement over the tech's own average, is less exposed.
• Hiring off a great ride-along. One strong day riding with a candidate is a sample of one. Expect the hire to be closer to ordinary than the day suggested.
• Interventions triggered by a bad month. Any program you launch after your worst month (a retraining, a new script, a marketing push, a callback review) will usually look like it worked, because the next month would have been better anyway. This is the most expensive form of the error, because it convinces you to keep paying for things that did nothing. If you want to know whether the program worked, compare against people or months that did not get it. See related: Running a Small Experiment Instead of Arguing.
• Firing after the worst quarter. If you replace whoever had the lowest numbers, the replacement will tend to look better than the person they replaced even if they are no better, because you sampled the old tech at their low point.
• Your own business year. A record year is often followed by a disappointing one, and owners who planned spending around the record year get caught. Plan around a multi-year average.

When it is not regression

Regression explains a one-off extreme. It does not explain a sustained one. Treat the change as real when:

• It persists. Three months running 5 or more tickets below the tech's own average is a pattern, not a draw.
• There is a mechanism you can name. A new territory, a change in the call mix, a family situation, a new pay plan.
• It repeats in the same direction after a reset. A tech who regresses back and then drops again.

The detailed test for whether a moving number is a signal belongs to the Analytics shelf. See related: A Number Moves: Is It Noise or a Real Signal?

One rule, two techs who both "fell off"

Take the trigger suggested below (a rolling 3-month average more than 15% under the tech's prior-year average) and run it against two techs from the same table.

• Dana, March to May: 41, 31, then 27 in May. To the eye she has fallen 14 tickets in two months, and an owner reacting to the month in front of him would have a word with her. Her rolling 3-month average is (41 + 31 + 27) / 3 = 33, which is 6 above her 27 average, not under it. No trigger. Nothing happened to Dana except that March was lucky.
• Kim, May to July: 19, 18, 20. No single month looks dramatic; she never had the worst month on the board. Her rolling average is (19 + 18 + 20) / 3 = 19, which is 6 under her 25 average, or 24% under. That clears the 15% line. Trigger: something has changed, and it is worth a conversation about call mix, territory or what is going on.

The same 6-ticket gap in both cases, pointing opposite ways, and the rule sorts them correctly where the eye sorts them backwards. The dramatic tech was fine and the quiet one was the real signal.

The habits that follow

• Judge people on a rolling 3-month or 6-month average, not on the month in front of you. Praise and correction both go on the rolling number.
• Praise behavior you saw, not a single outcome. "You walked every customer through the options sheet" stays true after a quiet month. "Best month ever" sets you up to look like a jinx.
• Intervene by rule, not by extreme. Decide in advance what level and duration triggers a conversation (for example, rolling 3-month average more than 15% under the tech's prior-year average), so you are not reacting to luck.
• Write down what you expect next month before you act. If you expected regression and got it, nothing you did is proven. See related: Keeping a Decision Journal.

References

• Daniel Kahneman, Thinking, Fast and Slow (2011), chapter 17 on regression to the mean and the flight instructor example
• Francis Galton, "Regression Toward Mediocrity in Hereditary Stature" (1886)
• See related: A Number Moves: Is It Noise or a Real Signal?, Running a Small Experiment Instead of Arguing
• See related: Outcome Bias: Judging a Decision by How It Turned Out, Keeping a Decision Journal