What AI Cannot Know About Your Job Site
Why this matters
A chat assistant answering a field question is answering the question it was given, about a typical house it has assembled from everything it has read. The tech is standing in a specific house that is not typical in at least one way, and that one way is usually the whole job. The answer that comes back is fluent, plausible for the typical house, and wrong for this one, and nothing in its tone says so. A tech who understands what never reaches the model can use it for what it is good at (explaining a system, listing possibilities, decoding a term) without letting it make the call that only someone on site can make.
The model receives your words and nothing else
Everything a model knows about the job arrives as the text, and sometimes the photo, you give it. It cannot ask the question a senior tech would ask on the phone, because it does not know what it does not know; it fills the gap with the typical case and answers as though the gap were not there. Why it fills rather than asks is the subject of its own card (see related: What a Language Model Is Actually Doing When It Answers). This card is about the contents of the gap, because if you can name what is missing you can tell, in the moment, whether the missing part matters to this answer.
There are five kinds of context that almost never make it into a prompt, and each one routinely decides a call.
1. What your senses are telling you
Smell, sound, vibration, heat under the hand, the feel of a valve that turns too easily. A tech registers these without thinking and rarely types them. They are often the most diagnostic information on site.
- Smell. Hot insulation, a faint sulfur note, a sweet chemical smell near a refrigeration line, sewer gas from a dry trap. Each points a direction the written symptoms do not.
- Sound. The difference between a bearing whine and a blower wheel out of balance, a water hammer bang versus a tick of thermal expansion, a relay chattering.
- Touch and heat. A breaker or receptacle face warm to the touch, a motor too hot to hold a hand on, a pipe cold where it should be warm.
Where it is a hazard, act before asking anything. If what you smell is gas, stop: get everyone out of the building, do not operate any switch, light or appliance, do not use a phone inside, and call the gas utility or 911 from outside. If what you smell is burning or hot electrical insulation, turn off and isolate the affected circuit or equipment at its disconnect or breaker if you can reach it without passing the source, keep people back, and treat it as a fire risk until it is found. No tool should be consulted with a live hazard in front of you; the response is the shop's, and it does not depend on the answer.
2. What happened here before
The customer's history with the shop, the system's service history, the last tech's shortcut, the repair that was done three times. A model does not know that this is the fourth capacitor in two summers, which turns "replace the capacitor" from the answer into the symptom. It does not know the previous owner did their own electrical work, or that the last visit's note says "customer declined recommended repair."
This is where a tech's own job history in the field software is worth more than any answer: a pattern across visits is invisible to a tool answering one visit at a time.
3. What is unusual about this house
The aftermarket modification, the mislabelled panel, the circuit fed from a subpanel in the garage, the condensate line that drains into a sump instead of a floor drain, the vent that was rerouted when the kitchen was remodelled, the well and softener that change everything about water chemistry, the 1950s addition with its own wiring. A model assumes the house was built to one standard at one time and never altered. Most houses you work in have been altered.
4. Who lives there and what they can do
An infant, an elderly parent on oxygen, a tenant who cannot reach the shut-off, a customer who will reset the breaker the moment you leave, a homeowner who says "it's always done that." The right advice for a capable homeowner is wrong for a customer who cannot follow it. A model advising "monitor it and call if it gets worse" has no idea whether anyone will be there to monitor it.
5. Where the house is
Local code amendments, the inspector's known preferences, climate, the water's hardness, soil, the utility's rules. A model will give you the model-code answer, or a blend of several, when your jurisdiction amended that exact section. It does not know that your county requires a permit for work the model calls minor.
A field-card of confidently wrong answers
These are the shapes of error that show up repeatedly. Each is a correct answer for the typical house.
| What the tech typed | Typical answer | What the tech on site knew | What it actually was |
|---|---|---|---|
| Breaker keeps tripping on a bedroom circuit | Overloaded circuit or a weak breaker; try replacing the breaker | A receptacle cover plate felt warm and smelled faintly of hot plastic | A loose, overheating connection at a receptacle |
| Water heater making rumbling noise | Sediment; flush the tank | House on hard well water, tank 3 years old, softener bypassed | Sediment was real, but the cause was the bypassed softener; flushing alone just resets the clock |
| Slow drains throughout the house | Use an enzyme cleaner, snake the main | Lawn spongy over the drain field, septic smell outside | A septic system problem, not a line clog |
| Ceiling stain under an attic | Roof leak, check flashing | Bath fan duct ended loose in the attic, stain right below it | Condensation from an unvented bath fan |
| AC runs but house not cooling | Low refrigerant, check charge | Filter slot empty, return grille covered by a bookcase | Restricted airflow, and adding refrigerant would have overcharged it |
In every row, the information that changed the answer was something the tech saw, smelled or knew from history, and none of it was in the prompt.
Worked example: one question, two answers
A tech on a no-heat call has a gas furnace that lights, runs for about two minutes, then shuts down and retries. He types: "Gas furnace lights, runs about two minutes, shuts off, retries. What should I check?"
The answer he gets is a typical list: flame sensor dirty, clean it; check the pressure switch; check the filter; check the high-limit. It is a good list for a typical furnace. It leads with the flame sensor, because a dirty flame sensor is the most common cause of short run cycles in that description.
What he knows that the model does not. He knows the customer had the furnace serviced three weeks ago by another company, which cleaned the flame sensor. He knows the house had new windows and doors put in this autumn. And when he pulled the burner door he noticed a faint sharp smell at the furnace that was not there last year, and the flame looked lazy and lifting rather than crisp.
The answer changes. A recently cleaned flame sensor drops to the bottom of the list. A house made much tighter by new windows and doors, a lazy flame and a changed smell point toward combustion air or venting: the appliance may not be getting the air it needs, or exhaust may not be clearing, and that raises the possibility of carbon monoxide in the living space.
The action changes with it, and it comes before any further diagnosis. He shuts the furnace off at its service switch and closes the gas valve to the appliance, puts on the personal carbon monoxide monitor he should have had on when he walked in, and checks the air near the furnace and in the living space. If the monitor reads elevated carbon monoxide, everyone goes outside, he opens doors and windows on the way out if it is quick to do, and he calls the gas utility or emergency services per the shop's procedure. Only once that is settled does he check for a blocked vent, a combustion air opening that was covered, or a draft problem, with the furnace left off until it is resolved.
What went wrong in the first answer was not the content, which was accurate for a typical furnace. It was that the prompt carried two minutes of runtime and a retry, and left out three weeks, new windows and a smell. The tool could not have known to ask. A senior tech on the phone would have asked "anything changed in the house?" in the first thirty seconds.
What would have changed the conclusion back. If the house had not been altered, the flame had looked normal and there was no smell, the typical list would have been a perfectly reasonable place to start, and the dirty flame sensor would have been a sensible first check even three weeks after a cleaning. The model's answer is not wrong in general. It is wrong exactly when the house is not typical, which is the case you cannot know from the prompt alone.
What you can and cannot hand to the tool
You can improve answers a great deal by putting the missing context into the prompt: the history, the sensory cues, what changed, the age and setup of the house. A good prompt from a tech who knows what matters will get a much better answer than "furnace short cycles."
What you cannot hand over is the judgment of which details matter, and the call itself. A model will weigh every detail you give it by how often that detail appears in what it has read, not by what is in front of you. The tech decides what the smell means and whether the house is safe to leave running. The tool, at its best, is a fast way to remember possibilities you might have forgotten, after you have already decided what kind of problem you are looking at.
That is also why the shop's field rule on wiring and other energized work exists separately (see related: The Tech Followed an AI Wiring Answer): where a wrong answer can hurt someone, the tool's answer is never what a tech acts on.
References
- See related: What a Language Model Is Actually Doing When It Answers
- See related: The Tech Followed an AI Wiring Answer
- See related: The Customer Arrives With an AI Diagnosis
- See related: The Invention Patterns to Expect in Trade Questions
- Appliance manufacturer installation instructions and your adopted fuel gas code for combustion air and venting requirements