The AI Quoted a Code Section That Does Not Exist

Why this matters

Nobody in this case set out to cut a corner. A customer asked a reasonable question, the office answered it quickly with a chat assistant, the answer cited a code provision by number, and the job went ahead on the strength of it. Three weeks later a home inspector found the gap, the customer's house sale slipped, and the shop spent the better part of two days repairing something a two-minute phone call would have prevented. The interesting part is not that the AI invented a citation. That is a known behavior. It is that three people handled the job and each one assumed somebody else had checked.

This case is reconstructed backwards, from the paperwork the owner had in hand when the angry call came in. The gaps in that paperwork turned out to be part of the finding.

The record as the owner found it

A three-plumber shop. The owner pulled everything with the customer's name on it the afternoon the customer called. In date order:

MAR 11  Email, office -> customer
        "Good news: a like-for-like water heater
        swap does not need a permit. Per IRC R105.2,
        Plumbing, item 4, replacement of a water
        heater with one of equal capacity and fuel
        type is exempt."
MAR 11  Estimate sent. No permit line.
MAR 14  Job: replace 40 gal gas heater. Tech note:
        "Installed, vented to existing, tested
        relief, leak checked gas. Done."
APR 02  Customer voicemail: buyer's home inspector
        flagged "water heater replaced, no permit
        on record." Closing on hold.

Four documents, and between them a complete story of what was done and no record at all of who decided the one thing that went wrong.

Reading the email that went out

The owner read the March 11 email first, because it was the only document that made a claim about the code.

The citation had the right shape. The International Residential Code's section R105.2 is the real list of work exempt from a permit, and it is organized by trade, with a short plumbing list. A sentence citing "R105.2, Plumbing" is pointing at the right neighborhood. What it attached to that address was invented. The plumbing list under R105.2 in the edition this town had adopted has two items, covering stopping leaks and clearing stoppages without replacing or rearranging valves, pipes or fixtures. There is no item 4. There is no water heater exemption anywhere in it. And the town's own permit page, which took the owner under a minute to find, lists water heater replacement as permit-required work, which is common practice for gas appliances because the venting and the gas connection are exactly what an inspector exists to look at.

So the claim failed three ways at once: the item number did not exist, the exemption it described did not exist in the model code, and even if a model code had exempted it, the town's adopted rules said otherwise. The mechanism behind that kind of answer, correct shape with invented specifics, is explained in: What a Language Model Is Actually Doing When It Answers.

What the chat history showed, and what it did not

The coordinator, Priya, still had the conversation in her chat history. The prompt was one line: "Does replacing a gas water heater with the same size need a permit? Cite the code." The answer was the paragraph she pasted into the email, nearly word for word.

"Cite the code" did the damage. Asking for a citation does not make the model look one up; it makes the model produce text in the shape of a citation, which it can do with complete fluency whether or not it has seen the real provision. The request for rigor produced the appearance of rigor.

What the history did not show was the thing the owner most needed: whether anyone checked. There was no second search, no visit to the town's site, no note on the job. Priya said she had assumed the plumber would flag it if a permit was needed, because the plumbers pulled permits all the time.

Following the assumption through the other hands

The owner then asked each person what they had believed on March 14.

• The estimator built the estimate from Priya's email and left the permit line off because the email said none was needed. He did not re-check because the email cited a code section; a cited section read as settled.
• The plumber saw no permit on the work order. On most jobs the office pulls it before he arrives, and a missing permit usually meant the town did not require one for that address. He did not ask, because asking felt like second-guessing the office on paperwork.
• The owner had not seen the job at all; it was routine.

Every one of those beliefs was reasonable from inside that person's job. The failure was at the joints, where a decision passed from one person to the next with the citation acting as proof that the decision had already been made. A specific-looking reference is the most persuasive kind of AI output for exactly this reason: it stops the next person from checking.

What it cost

Costed in the shop's own terms, not money:

• Owner time: about 6 hours over two days. Calls to the customer, the buyer's agent and the town, filing the after-the-fact permit, and meeting the inspector on site.
• Plumber time: 2 hours for the inspection visit, including pulling the access panel and having the venting and gas connection ready.
• The permit fee, absorbed by the shop.
• The customer's closing slipped five days while the permit was opened and the inspection scheduled. That was the part the customer remembered.
• Trust. The install itself passed inspection without corrections, which the owner made sure the customer heard. But the customer's review, posted the week after closing, did not mention the install. It said the shop "told me in writing no permit was needed and that turned out to be false." That review is still the first thing a search for the shop's name shows.

Eight hours of labor across two people is recoverable. A written statement from the shop, quoting a code section that does not exist, now forwarded to a buyer's agent and a home inspector, is not.

The fixes the owner considered first, and why he dropped them

His first three instincts were all aimed at the tool, and each would have left the joints open.

• "Tell it to only cite real sections." He tried it. Adding "only cite sections that actually exist" to the prompt produced an answer with the same confidence and a different invented item. The model cannot tell which of its citations are real, so an instruction to cite only real ones changes the wording around the citation, not its accuracy.
• "Make her ask it whether it is sure." Priya tried that too. The tool apologized and gave a third version. A changed answer under pressure shows the first one was weakly held; it does not show which version, if any, is right.
• "Use a better tool." Some tools search the web before answering and show their sources, and those are genuinely less likely to invent a section. But a search can land on a forum post about another state, or a model code edition the town never adopted. The source still has to be opened and read, which is the step that was skipped.

All three would have made the office feel safer while leaving a citation in the email as proof that someone had decided. That is why the changes below target the hand-offs instead.

The process change

The owner made three changes, each aimed at one of the joints rather than at the tool:

1. Code and permit questions came off the AI task list entirely. Permit determination for any job is made by the lead plumber from the town's own permit page or a call to the building department, and recorded on the job. The office may use AI to tidy the wording of the answer afterwards, never to produce it. This matches where such questions sit on the shop's risk grid; see related: Where AI Helps a Small Shop and Where It Quietly Costs You.
2. No code citations in customer emails. The shop's message templates now say "the town requires a permit for this work" or "the town does not require a permit," with the source being the town, not a section number. A citation a customer cannot check adds nothing they need, and a wrong one is quotable against you forever.
3. The permit line is never blank. Every estimate carries a permit field with three allowed values: required, not required (with who decided and how), or to be confirmed. An estimate cannot be sent with "to be confirmed" still showing. That one field makes the decision visible at every hand-off, which is precisely what was missing between Priya, the estimator and the plumber.

The general rule the first change applies, that code citations are checked against the source or not sent, is owned by The Verify-Before-You-Send Rule for AI Output.

How they confirmed it held

The owner checked two ways over the next quarter. First, every estimate sent in that period had its permit field filled with required or not required, and each not required carried a name and a source: 31 estimates, none blank, four marked not required, all four traced to the town's permit page. Second, he asked Priya to run the original one-line prompt again in a fresh conversation, as a test. The new answer cited a different item number with a different invented exemption. That was the most useful thing the exercise produced: the error was not a one-off glitch that would not recur, it was the tool behaving normally, and it would do it again to the next person who asked.

What would have caught it on March 11 for almost no cost: a single sentence in Priya's habits, "if the answer cites a rule, the rule's own source has to be open on my screen before it goes in an email." The permit page was one search away.

References

• International Code Council, International Residential Code, Section R105.2 (Work exempt from permit), in the edition your jurisdiction has adopted; your local building department's permit requirements govern work there
• See related: The Verify-Before-You-Send Rule for AI Output
• See related: What a Language Model Is Actually Doing When It Answers
• See related: The Invention Patterns to Expect in Trade Questions
• See related: Where AI Helps a Small Shop and Where It Quietly Costs You