Using AI to Ask Questions of Your Own Job Data

Why this matters

Upload a year of jobs to a chat assistant, ask which job types are slowest to pay or which tech has the most callbacks, and you get a confident, well-formatted answer in seconds. Two things can be wrong with it, and neither shows. The tool may have done the arithmetic in a way you would not have chosen, such as treating a blank as a number. Or the arithmetic may be right and the pattern may be a fingerprint of how your office enters data rather than anything about the work. Act on either and you chase the wrong invoices or have the wrong conversation with a tech. The procedure below carries one real-shaped export through every step, so you can see where each problem appears.

The example is a fourteen-person plumbing shop exporting twelve months of completed jobs: 3,120 rows. Two questions: which job types are slowest from invoice to payment, and which techs generate the most callbacks.

Step 1: Write the question and the shape of the answer before you export

One sentence each, with the unit. "Average days from invoice date to paid date, by job type, for jobs completed in the last 12 months." "Count of callbacks per tech, where a callback is a return visit to the same customer for the same problem within 30 days."

Why first: the second definition decides which columns you need and exposes a problem before you start: does your data actually record "same problem"? At this shop it did not, which turned out to matter at Step 6. What breaks if you skip it: you ask a loose question, the tool picks a definition for you, and you never learn which one it picked.

Step 2: Export only the columns the question needs, and strip people out

For these two questions: job number, job type, assigned tech, completion date, invoice date, paid date, customer ID (a code, not a name), and the job note field. No customer names, addresses, phone numbers or email addresses: the questions do not need them, and what customer data may go into an outside tool at all is a shop policy decision, not an analysis decision. See related: The Shop AI Use Policy SOP.

Why: fewer columns means fewer things the tool can misread and less exposure if the data leaves your control. What breaks if you skip it: you send a year of customer contact details to a third party to answer a question about payment timing.

Step 3: Look at the export yourself before the tool does

Open it in a spreadsheet and spend ten minutes on three checks: row count, blanks in each column, and the list of distinct values in any category column. At the plumbing shop:

• 3,120 rows, matching the job count in the field software for the year.
• 214 rows with a blank paid date. Some will be unpaid. Some may be paid but not recorded.
• Job type "Other" on 388 rows, about 12 percent of 3,120.
• A tech named "Office" on 140 rows: jobs booked under the default user and never reassigned.

Why this step earns its place: every one of those four facts changes an answer later, and the tool will not mention any of them unless asked. What breaks if you skip it: you meet these facts for the first time inside a wrong answer, if at all.

Step 4: Ask for the method and the counts, not just the answer

Upload the file and ask the question from Step 1, adding: show the number of jobs in each group, and state exactly how you computed it, including what you did with missing values. Here is what came back for the first question:

YOU:  Using jobs.csv, average days from invoice
      date to paid date by job type, last 12
      months. Show the job count per group and
      your exact method, including blanks.

TOOL: Water heater replacement  31.0 days  96 jobs
      Repipe                    24.5 days  41 jobs
      Drain cleaning             6.2 days  1,140 jobs
      ...
      Method: days = paid_date minus invoice_date.
      Where paid_date is missing, the invoice is
      still open, so I used the export date to
      reflect time outstanding so far.

The answer looks finished. The method line is where the problem is: the tool treated every blank paid date as "paid today". That is a defensible choice for "how long has money been outstanding" and the wrong one for "how long do paying customers take", which is the question asked.

Why ask for the method: it is the only way to see a choice like this. The tool will not volunteer it, and the number on its own gives no hint. What breaks if you skip it: you read 31 days as how long water heater customers take to pay.

Step 5: Re-check two groups by hand

Pick the group the answer singles out and one ordinary group, and recompute them yourself in the spreadsheet with a formula you can see.

Water heater replacements, 96 jobs. Filtering out blank paid dates leaves 87 paid jobs, averaging 18 days from invoice to payment. The other 9 have blank paid dates; measured to the export date they averaged about 157 days. Check the tool's figure from those pieces: 87 times 18 is 1,566 days, 9 times 157 is 1,413, together 2,979 days over 96 jobs, which is 31.0. So the tool's arithmetic was right, and its 31 days was 18 days of real payment behavior plus nine old open invoices pulling the average up.

Then the ordinary group, drain cleaning: the hand check matched the tool within a tenth of a day, because only a handful of drain jobs had blanks.

Why two groups: the extreme group is where an artifact usually lives; the ordinary one tells you whether the tool's general method holds. What breaks if you skip it: you raise prices or tighten terms on water heater jobs whose customers actually pay in 18 days.

The nine open invoices are a real finding, just a different one. Reading their notes, 4 had been paid by check recorded only in a job note, never entered as a payment. The other 5 were genuinely unpaid and went to collections follow-up the same week.

Step 6: Hunt for patterns that are really about data entry

Now the callback question. The tool reported Dana with 22 callbacks in the year, more than three times any other tech, and offered to draft talking points for a performance conversation.

Before anyone talked to Dana, the office manager checked how callbacks get recorded. The shop has a "Callback" job type. Dana uses it every time she goes back to a job. Most of the other techs book their returns as an ordinary "Service call" and mention the earlier visit in the note, if at all. So the count was not callbacks per tech; it was use of the Callback job type per tech. Dana was the only tech recording her callbacks honestly, and the analysis would have punished her for it.

This is the step the tool cannot do for you, because the pattern is real in the data. Things that commonly create one:

• A job type used by one person and not others. The callback case.
• A default value: the 140 "Office" jobs would make the office look like the busiest tech in the shop.
• A catch-all category: 388 "Other" jobs hide whatever they really were.
• When the office closes jobs. If jobs are closed in a batch on Monday, completion dates spike on Mondays regardless of when the work happened.

To answer the real question the shop built a proper definition from Step 1: any job for the same customer ID within 30 days of a previous one, read by a person to confirm it was the same problem. That took an afternoon on the 3,120 rows with a spreadsheet filter, and Dana landed in the middle of the pack.

What breaks if you skip it: a decision about a person resting on a quirk of how they fill in a form.

Step 7: Margins stay in a spreadsheet you can see

If the question involves margin or profit, the tool does not compute it unsupervised. Margin needs costs: labor at each tech's real cost, parts at what you paid, and an overhead decision. An export rarely holds all three, and a tool asked for margin will build one from whatever column looks like a cost, often treating parts cost as total cost, or confusing markup (profit over cost) with margin (profit over price). Both produce a clean percentage that is wrong.

Use the tool to suggest the formula and explain it if that helps. Then build the calculation in a spreadsheet where every cell shows what it is made of, check one job by hand against its invoice and its time entries, and only then read across job types.

What breaks if you skip it: a job type looks profitable because its labor cost was never in the file.

Step 8: Write down the answer with its conditions and fix the data

The office manager's note for the owner read, in full: "Water heater customers pay in about 18 days on the 87 paid jobs; the tool's 31 days counted 9 old open invoices as paid today. 5 of those 9 are genuinely unpaid and are in follow-up; 4 were paid by check and never entered. Callback counts by tech are not usable yet because only one tech uses the Callback job type."

Then the data fixes, each with an owner: checks entered as payments on the day, not as notes; the Callback job type required for any return within 30 days; "Office" jobs reassigned weekly; "Other" restricted so it needs a note. In three months the same questions can be asked again with far fewer traps in the file.

What breaks if you skip it: the number travels without its conditions, and six months later someone quotes "31 days" in a pricing meeting.

How to know an answer is safe to act on

• You can state the method in one sentence, including what happened to blanks.
• Two groups were recomputed by hand and matched, or the difference is explained.
• You know how each field in the answer gets filled in day to day, and by whom.
• No answer about a person rests on a field only some people use.
• Any margin figure was built in a spreadsheet with visible formulas and checked against one real job.

References

• See related: The Shop AI Use Policy SOP
• See related: What Not to Paste Into an AI Tool
• See related: The Verify-Before-You-Send Rule for AI Output
• See related: What a Language Model Is Actually Doing When It Answers