Accuracy, Resolution and Repeatability Are Three Different Things

Why this matters

A shop buys the instrument with the most digits, uses it for the one task those digits cannot help with, and then spends two seasons wondering why its numbers argue with the manufacturer's limit. The three properties on a spec sheet fail independently of one another, and any given field task is limited by exactly one of them. Name which one, and both questions answer themselves at once: which instrument to carry, and which digit on the display you are allowed to believe.

Get it backwards and the failure is quiet. Nothing reads as broken. You just make one class of call slightly wrong, over and over, in the same direction.

The three properties, and the one question that separates them

Resolution is the smallest step the display can show. It is a property of the readout and it costs the manufacturer almost nothing, which is why it leads the marketing.

Accuracy is how far the displayed value may sit from the true value. It is a published claim, valid only inside the conditions the manufacturer states, and it is nearly always several times larger than the resolution.

Repeatability is the spread you get when you measure the same steady quantity several times in a row, without changing anything. It says nothing about whether the readings are right, only whether they agree with each other.

The question that sorts them: does your task compare the reading to something outside the instrument, or to another reading from the same instrument?

Against something outside it - a published limit, a nameplate, a specification, another shop's measurement - the whole error lands on you, and accuracy governs. Against another reading from the same instrument on the same equipment, any fixed error the instrument carries appears in both readings and subtracts away, so accuracy largely drops out and repeatability governs. Resolution only ever governs when the step you need to see is smaller than the display can show, which is rarer than it feels.

That is the whole model. An instrument that is consistently wrong is dangerous for the first kind of task and perfectly usable for the second, and no amount of care in the field changes which one you are doing.

The last row is the trap, and it turns on one condition: whose errors are they. Two DIFFERENT instruments, or a random component, means the two percent-of-reading errors are independent, they do not cancel, and the small difference inherits an error scaled to the LARGE readings, which is why subtracting two big numbers to get a small one is the worst thing you can ask an instrument to do. ONE instrument carrying a fixed proportional offset is the opposite case: the offset scales both readings alike and comes out as that same percentage of the DIFFERENCE, which is negligible. Name which case you are in before you trust a difference.

Before you read the same point ten times

Establishing repeatability means taking a series of readings, and a series is a series of exposures. That is a hazard the instruction itself creates, not one that comes at you from the equipment.

Run the series on a bench source, a known reference, or a permanently installed test point rather than by repeatedly opening an enclosure. Where the point can only be reached live, 29 CFR 1910.333(a)(1) permits energized testing only where de-energizing is infeasible, and it gates the whole series, not the first reading: open the enclosure once, take the series without breaking the setup, and close it. The instrument, leads and probe tips carry a measurement category and voltage rating at or above the circuit under IEC 61010-1, which binds through the listing mark on the instrument, and 29 CFR 1910.334(c)(2) requires inspecting the instrument, leads, cables, probes and connectors for external defects before use. Wear the protective equipment 29 CFR 1910.335(a) requires for the exposure and keep one hand clear of the enclosure. When the work turns into a repair, lock out under 29 CFR 1910.333(b)(2) in general industry or 29 CFR 1926.417 in construction and prove dead with the live-dead-live sequence at 120.5 of NFPA 70E-2021, in whichever edition your employer's electrical safety program has adopted.

Repeatedly seating and unseating a probe on a hot surface, or repeatedly breaking a fluid connection to re-read it, is the same problem in a different trade. Set the coupling once and leave it.

The case: a shop that bought digits and needed steadiness

A shop replaced its field instruments for a routine condition-monitoring task and chose between two units. The figures below are illustrative stand-ins for whatever your own candidates publish, but the shape is what matters.

Instrument X displays to 0.01 units, published accuracy plus or minus 2 percent of reading over its stated ambient band.

Instrument Y displays to 0.1 units, published accuracy plus or minus 0.5 percent of reading over the same kind of band.

X shows two more digits, so the shop bought X for everybody. Note that both accuracy figures are simple percent-of-reading here for clarity; most real specs add a fixed counts term as well, and the sibling card on accuracy and resolution owns how that term behaves and why it dominates at the bottom of a range.

Task one, pass or fail against a published limit of 5.00 units. A tech reads 5.06 on X. Accuracy is 2 percent of 5.06, which is 0.10, so the true value sits somewhere between 4.96 and 5.16. The limit is 5.00, and the band straddles it. The instrument cannot say which side of the limit the equipment is on, and the two extra digits contributed nothing to answering that, because the question was never about display steps.

The same measurement on Y reads 5.1, since 0.1 is the display step. Accuracy is 0.5 percent of 5.1, which is 0.026, and the display step itself contributes half a step of quantizing, which is 0.05, for a total of about 0.08. The true value sits between 5.02 and 5.18. Entirely above the limit. The instrument with fewer digits is the one that decides the question, because pass-or-fail against an external number is an accuracy task and Y is four times the instrument on accuracy while looking like a tenth of it on the display.

The same two instruments, the other task, the opposite answer

Now the monthly condition-monitoring reading on one machine, where the question is whether the value has moved by 0.3 units across four months. Here the comparison is internal: same instrument, same machine, same point.

Say a repeatability trial on a steady source gives X a spread of about plus or minus 0.02 units and Y a spread of about plus or minus 0.05, to which Y adds its 0.05 of display quantizing for about plus or minus 0.10 in total. Repeatability figures are frequently absent from a spec sheet, so treat these as the outcome of a trial you ran, not as published values.

A change is the difference of two readings, so the spread has to be combined, and how you combine it depends on what kind of error it is. Repeatability spreads are independent, so they combine as a root-sum-square: multiply by the square root of two rather than adding. On X that is 0.02 times 1.41, about 0.028, against a 0.3 unit movement, a ratio near 10 to 1, and the trend is unambiguous. On Y it is 0.10 times 1.41, about 0.14, against the same 0.3, a ratio near 2 to 1, which is at the edge of what anyone should call a finding. Adding them instead gives 0.04 and 0.20 and ratios of 7.5 and 1.5, which is the hard worst case rather than a likely band. Use the root-sum-square when you want a probable band and the sum when you want a bound you cannot be wrong about, and write down which one you used, because "plus or minus" and "bounded by" are different claims.

X wins this task decisively, and its 2 percent accuracy is nearly irrelevant to it. If X reads consistently 2 percent high, every monthly reading is 2 percent high, and the difference between two of them is off by 2 percent of the difference, which on 0.3 units is 0.006. That is the whole point: a consistently wrong instrument is safe for trending and unsafe for pass-fail, and the same instrument can be the right and the wrong choice on the same truck.

One condition genuinely inverts this. The moment a trend is being watched because it is approaching a published limit, the task has become both kinds at once, and it reverts to being accuracy-limited at the point where the call gets made. The workable answer is to trend on the repeatable instrument and take the pass-fail reading on the accurate one, recording which instrument produced which number, because a trend line and a limit comparison built from mixed instruments is a number nobody can defend later.

Where a repeatability number comes from

Nobody hands you one, so the properties diverge in how you learn them. Accuracy comes from the manufacturer's published statement, valid only inside its stated ambient band, frequency range and other conditions. Resolution you can read off the display in about two seconds on the range you will actually use. Repeatability you establish yourself, and it is specific to the instrument, the point on the range, the coupling method and the operator, so a figure established on a bench source is a floor rather than a promise about a field reading through a clamp or a surface probe.

A sibling HowTo owns the trial itself and the card it produces. What belongs here is the reason the trial is worth an hour: repeatability is the only one of the three properties that includes your technique, which means it is the only one you can improve without buying anything.

How to verify you got this right

Take the last pass-or-fail call your shop made against a published limit and check three things: which instrument took it, what that instrument's published accuracy works out to at that specific reading, and whether the margin to the limit was bigger than that figure. If the margin was smaller, the call was not supported, and the fix is a better instrument or a direct measurement of the small quantity, never a more careful look at the same display.

Then take your longest-running trend record and ask whether every point on it came from the same instrument. A trend assembled from whatever was on the truck that month has each instrument's individual offset baked into it as a step change, and those steps look exactly like the equipment changing.

References

• Manufacturer published accuracy statements, including the ambient band, frequency range and other stated conditions that bound the claim
• 29 CFR 1910.333(a)(1) energized-testing gate; 29 CFR 1910.333(b)(2) and 29 CFR 1926.417 electrical lockout; 29 CFR 1910.334(c)(2) pre-use inspection of instruments and leads; 29 CFR 1910.335(a) protective equipment
• NFPA 70E-2021, 120.5, adopted through an employer electrical safety program or contract; IEC 61010-1 measurement categories, binding through the instrument's listing
• See related: The Difference Between Accuracy and Resolution, which owns the uncertainty budget and the counts term; How to Tell Whether Your Instrument Can See the Difference You Care About