Testing a Restore Before You Need One

Why this matters

A backup is a claim. Until you have restored something from it and opened what came back, you do not know whether it works, how old the newest copy really is, or how long getting it back takes. Shops find out on the worst possible day: the office computer is encrypted, the accounting file is corrupt, somebody deleted the customer folder, and the backup that has been "running fine" for two years turns out to have stopped in the spring when a password changed. A restore drill once a quarter turns that discovery into a calm Tuesday afternoon problem. Setting up the backups in the first place belongs to its own article; this one is about proving them. See related: The Data Backup Habit That Saves a Shop From Disaster.

Two numbers to set before you test anything

A drill needs a pass mark, and the pass mark comes from two terms worth knowing by name.

  • Recovery point objective (RPO): how much data you can afford to lose, measured as time. If your RPO for the accounting file is one day, then after a disaster you must be able to get back a copy no more than one day old.
  • Recovery time objective (RTO): how long you can afford to be without the system. If your RTO for the office computer is four hours, a restore that takes a day and a half has failed even if every file comes back.

Write both down per system before the first drill. A sensible starting point for a small shop: one day of RPO and four hours of RTO for whatever you invoice and schedule from, one week of RPO and one working day of RTO for shared documents and photos. Tune them to how much a day of lost work actually costs you to re-enter. Without the two numbers, a restore that "worked" can still be useless, and nobody will notice.

The two drills, and why both

Each quarter, run two tests:

  1. One real file. Small, fast, done in minutes. It proves the backup is capturing recent changes and that someone in the shop knows how to get a file back.
  2. One whole system, rotated through a list each quarter. The office computer one quarter, the accounting company file the next, a mailbox the next, the export from your field-service software after that. It proves the backup can bring back something big enough to matter and shows you how long that takes.

A file test alone hides the failures that only appear at scale: a backup that holds individual files but cannot rebuild a working machine, a restore that takes nine hours over the office internet line, a program that will not open its own data without a license key nobody kept. A system test alone runs too rarely to catch a backup that stopped last month.

The file drill, step by step

1. Pick a file somebody changed in the last week, chosen by someone other than the person running the drill. Not the test file you know is there. Ask the office manager for a spreadsheet she edited on Thursday. Why: a drill on a file you picked because you know it is backed up tests nothing. What breaks if skipped: you prove that an old, static file survives while the folder people actually use has not been captured since a reorganization.

2. Restore it to a different location, never over the original. A folder named for the drill date, on the desktop or a spare drive. Why: restoring over a live file replaces today's version with an older one. What breaks: the drill destroys the very work it was meant to protect.

3. Open it in the program that uses it, and check the newest entry. Do not trust a file size or a green tick. Open the spreadsheet and find the last row someone added. Open the photo. Open the PDF. Why: a backup can faithfully store a file that is already corrupt, or a zero-byte placeholder a cloud-sync folder left behind. What breaks: you record a pass on a file that would not open in a real emergency.

4. Compare the newest data against your RPO. If the last row was added Thursday and the backup's newest copy has it, the age of the restored data is under a day. If the newest copy is from three weeks ago, that is your real RPO for that folder, whatever the backup settings say.

5. Delete the restored copy. It is a duplicate of business data sitting in an odd place, which is exactly the kind of copy that later leaks or confuses someone.

The system drill, step by step

1. Restore to somewhere that is not production. A spare computer, an old laptop, a virtual machine, or a test company file in the accounting program. Never the live office computer. What breaks if skipped: a restore that fails halfway leaves you without the machine you needed to keep working.

2. Time it from the moment you start to the moment someone could actually work on it, including finding the backup login, downloading the data, installing the program and entering any license key. That end-to-end time, not the progress bar, is what to compare against your RTO.

3. Log in and do a real task. Open the accounting file and look up last month's sales total against the number on your last month-end report. Send a test email from a restored mailbox. Find a customer in the restored field-service export. Why: a system can restore every file and still not run, because of a missing license, a database that needs a service started, or a password stored nowhere else.

4. Note every manual step you had to work out. Each one goes into a short written restore guide kept with the backup credentials, in a place you can reach if the office computer itself is the thing that died.

5. Wipe the test copy when you are done, or keep it switched off if it is a deliberate standby.

What to record

Every drill leaves one entry in a running log. The log is what tells you a backup is drifting before it fails completely, and it is the evidence an insurer or a customer asks for when something goes wrong. A real entry from one quarter, with one test passing and one failing:

RESTORE DRILL  Q3          Run by: D. Okafor
                           Picked by: office mgr
FILE TEST
  Target    Schedule_Week38.xlsx (edited Thu)
  Source    cloud backup, newest copy Fri 02:00
  Restored  to Desktop/drill-Q3, opened OK
  Newest    Thu entry present
  Age       under 1 day      RPO 1 day   PASS
  Time      6 min
SYSTEM TEST
  Target    accounting company file
  Source    nightly job to office NAS
  Newest    47 days old      RPO 1 day   FAIL
  Cause     job stopped after password change
  Time      not run to completion
  Action    job fixed, re-run full test Fri

Note the fields: who picked the target, where the copy came from, how old the newest data was against the RPO, how long it took, and what was fixed. A log that says "backup tested, OK" proves nothing to anyone, including you.

A worked example, carried through

The failing entry above is from a seven-person plumbing shop. The accounting file was supposed to back up every night to a network storage box in the office (a NAS, network-attached storage). When the office manager changed the shared password in the spring, the backup job lost access and stopped. It logged an error every night that went to an email address nobody read. The newest good copy was 47 days old against a one-day RPO, so the backup was 46 days outside its own standard.

Had the computer died that week, the shop would have re-entered 47 days of invoices, payments and bills from paper and bank statements. At roughly 25 transactions a working day and about 33 working days in those 47 calendar days, that is around 825 entries, or by the office manager's estimate about two full working weeks of one person's time, during which month-end close would not happen.

The fix the same day: update the job's credentials, run it manually and confirm a fresh copy, and change the error alert to go to two people. The following Friday the shop re-ran the system test to completion on a spare laptop. The newest data was 9 hours old, inside the one-day RPO. The restore took 5.5 hours end to end, mostly installing the accounting program and finding its license key, against a four-hour RTO. That is still a fail, by 1.5 hours.

The shop had a choice there, and it is a real fork. It could accept a six-hour RTO for the accounting file, on the reasoning that invoicing can wait half a day. Or it could cut the restore time by keeping the license key and installer with the backup credentials and pre-installing the program on the standby laptop. It did the second, because it cost one hour of setup, and the next system test of that file came in at 1.75 hours.

What changes the drill

A shop whose systems are all cloud software has less to restore and a different thing to test. The software vendor backs up its own servers, but that backup exists to restore the vendor, not to give you back the job you deleted last Tuesday or your data if you leave or the vendor fails. The drill becomes: run the vendor's export, open it, and confirm you could find a customer, their equipment and their last invoice in it. If there is no usable export, that is the finding. See related: Vetting a Software Vendor Before You Sign.

A backup that sits on a drive always plugged into the office computer will usually be encrypted along with everything else in a ransomware attack. At least one copy needs to be offline or held by a backup service that keeps versions you cannot delete from the office. Include that copy in the rotation; it is the one you will actually need. See related: Ransomware Locked the Office Computer.

References

  • CISA guidance on backups and recovery planning for small organizations
  • NIST Special Publication 800-34, Contingency Planning Guide, definitions of recovery objectives
  • Your backup service's and accounting program's documentation on restore and licensing
  • See related: The Data Backup Habit That Saves a Shop From Disaster
  • See related: Ransomware Locked the Office Computer