Do checklists work? What the surgery, ICU and Ontario studies measured

A checklist is the cheapest safety tool there is: a page, a pen, thirty seconds. It is also easy to dismiss as box-ticking. So the useful question isn't whether checklists are good in general. It is: when someone actually measured, what changed?

Four large studies give real numbers. Three found big drops in deaths, infections and complications. One, covering more than 200,000 operations, found almost nothing. Read together, they say a lot about when a checklist works and when it is just paper. At the end we hold our own 50 checklists up against the same rules.

Where the checklist came from: 30 October 1935

At Wright Field in Dayton, Ohio, the US Army Air Corps was choosing a new bomber. Boeing's Model 299, the prototype of the B-17, climbed off the runway, stalled and crashed. Major Ployer "Pete" Hill and Boeing test pilot Leslie Tower died of their injuries. The cause was a single forgotten step: the gust locks, which hold the control surfaces still while the plane is parked, were still engaged. A January 1936 report put it plainly: "pilots taking off with locked controls."

Nobody on board was careless or untrained. The aircraft simply had more steps than an expert could reliably keep in his head. As Gawande tells the story, the Army's test pilots answered with short written check lists for take-off, flight and landing, and the B-17 went on to be built more than 12,500 times during the war. Aviation has run on checklists ever since. Medicine took another seventy years to test the idea properly.

Study 1: WHO Safe Surgery, eight hospitals, 2009

The World Health Organization drafted a 19-item Surgical Safety Checklist, read out loud at three pause points: before anaesthesia (sign in), before the first cut (time out) and before the patient leaves the room (sign out). The items are simple: confirm the patient and the site, check the oxygen monitor works, ask about allergies and expected blood loss, give antibiotics within 60 minutes before the incision, count the swabs and needles at the end, and everyone in the room says their name and role.

Alex Haynes, Atul Gawande and colleagues tried it in eight hospitals in eight cities, rich and poor: Toronto, New Delhi, Amman, Auckland, Manila, Ifakara (Tanzania), London and Seattle. They recorded 3,733 consecutive adult patients before the checklist and 3,955 after it (October 2007 to September 2008).

Deaths in hospital: 1.5% before, 0.8% after (P = 0.003).
Patients with a complication: 11.0% before, 7.0% after (P < 0.001).

Per 1,000 operations, that is 15 deaths down to 8 and 110 patients with a complication down to 70: 7 lives and 40 complications per thousand, for a list that takes a couple of minutes per operation. Relative to the baseline, deaths fell by 47% and complications by 36%.

Study 2: Keystone ICU, 103 intensive care units, 2006

Before the surgery study, Peter Pronovost's team at Johns Hopkins went after infections from central lines, the tubes put into a large vein in intensive care. The checklist had five steps for the doctor putting the line in: wash hands, use full barrier precautions (cap, mask, gown, gloves, full drape), clean the skin with chlorhexidine, avoid the groin vein where possible, and take out any line that is no longer needed. Nurses were given the authority to stop the procedure if a step was skipped.

103 ICUs, mostly in Michigan, reported 1,981 ICU-months of data covering 375,757 catheter-days.

Median infections per 1,000 catheter-days: 2.7 at baseline, 0 three months after the start.
Mean infections per 1,000 catheter-days: 7.7 at baseline, 1.4 at 16 to 18 months.
Incidence-rate ratio: 0.62 in the first three months, falling to 0.34 at 16 to 18 months, a reduction of up to 66% that held for the whole study.

To see what that means for one unit, take an ICU that has 10 central lines in at any time: 10 × 365 = 3,650 catheter-days a year. At 7.7 per 1,000 that is about 28 bloodstream infections a year; at 1.4 it is about 5. Note that this was a whole programme (training, feedback on infection rates, a culture-of-safety effort), with the checklist as its spine, not the five lines alone.

Study 3: SURPASS, six Dutch hospitals, 2010

Eefje de Vries and colleagues in the Netherlands built a longer checklist that followed the patient through the whole hospital stay, from the ward before surgery to discharge, because more than half of surgical errors happen outside the operating room. It was tried in six hospitals that already had high standards of care, with five similar hospitals as a control group that did not get it.

Patients: 3,760 before, 3,820 after.
Complications per 100 patients: 27.3 before, 16.7 after: 10.6 fewer.
Patients with at least one complication: 15.4% before, 10.6% after (P < 0.001).
Deaths in hospital: 1.5% before, 0.8% after.
Control hospitals: no change.

The control group matters. It makes it much harder to say the drop would have happened anyway, which is the weak point of plain before-and-after studies.

Study 4: Ontario, 101 hospitals, 2014, and nothing changed

In 2010 Ontario asked every hospital to use a surgical safety checklist and to report publicly how often it was used. That gave David Urbach and colleagues a natural experiment: the same checklist idea, in ordinary hospitals, at the scale of a whole province. They compared three months before and three months after each hospital adopted its checklist.

Procedures: 109,341 before, 106,370 after, in 101 hospitals.
Death within 30 days: 0.71% before, 0.65% after (odds ratio 0.91, P = 0.13; not significant).
Surgical complications: 3.86% before, 3.82% after (odds ratio 0.97, P = 0.29; not significant).

Same idea, fifty times as many operations, no measurable effect. This is the study checklist sellers tend to leave out, and it is the most useful one.

The four studies side by side

WHO Safe Surgery (2009): 8 hospitals, about 7,700 patients. Deaths 1.5% → 0.8%; complications 11.0% → 7.0%. Before/after, no control group.
Keystone ICU (2006): 103 ICUs, 375,757 catheter-days. Line infections 7.7 → 1.4 per 1,000 catheter-days. Before/after with a programme around the checklist.
SURPASS (2010): 6 hospitals plus 5 controls, about 7,600 patients. Complications 27.3 → 16.7 per 100; deaths 1.5% → 0.8%. Control hospitals unchanged.
Ontario (2014): 101 hospitals, about 215,700 procedures. Deaths 0.71% → 0.65%; complications 3.86% → 3.82%. Neither change significant.

Why the same idea worked in three studies and not in the fourth

The authors and their critics have argued about this in print for years (including a 2019 BMJ debate between Urbach and the WHO study's Haynes and Gawande). The explanations that hold up best:

1. Room to improve. Complications were 11% in the WHO hospitals and 3.86% in Ontario. A checklist catches missed steps; if few steps are being missed, there is less to catch.
2. Used, or just signed. Ontario measured outcomes, not whether teams actually stopped and read the list out loud. A list ticked at the end of the shift from memory is a record, not a check.
3. Time to settle in. Ontario looked only three months after adoption. In Keystone the infection rate kept falling for a year and a half.
4. A team behind it. Keystone gave nurses the right to stop a doctor. The WHO list makes everyone say their name. The checklists that worked changed who speaks up, not just what is written down.
5. Being watched. In a study, people know they are being observed and try harder (the Hawthorne effect). That flatters the small studies and is one reason the control group in SURPASS is worth so much.

The short version: a checklist works when the steps are really being missed, when the team stops at a fixed moment and goes through it together, and when it has been in use long enough to become normal.

Rules for a checklist that gets used

Aviation has studied checklist design longer than anyone. Asaf Degani and Earl Wiener's NASA report on flight-deck checklists (1990) and Daniel Boorman of Boeing, as quoted in Gawande's The Checklist Manifesto (2009), agree on a handful of rules:

Have a pause point. Decide the exact moment the list is used: before take-off, before the first cut, before the truck leaves the yard. A list without a moment gets skipped.
Pick the type. READ-DO: read a line, do it, tick it; for rare or stressful jobs. DO-CONFIRM: do the work from memory, then stop and confirm nothing was missed; for routine jobs done by people who know them.
Keep each block short. Boorman's rule of thumb is five to nine items per pause point, about the limit of working memory, and a minute or so to read.
Only the killer items. Steps that are dangerous to miss and that people actually do miss. Not every step of the job.
Short, plain lines. One action per line, in the words people say on the job. The WHO list reads "Is the site marked?", not a paragraph.
Test it on the job and change it. Every list in the studies above was revised after a trial run.

Our own test: 50 checklists against those rules

We counted the items in the 50 prebuilt checklists on checklists.one (October 2026), from road-accident response and forklift checks to backpacking and wedding days.

Items per checklist: 8 to 17, median 11.5, mean 12.3.
Sections (pause points): 185 in total, a median of 3 per checklist.
Items per section: 1 to 10, median 3. 184 of 185 sections have nine items or fewer.
The one section over nine: the 10 essentials in the day hike checklist, a standard list hikers already know by that name, so we kept it whole.

Whole checklists run past nine items, so we split them into sections, each one a pause point. The road accident response list has 13 items in four blocks: Scene (4: park safely, triangle out, check for fuel and power lines, engines off), Call (2), Casualties (4) and Hand-over (3). The truck pre-trip inspection follows the parts named in the US driver vehicle inspection rule, 13 items in four blocks: outside, brakes and steering, inside, sign-off.

Making one work for your team

1. Start from a list, then cut. Open a prebuilt checklist and take out every item your team never forgets. What stays is your killer items.
2. Name the pause point in the title: "Before the doors open", "Before we leave the depot".
3. Split anything over about nine items into sections, one per moment.
4. Read it together for READ-DO lists: one person reads, the person doing the work answers.
5. Give everyone a stop. Agree that anyone can halt the job if a line isn't done, as the Keystone nurses could.
6. Revisit it after a month. Add the step that went wrong; drop the one nobody ever misses.

On checklists.one you can do steps 1 to 3 in a couple of minutes: search a checklist, start it, leave items out or add your own, and send it to your team so everyone ticks the same list on their phone.

What the research does not show

That any checklist helps. The strongest result (Keystone) came with training, feedback and a change in who could speak up. The largest study (Ontario) found no effect from adoption on its own.
Proof of cause from the three positive studies. WHO and Keystone were before-and-after studies without a control group; outcomes might have improved anyway. SURPASS had controls, but in only five hospitals, and none of them were randomised.
Anything outside medicine and aviation. There are no comparable large studies for warehouses, events or travel. The rules above carry over because the failure is the same (a step forgotten under time pressure), but that is our reasoning, not a measured result.
That shorter is always better. The five-to-nine rule is a design rule of thumb from aviation, not a tested threshold.

Sources

Haynes, A. B., Weiser, T. G., Berry, W. R. et al. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine 360(5), 491-499. doi:10.1056/NEJMsa0810119
Pronovost, P., Needham, D., Berenholtz, S. et al. (2006). An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine 355(26), 2725-2732. doi:10.1056/NEJMoa061115
de Vries, E. N., Prins, H. A., Crolla, R. M. et al. (2010). Effect of a comprehensive surgical safety system on patient outcomes. New England Journal of Medicine 363(20), 1928-1937. doi:10.1056/NEJMsa0911535
Urbach, D. R., Govindarajan, A., Saskin, R., Wilton, A. S. & Baxter, N. N. (2014). Introduction of surgical safety checklists in Ontario, Canada. New England Journal of Medicine 370(11), 1029-1038. doi:10.1056/NEJMsa1308261
Urbach, D. R., Dimick, J. B., Haynes, A. B. & Gawande, A. A. (2019). Is WHO's surgical safety checklist being hyped? BMJ 366, l4700. doi:10.1136/bmj.l4700
World Health Organization. Surgical Safety Checklist (2009). who.int
Degani, A. & Wiener, E. L. (1990). Human factors of flight-deck checklists: the normal checklist. NASA Contractor Report 177549. NASA NTRS
Gawande, A. (2009). The Checklist Manifesto: How to Get Things Right. Metropolitan Books.
Flight Safety Australia (2018). One thing at a time: a brief history of the checklist. flightsafetyaustralia.com
checklists.one prebuilt checklists, item counts taken 26 October 2026. checklists.one