Psych Safety, by Iterum

Checklists and Monitoring in the Cockpit: Why Crucial Defenses Sometimes Fail

Dismukes & Berman (2010) · Untrapped deviations on line flights

Dismukes & Berman · Primary Research · Safety & Error, Voice & Silence, Power & Equity · NASA Technical Memorandum NASA/TM-2010-216396 · 2010 · Open access

Open this neighbourhood on the live map →

A jumpseat study of how checklists and monitoring, two essential defences against equipment failures and pilot errors, perform on routine airline flights, and why they sometimes fail. Its sharpest result is how little was trapped: only 18 per cent of the deviations observed, even those that were clearly errors, were caught and corrected or even discussed, which Dismukes and Berman call 'a disquieting finding'.

Berman, an airline pilot experienced as a Line Operations Safety Audit observer, rode the jumpseat on 60 normal flights at three airlines, aiming to record every observable deviation from procedure, however minor. They write deviation rather than error because some deviations may have been appropriate or hard to avoid. There were 899, 194 in checklist use, 391 in monitoring and 314 in primary procedures, a mean of 15.0 per flight, though per opportunity the rate was probably well under one per cent. The commonest checklist deviations were running a flow-check as read-do and responding without looking; over half the monitoring deviations were late or omitted callouts.

Trapping ranged from 35 per cent of primary procedure deviations to 14 per cent of checklist and 6 per cent of monitoring deviations. Captains as monitoring pilot trapped 27.9 per cent of the flying first officer's deviations, first officers as monitoring pilot 12.1 per cent of the captain's, a significant difference. The data do not reveal whether first officers were less likely to notice or to challenge; taken with flight simulation studies, they read it as some deviations going unnoticed and first officers who notice being less comfortable challenging. Captains too failed to catch most deviations. Crews on their first flight or day together made more deviations, though by category the difference was significant only for primary procedures; they suggest pilots may become more comfortable challenging as they settle in. In a 31-flight sample 88.8 per cent of deviations had no discernible outcome beyond weakened safeguards, but 44 led to an undesired aircraft state.

The stated limits: one observer, so reliability cannot be assessed; uneven familiarity with the aircraft and procedures; a small sample; no chance to ask crews about their deviations; and cognitive accounts they call 'somewhat speculative, but arguably plausible'. Simply telling first officers to challenge, they argue, is not enough to counter their hesitation.

The term psychological safety does not appear. They commend a captain's briefing, 'I expect I will make errors on this flight', which they say gives the first officer 'permission to speak up'. We read the gap between noticing a deviation and voicing it as where psychological safety bears on monitoring.

Dismukes, R.K. and Berman, B. (2010) Checklists and Monitoring in the Cockpit: Why Crucial Defenses Sometimes Fail. NASA Technical Memorandum NASA/TM-2010-216396. Moffett Field, CA: NASA Ames Research Center.

Fog at Los Rodeos, a Learning Lab session on the Tenerife disaster (1977), draws on this paper: run it with your team. Open the Learning Lab →

Save this paper to a bibliography, keep notes on it, and export citations. Create a free account →

Related psychsafety.com articles (5)

Related papers (4)

View the source paper →