# The 1% rule and air disaster avoidance

13 May 16

I asked my friend, Robert Dawson, to help me understand the numbers behind risk assessment and the 1% rule. Robert writes fantastic science fiction and poetry, however I contacted him in this case because he teaches mathematics in Nova Scotia at Saint Mary’s University and is absolutely excellent at explaining numbers to me.

Although he doesn’t have an aviation background, he immediately grasped the issues that I was grappling with and wrote me a detailed explanation. I’m thrilled that he was willing to expand on it so that I could share it with you. I think you’ll find his explanation as fascinating as I did.

Just about everything in life has risks attached. If you drive across North America, you have (according to various sources) about one chance in 14,000 of being killed in an accident during that trip. If you take the same trip in a commercial jet plane, the odds are about one in seven million – five hundred times less. Of course, these numbers are only averages. While everybody has about the same survival odds on the airplane, a very safe or very foolhardy driver would have different odds on the road trip. (Lists of risks like this need to be read with care. It’s easy to find lists online that compare lifetime risk of dying of heart disease with the risk of a fatal accident on a single air voyage.)

There are many things that can go wrong with a commercial air flight; the industry’s excellent safety record shows that they are all unlikely. Some of them – birds in the engine, for instance – may be hard to do much about. Others, like mechanical failure, can be made much more unlikely by foresight and careful maintenance. But incidences like the GermanWings murder-suicide remind us that one of the hardest factors to quantify is the mental and physical health of the people in the cockpit. Critical parts of the plane can tested to destruction in a laboratory, and once their life cycle is determined they can be replaced before they start to fail. It’s more complicated with human pilots.

## What risk is acceptable?

The ICAO Manual of Civil Aviation medicine starts top-down by setting acceptable levels of risk. Their round (and hence probably somewhat arbitrary) starting figure, for all catastrophic flight failures, was one disaster in ten million flying hours. Is this standard reasonable? The number above suggests that the industry can and does meet it. A pilot is permitted to fly about a thousand hours per year, and few passengers fly more; so that represents a personal risk of at most one in ten thousand years. Over a generous fifty years, that would be a lifetime risk of one chance in 200 of dying in a plane crash for somebody who flew about as much as it’s possible for a human being to fly. That’s comparable of the risk of dying by any one of drowning, poisoning, or in a fire if you stay on the ground.

The manual furthermore sets out that not more than 10% of failures should be due to a single system (eg, pilot failure of all kinds), and not more than 10% of failures of a single system should be due to a single subsystem (in this case, medical incapacitation.) The classification of systems and subsystems is of course somewhat arbitrary: if “systems” were defined coarsely enough that a plane had fewer than ten, or finely enough that some systems had fewer than ten subsystems, the requirement would be mathematically impossible to meet! But the underlying idea appears to be a good one: if more than 10% of the risk is concentrated in one system, a problem has been identified that can and should be fixed. In other words, this rule does not directly guarantee safety, but it may lead to the identification of critical points where safety can be improved. (Of course, as there is no really natural definition of what constitutes a “system” or “subsystem,” the rule could easily be gamed by regrouping and redefining systems. As a total outsider, I have no idea to what extent this takes place.) This means that only 10% of the acceptable risk of disaster can be due to pilot failure, and only 10% of that due to pilot medical incapacitation. This leads to an acceptable risk of disaster due to pilot medical incapacitation 10-9 disasters per flying hour, or one disaster in a billion flying hours.

That sounds like a lot, but a 2014 ATAG report said that there were over 100,000 commercial flights per day. If flights average two hours, that’s 75 million flying hours per year, so a billion flying hours is about 13 years’ worldwide commercial flying.

## Is this goal being met?

Within the last fifty years there have been at least five commercial flights lost due to pilot incapacitation, mostly at the beginning of the period; the per-year rate appears to have dropped significantly, and the per-flying-hour rate still more so. Overall, the rate is about one disaster per decade; arguably the current rate is significantly less, though the fortunate shortage of data makes it hard to be certain if this change is an actual phenomenon.

There were three possible suicide-by-pilot disasters on commercial flights in the late 1990s, and two considered as definite in recent years. These are not getting rarer; they also appear to come in clusters, consistent with the idea of copycat suicides.

If “pilot mental health” and “pilot physical health” are considered as separate “subsystems” the goal is, approximately, being met. If they are lumped together as one “subsystem” it is probably not.

## What is being done?

Pilot medical incapacitation is usually hard to predict: if a pilot knew she was going to have a heart attack on a particular day, she would be in the hospital ICU, not the cockpit, when it happened! To avoid disasters of this sort, little can be done but ensure that the pilot and copilot are healthy ahead of time. But what standard of health should be used? Air forces retire fighter pilots in their forties, as soon as reflexes start to slow; but, in the absence of an obvious safety argument, commercial airlines cannot (and should not) do this. Some sort of probabilistic standard has to be applied, but what?

There are, of course, many scenarios for disaster; but some – pilot and copilot having simultaneous heart attacks, for instance – are so unlikely that they may be ignored. During most phases of flight, the incapacitation or death of a single pilot, while tragic, would not lead to the loss of the plane. The most likely disaster scenario for physical incapacitation is as follows:

The pilot at the controls is incapacitated during a critical phase of flight; and the other pilot then fails to take control effectively.

The probability of this happening is found by multiplying the probability of the two events. Simulator studies have shown that in such cases the other pilot will fail to take control about one time in 400. Presumably to allow for the greater stress of a real-world situation, with a colleague’s life in danger, the ICAO replaces this by one chance in 100. (Note that this has nothing to do with the (1/10 x 1/10) above.) Thus the requirement is that

`(1/100) x (risk of pilot incapacitation at a critical time) ≤ 10-9/hr (once in a billion hours)`

so the incidence of pilot incapacitation at a critical time should be at most 10-7/hr (one in ten million hours).

Now, we must be careful here. This is not the incidence of pilot incapacitation; this is the incidence, taken over all flight hours, of events where the pilot is incapacitated and this happens at a critical phase of flight. The ICAO estimates that critical phases comprise at most 10% of flight time; thus the requirement is that the incidence of pilot incapacitation at any time during flight is at most

`(10-7/hr)/(10%) = 10-6/hr`

or one in a million hours.

Asking a doctor to make assessments like this would be like asking a pilot to estimate stall velocity in furlongs per fortnight, and just as disastrous in practice! To make the task easier, the ICAO assumes that the risk of sudden incapacitation is the same in the cockpit as elsewhere. (This is probably a conservative estimate, as heavy physical activity is a short-term risk factor.)

The doctor is asked to assess the probability that the pilot, in her current state of health, will suffer an incapacitating medical condition within the next year. There are 8760 hours in a year; rounding this to 10,000, the doctor is asked whether the risk is less than

`(10-2/yr) ≈ (8760 x 10-6/yr) = (10-6/hr)`

that is, 1% per year. (Again, this 1% has nothing directly to do with the other two one-percents above! Because all numbers are rounded to powers of ten, we can expect coincidences.) If the risk is over 1%, the pilot is grounded.

## Murder-suicide by the pilot

One weak link in this calculation is the assumption that the ability of the other pilot to take over is independent of the first pilot’s inability to fly the plane safely. In the case of a stroke or heart attack this is a good assumption. If they had both had the same lobster sandwiches from the Business Class menu, and the lobster was tainted, this assumption might not hold (you saw Airplane!, right?) More seriously, if one pilot actively wishes to crash the plane, the odds that the other pilot can somehow take control are anywhere from moderate to zero (if, for instance, he locks her out of the cabin). We lose a factor of 100 in our safety calculations. The GermanWings accident report takes this into account, and suggests an acceptable risk of 10-4 or 0.01%.

Consequently, mental incapacitation should not be treated the same way as physical incapacitation because the risks they generate cannot be mitigated in the same way by the two-pilot operation principle. Therefore, the target of acceptable risk for non-detection of a mental disorder that may result in a voluntary attempt to put the aircraft into an unsafe condition should be more ambitious than the one usually accepted for “classical” physical incapacitation risk. If one follows the calculation methodology developed in ICAO’s Manual of Civil Aviation Medicine (Doc 8984) and described in paragraph 1.17.2, a quantitative target should be lower by at least two orders of magnitude, or 0.01%.

It may be argued that is too optimistic. It does not appear to take into account that in this scenario, also, all phases of flight are equally critical. This loses a further factor of 10 in safety.

But the decision to crash the plane need not be made on the spur of the moment; a suicidal pilot may bide his time until his next flight. Thus, it may be argued that the the pilot’s time cannot be broken up into flying and nonflying, critical and non-critical. If this is accepted, the criterion of one disaster of this type per billion flying hours must be thought of simply – at 1000 flying hours per year – to one decision to commit murder-suicide per million pilot-years, (10-6/yr) and this is the only relevant criterion.

The quote above is somewhat misleading in that it compares “non-detection of a mental disorder that may result in a voluntary attempt to put the aircraft into an unsafe condition” with the risk that a detected physical disorder does incapacitate a working pilot. A doctor can reasonably estimate the second for an individual; it is not at all clear to me as a layman how the first should be assessed.

Most of the uncertainty is not in the “detection” (which, as I understand it, psychologists usually do fairly well) but in what “may result” (which is very hard to predict.) There is also the question as to how many nondomestic murder-suicides result from religious or political fanaticism rather than from any diagnosable mental disorder. Nondomestic murder-suicide is very rare among the mentally healthy, and almost as rare among those who are not. A protocol to deal with this problem will probably need to be more than a tweak to one designed for another type of illness.

Robert Dawson teaches mathematics at Saint Mary’s University in Halifax, Nova Scotia. He has written more than sixty academic articles, mostly on pure mathematics but including topics like locked gimbals, Charles Dodgson’s probability problems, and the statistics of lichen growth. In his spare time he fences, volunteers with a Scout troop, and writes science fiction: links to many of his stories can be found at http://cs.smu.ca/~dawson/Writing/.

I hope you found that as interesting as I did. Even though I looked into this in detail when I went over the GermanWings flight 9525 final report and then again when I was analysing the Mozambique flight 470 report, I had absolutely not understood it. I was especially interested to realise that the risk percentage was based on phases of critical flight (departure and approach), which hadn’t occurred to me.

Any questions? Leave them in the comments!

Category: Demystifying,

• Rudy Jakma says:

I will have to read it a few times before I understand it all. Very theoretical but yes, very interesting. Fascinating is a better word.
For me, it always has been something inconceivable. If I do not like being part of society inhabiting this little planet, I can use other ways to get off without taking hundreds of innocents with me.
Interesting your use of the word “she”. I think of pilots as being either male or female. And it would be interesting to do research on the ratio of male / female pilots committing suicide (and murder !) by deliberately crashing their aircraft. I cannot think of any case where a female pilot did this. Coincidence or a difference in the way females and males respond to pressure? Or are males more exposed to pressure? In other words: are there factors that are not immediately apparent, sociological perhaps? Not directly nor apparently related to their job as pilots?
The possibilities for research in this matter are far from exhausted, or so it would seem to me.
BTW: under the header “murder-suicide” you write: “if he locks her out of the cabin”. That is just the action we would WANT a suicidal pilot to take. The problem is that he lock her out of the COCKPIT (assuming that “she” is the sane cockpit crew member !).

• I agree that I can’t think of a case where a female pilot has deliberately crashed the plane. I know that there are differences in how men and women choose to commit suicide. It’s not across the board, of course, but generally women appear to use less violent methods and be more concerned about not causing problems for someone else. But it could just be a raw numbers game: pilot suicide is very rare and male pilots still greatly outnumber female ones.

• Robert Dawson says:

Whoops, should have caught that one. My error.

• John Ceresole says:

Very interesting read. Basically flying is pretty safe.

• Rudy Jakma says:

John,
Yes I agree. The people reading all these articles should keep things in perspective: Sylvia is discussing, and putting up for discussion, events that actually, I would say by definition, are extremely rare.
It also is a particular feature of aviation that events, especially those that led to incidents or near-incidents, are discussed and analysed.
The results of all this are fed into the world of aviation: designers, law makers, operators and pilots.
And that has led to an increased – and still increasing – level of safety that has surpassed every other form of transportation.
I still remember the discussions that went on during the introduction of the B 747. There were concerns that a crash of such a large aircraft would also have repercussions that could not be foreseen.
Aviation has survived the Tenerife crash, still the largest loss of life in one single aviation incident.
But the changes made as a result have been incalculable.
The public has no notion of the way the “cockpit culture” has changed, how CRM and other programs have contributed to a better interaction between cockpit crew.
Not surprisingly, KLM is one of the companies that has been one of the leaders in this field.
So yes, aviation is very safe.
This is also, probably not to a small extend, due to the frank and open discussions about why or how incidents have happened.
The results of these discussions, insofar as legal restrictions do not apply or have been lifted, are made available and disseminated through the international world of aviation.
And of course, to the public in general if they are interested.
It can lead to a fear of, not just landing but also to flying.
So we must take a step back from time to time and realise that what we read here, or on similar sites, are events compressed in time and mainly isolated by time.

• John says:

Interesting article Sylvia,
I don’t think I would have tried to understand the 1 percent rule without you explaining it in a digestible way. Thank god the risk of pilot-suicide is extremely low. (but indeed very scary). I wonder if any protocols will ever be able to stop a determined pilot. One could ask oneself if the low risk warrants introduction of these protocols in the first place.

In terms of overall safety I would like to add something to the discussion though. There are indeed a million different ways of presenting statistical risk of fatality in aviation. Airlines and “the industry” really like to put forward the fatality per lifetime risk or the risk of fatality per mile which both make aviation appear very safe due to, respectively, the few flights per lifetime and the large number of miles travelled.
The statistics per hour per person or even worse, per journey per person, show a different picture though.
Then there is also General Aviation of which I suspect many of your readers are a part of. It is a little harder to find good data and even harder to find a good analysis but I suspect, on the basis of the number of GA pilots/airplanes in my country and the yearly rate of fatal accidents that GA flying as a whole is relatively unsafe.
A quick and rather crude analysis is done here. http://www.meretrix.com/~harry/flying/notes/safetyvsdriving.html
This webpage has links to the data and agencies where the data was sourced which makes for an afternoon of interesting reading.
There are of course many caveats based on the type of GA aviation, type of driving, country (in this case the US), and the way the data has been compiled in the first place.
However if I interpret everything correctly at the very least we can state that commercial (airline) flying is a lot less safe than is usually presented to the general public and that GA flying is quite a lot more risky than most pilots probably think.
And, oh yeah, stay away from motorcycles..
Really.
If only they were not so much fun!

This site uses Akismet to reduce spam. Learn how your comment data is processed.