Last week, I covered the sequence of events that led to a Boeing 767-300 ER wheels-up landing at Warsaw airport. A hydraulic hose connecting the hydraulic system on the right leg of the main landing gear failed as they departed Newark. The flight crew decided to continue to Warsaw but once there, they found that the alternate landing gear extension system had also failed and they were forced to land on the runway with no gear.

One thing I hadn’t noticed last week was that the flight was on a public holiday. The 1st of November is Wszystkich Swietych or All Saints Day, one of the most important celebrations in Poland. Thousands of people visit cemetaries where they decorate graves and tombs with candles and flowers and wander the illuminated paths until late in the night. The appendix to the report mentions that the airport and the operations centre would have been operating with fewer staff than usual.

The landing itself was flawless but, once investigators arrived, they found that a circuit breaker was popped out. As the aircraft was being lifted off of the runway, they pushed it in and used the lever. The alternate landing gear extension system worked just fine.

This week, I’d like to follow along the lines of inquiry by the investigators, because it makes for quite an interesting case study.

The circuit breaker in question was C829 which is located on panel P6-1.

C829 protects 13 circuits, including the alternate landing gear extension system. That was one mystery solved: the circuit breaker was out which meant that the alternate landing gear extension system had no power.

So, there are a couple of different possible explanations as to why the circuit breaker was popped out

1) the circuit breaker had malfunctioned and popped out without reason.
2) the circuit breaker had too much current flowing through it and was working as designed, protecting the circuit.
3) a person pulled out the circuit breaker to check or reset it and didn’t push it back in.
4) a person unintentionally popped out the circuit breaker and no one noticed.

There’s another key question related to all this:

• Was circuit C829 active during the flight and if so, could they find any activity that could have caused it to trip?

Investigators now had a number of premises which they could test by attacking the problem from different perspectives.

The circuit breaker itself did not show any sign of a malfunction and worked perfectly well once pushed back in, so investigators could quickly dismiss the idea that it popped out without reason.

It was not so easy to determine if the circuit breaker was pushed in during the flight, as it turned out that most of the functions protected by that circuit breaker were for special circumstances or stand-by circuits: if the circuit breaker had been off for the entire flight, it would not make make any difference that the crew would notice. Each of the functions were activated under very specific conditions, none of which arose other than the need for the alternate landing gear extension at the end of the flight.

There was only one circuit which C829 protected which had any use: the BUS POWER CONT UNIT (BPCU). And as it happens, that unit recorded some malfunctions specifically to do with its power supply. The BPCU had continuous power while the engines were on but when the flight crew shut down the engines at the end of the flight, the BPCU completely lost power. This told investigators two facts: the C829 circuit breaker was already off when the engines were shut down and that the BPCU did not cause the circuit breaker to trip.

Each of the thirteen circuits protected by the C829 circuit breaker has its own circuit breaker with rated currents from 2,5A to 7,5A, that is, much less than 25A. So if one of the circuits tripped C829 circuit breaker, it should have tripped its own circuit breaker first. C829 is a failsafe where multiple circuits acting together could trip the circuit even though no single one tripped an individual circuit.

Now we’ve already established that the alternate landing gear extension is protected by C829 which has a rated current of 25A (amps). It’s also connected to a second circuit breaker, C4248 which has a rated current of 7.5A. The landing gear can only be extended in the alternate system if both circuit breakers are on/pushed in. As we know, the alternate system did not work and when the aircraft landed, the C829 circuit breaker was found to be off/pulled out. The interesting thing though is that C4248 was pushed in and on, even though it had a lower rated current. This told investigators another important thing: the alternate landing gear extension did not cause C829 to trip.

More importantly, none of the thirteen circuit breakers protected by C829 had tripped so it is extremely unlikely that several of the branch circuits marginally overloaded at once, not enough to trip their own breakers but drawing in enough total power to trip the main.

The testing confirmed that all of the components of the alternate landing gear extension system were mechanically and electrically fit. No evidence of an electric overload occurred and there was no mechanical reason why the C829 circuit breaker had tripped.

Note that none of this information helps us to determine when the circuit breaker (was) popped out.

That leaves us with human factors: someone pulled out the circuit breaker, either intentionally or accidentally.

1) the circuit breaker had malfunctioned and popped out without reason
2) the circuit breaker had too much current flowing through it and was working as designed, protecting the circuit
3) a person pulled out the circuit breaker to check or reset it and didn’t push it back in
4) a person unintentionally popped out the circuit breaker and no one noticed

The thing is, C829 is located in the P6 panel, in the footwell of the right seat. The panel is low and inaccessible: you can’t tell that the breaker is popped out unless you are actively checking that panel. And as we saw above, the circuits protected by C829 aren’t used during normal flight, so there was nothing that would make the problem visible to the crew.

The pilots knew how to use the alternate landing gear extension system and clearly attempted to use it which is strong evidence that the circuit breaker was already out during approach. Both pilots were also clear that the pre-flight check was completed normally and that there were no irregularities. So there’s reason to believe that C829 was active at the time. In the end, it’s not possible to prove if the circuit was already popped out before the flight began or it was inadvertently popped out during the flight.

Both pilots were clear that they had not pulled out the circuit breaker to check or reset it. There was no one else who was in a position to do so. This means that it is far more likely that the circuit breaker was unintentionally popped out and no one noticed. But again, it’s not possible to prove it.

Inadvertent opening C829 circuit breaker during LO 16 flight was highly probable because of the technical factors (location of the circuit breaker, lack of signaling, lack of guards) and human factors (long flight with a failure of the hydraulic system and detection of another failure during the landing approach, which is the most difficult phase of the flight).

The Commission took the above circumstances into consideration, but stated that it can not be determined that the crew inadvertently set OFF the C829 circuit breaker during flight LO 16 from Newark to Warsaw.

However, a simple look at the location of the circuit breaker makes for a pretty convincing case that this is what happened.

The P6 panel, located in the right-seat footwell, is vulnerable to physical contact from feet and flight bags. It is clearly very easy to accidentally knock the lower circuit breakers without realising.

In fact, the circuit breakers in that location were so often knocked by luggage, feet and cleaning services that a number of 767 operators had complained to Boeing about the problem.

In response, Boeing developed a panel guard to protect the circuit breakers at the lower part of the panel. This panel guard was offered to operators as a pay-for add on and was not seen as urgent. Later Boeing 767s had this guard included in production but that was way too late for the aircraft in question. The Boeing 767 that day did not have the guard and the lower circuit breakers weren’t protected, including C829.

As mentioned in the comments last week, the location of the circuit breaker also makes it hard to see – the investigators described the panel as an ‘extremely peripheral location’.

However, once they knew that the alternate landing gear extension system was not working, it seems an obvious thing to check. It’s one thing to knock into it with your foot or a bag, another not to have noticed at all. How did this happen?

When the landing gear did not extend, the crew assessed the situation and tried to find out more information so that they could solve the problem. They checked the Quick Reference Handbook and made sure that their execution of the landing gear extension procedure for the alternate system was correct. They followed the HYDRAULIC SYSTEM PRESSURE checklist which also included ALTN GEAR EXTEND but the procedure’s next step was to confirm that the GEAR DOWN lights illuminated. As the GEAR DOWN lights did not illuminate, the crew couldn’t proceed to the next step, which was to move the landing gear level to the down position.

As we saw last week, there was no further help from the handbooks or the checklists – everything seemed to lead to a dead end, with no advice for what to do if the landing gear simply did not extend.

In the case of a direct threat to the lives of the crew and passengers, a pilot is looking for information to make a proper analysis of hazardous circumstances and take an action based on his best judgment. However, under stress the processes of recalling information can be impeded and subject to disorder. In such a situation only actions based on checklists contained in QRH ensure execution of all actions in a proper order and flight crews are trained to follow the checklists and rely on them in non-normal situations.

Having been let down by the processes and procedures in place to deal with unexpected issues such as this, the flight crew’s situational awareness would have abruptly decreased. They had no instructions or information that could provide a solution to the situation they found themselves in.

Add to that the fact that the flight crew were at the end of a long flight which they spent being aware that their aircraft had a major hydraulic malfunction. They were landing in the early hours of the morning Polish time, during the low-point of their circadian rhythm (between 3am and 6am). In a difficult phase of flight (the approach), they had a surprising and unexpected circumstance arise. Human factors analysis has already taught us that in this type of circumstance, cognitive processes become limited as field of attention narrows. People tend to over-focus on the central information and they struggle to pick up additional information. Specifically, stimuli occurring in the peripheral field of vision are not noticed – like a panel in the footwell.

The cockpit voice recordings show that the pilots were both calm and expressing themselves clearly as they discussed the problem. The ground engineer advised that the problem might be resolved by resetting the circuit breaker, which the first officer apparently did, but it didn’t help. The landing gear still did not extend.

The problem was ground engineer had specifically referred to circuit breaker C4248. There was no mention of the C829 circuit breaker. So although the first officer followed the recommendation, it did nothing to solve the problem. The first officer pulled out and reset the C4248 circuit breaker and reported to the operations centre that the circuit breakers had been checked.

If the ground engineer had been allowewd more time for problem solving, rather than running around the airport trying to find a working radio with which he could talk to the crew, he would have analysed the diagram of the alternate landing gear extension system and seen the reliance on C829. However, the aircraft could not keep circling indefinitely. He never got the chance.

As the stress levels and time pressure increased, the flight crew became less and less able to receive and process information. At the same time, the captain was focused on the aircraft flight controls and unable to monitor the first officer’s actions effectively.

The first officer checked the switch of the alternate landing gear extension and checked the circuit breaker on the P-11 panel. Then he left his seat, removing his headphones, in order to check the P-6 panel, while the captain relayed the instructions from the ground engineer.

The landing gear did not extend.

The captain asked the first officer to check the circuit breaker panel again, this time with the chief flight attendant in the cockpit. The first officer got out of his seat, pushing it back so that he could get on his knees to check the panel, the only way to see it. He did not see the circuit breaker was out. The Chief Flight Attendant looked as well and did not see anything.

But here’s the view of panel P6-1 with a briefcase in the footwell. As you can see, that particular panel is completely obscured. Without an active decision to check that circuit breaker on that panel, it was easy to miss that it was popped out. Even having knelt on his knees to check (and presumably moved the briefcase), the position of circuit breaker C829 at the bottom left corner is very close to the side of the pilot’s seat and in a peripheral position.

By now, they’d been circling for over an hour. They started by troubleshooting the problem and going through the checklist. When they informed air traffic control, two F-16s were sent up to verify that the landing gear had not extended. When they asked the operations centre for expert assistance, they had to wait for twenty minutes for the engineer to find a radio. Finally, when his advice did not help them to fix the problem, the time had come for a decision. They were going to have to land the plane without the landing gear.

As the flight crew prepared for the emergency landing, their voices became more strained and their tone impatient. The captain, as Pilot Flying, focused on the flight controls. Later, he conceded that because of this, he did not oversee the First Officer’s cycling (resetting) of the circuit breakers. The captain could not see panel P6 from his seat.

There’s no sensible reason why the flight crew would have intentionally left the circuit breaker pulled out. The only remaining scenario is that the captain couldn’t see it and the first officer (and the cabin crew in the cockpit) simply didn’t notice it, even when they were looking at that panel.

The analysis of the psychological issues in the cockpit focuses on “hidden factors” – those human factors caused by people removed from the actual emergency, who never have to deal with the results of their failures. In this case, we can see a number of them:

• The Quick Reference Handbook had no reference to a failed landing gear extension

• The checklists ended with no advice or further resource for crew who could not extend their landing gear

• The pilots’ training did not include simulations for dealing with the situation

• The operations centre did not have a structure in place for dealing with a crisis situation

Then on top of that we have the human factors in the cockpit.

• The first officer was convinced he had checked panel P6-1 twice and yet clearly he never spotted the circuit breaker which by now had clearly popped out

• The captain, as pilot flying, was overly focused on a perfect fulfilment of his primary duties and did not, or could not, monitor the first officer’s actions. That is to say, there was no effective cross-checking as would be normal in an emergency scenario.

The official report went into great detail about the human factors before concluding with the facts that they’d been able to establish.

Causes of the Accident

1. Failure of the hydraulic hose connecting the hydraulic system on the right leg of the main landing gear with the center hydraulic system, which initiated the occurrence.
2. Open C829 BAT BUS DISTR circuit breaker in the power supply circuit of the alternate landing gear extension system in the situation when the center hydraulic system was inoperative.
3. The crew’s failure to detect the open C829 circuit breaker during approach to landing, after detecting that the landing gear could not be extended with the alternate system.

Factors contributing to the occurrence were as follows:
1. Lack of guards protecting the circuit breakers on P6-1 panel against inadvertent mechanical opening; from 863 production line the guards have been mounted in the manufacturing process (SP-LPC was 659 production line).
2. C829 location on panel P6-1 (extremely low position), impeding observation of its setting and favoring its inadvertent mechanical opening.
3. Lack of effective procedures at the Operator’s Operations Centre, which impeded specialist support for the crew.
4. Operator’s failure to incorporate Service Bulletin 767-32-0162.

There were nine safety recommendations which can be quickly summarised:

• Boeing should review and expand their checklists, including a checklist for crew suffering a total failure of the landing gear extension systems. They should also issue a mandatory bulletin for the guard for panels in areas with direct contact with shoes, cleaning equipment and luggage.

• LOT Airlines should also check and modify their checklists and put guards on the panels in the footwell.

• Warsaw Chopin Airport should work on their procedures for coordination and transport of passengers after an evacuation.

I’m a bit surprised they didn’t also make recommendations regarding the evacuation procedure as followed (or not) in the cabin, however, the report concludes that the evacuation was successful and that the cabin crew did well to prevent panic on board and to be flexible in an untypical situation. I think it’s likely that LOT will nevertheless be looking at their training and procedures in this area.

As of November 2017, Boeing had verified that the checklists had been reviewed although they disagreed with many of the specific modifications asked for by the report. They did release a service bulletin for all operators to add circuit breaker guards consistent with those being installed in production on later aircrafts.

LOT Airlines also reviewed their checklists and made a number of changes in response to the recommendations.

Warsaw Chopin Airport undertook a full analysis of the incident which led to a number of changes being implemented at the airport. In addition, they set up specific components to mimic such a scenario and test reaction time and organisation as a part of their quarterly exercises. And finally, they now have the ability to remove Boeing 787s (and thus presumably 767s!) in case another one should become disabled.

And yet, even taking all that into account, I’m still impressed with the flight crew that day.

The captain landed gear up on the Boeing’s belly so gently, the senior cabin crew member thought the wheels much have come down after all.

That’s one helluva landing.