B773, Singapore, 2016
B773, Singapore, 2016
On 27 June 2016, a Boeing 777-300ER powered by GE90-115B engines returned to Singapore when what was initially identified as a suspected right engine oil quantity indication problem evidenced other abnormal symptoms relating to the same engine. The engine caught fire on landing. The substantial fire was quickly contained and an emergency evacuation was not performed. The cause of the low oil quantity indication and the fire was a failure of the right engine Main Fuel Oil Heat Exchanger which had resulted in lubrication of the whole of the affected engine by a mix of jet fuel and oil.
On 27 June 2016, the augmented flight crew of a Boeing 777-300ER (9V-SWB) being operated by Singapore Airlines on a flight from Singapore to Milan observed an indication of low oil quantity in the right engine after having reached their initial cruise altitude. After further abnormalities in respect of the performance of the right engine occurred, it was decided to set the right engine thrust to flight idle and return to Singapore at FL170. A subsequent landing on runway 20C after fuel dumping was followed by signs of substantial right engine distress and the outbreak of fire in and around the right engine and adjacent wing structure. The initially substantial fire was brought under control within about 5 minutes and did not affect the fuselage so it was determined that an emergency evacuation was not necessary and passengers were subsequently disembarked from the aircraft via steps. The fire effectively destroyed the right engine and parts of the right wing and ignited pooled fuel which also damaged the runway surface.
An Investigation was carried out by the Singapore Air Accident Investigation Bureau (AAIB) and subsequently published under the auspices of the Singapore Transport Safety Investigation Bureau (TSIB) which was formed on 1 August 2016 and comprises the Air Accident Investigation Bureau (AAIB) and the Marine Safety Investigation Branch (MSIB). FDR, CVR and QAR data were downloaded and all were of good quality. Video footage from airport operator and ARFF vehicle cameras were also available and these were particularly useful in helping to establishing the sequence of events.
It was noted that as the aircraft was climbing to its cruising altitude, weather was encountered which necessitated avoidance manoeuvres. Subsequently, about half an hour after take-off and level at FL 300, it was noticed that the EICAS oil quantity showed 17 units for the left engine but only 1 unit for the right engine. The crew also noticed that the right engine oil pressure was fluctuating between 65 and 70 psi and the right engine oil temperature was 10°C higher than the left engine. Both oil pressure and temperature were within the normal operating range. The crew was unable to find a procedure that addressed the low engine oil quantity situation and initiated contact with Engineering Control. After a series of discussions with them and Technical Services personnel, it was agreed that, in the absence of any other abnormal indications, it would be safe to continue the flight.
However, shortly after these conversations had finished, an unusual vibration was felt in the control column and the flight deck floor. It was found that this vibration disappeared when the thrust setting for right engine was reduced. At about the same time, a "burnt smell" was detected in the flight deck but it very quickly disappeared. Engineering Control were advised of the vibrations and it was decided that the aircraft should return to Singapore with the right engine operating at idle power. During this exchange with Engineering, the senior cabin crew advised of a burnt smell detected in the cabin, especially in the forward area where wet towels had been distributed to passengers to hold over their nose and breathe through. In response, the right engine bleed system was switched off.
The aircraft was turned around and the descent to 17,000 feet (which would have been required for single engine operations) was commenced. Once level, the right engine thrust was reduced to Flight Idle. When the cabin crew reported that the burnt smell in the cabin was still present, the right air conditioning pack and re-circulating fans were switched off and soon after this, the smell "subsided". When still about 90 minutes from Singapore, the EICAS 'FUEL DISAGREE' message appeared and the crew consulted the corresponding Checklist. This "suggested four scenarios in which a fuel leak should be suspected" and when the flight crew should perform the FUEL LEAK Checklist, one of which was when the Fuel Remaining figure based on tank contents was less than the FMS 'Fuel Calculated' figure based on independent methodology which was the case by a margin of 4 tonnes. However, after considering the unusual situation they were in - one engine at idle thrust and no longer on the flight planned route - and making some approximate calculations of their own, the crew "concluded that the FUEL DISAGREE Message was a spurious one and that there was no need to proceed with the FUEL LEAK Checklist".
After dumping 41.5 tonnes of fuel to bring the landing weight within the AFM maximum, an uneventful approach was flown to a touchdown 4 hours 25 minutes after take-off. Then, "about 20 seconds after the thrust reversers on both engines were deployed, the occupants in the cabin heard two loud bangs, accompanied by two flashes, originating from the right engine area. At the same time, the flight crew heard a soft thud". ARFF personnel who were on precautionary standby for the landing saw fire at the right engine and informed TWR who informed the flight crew and told them to stop the aircraft on the runway at RET E7. No engine fire warnings were annunciated in the flight deck.
The first ARFF tender arrived at the aircraft after about a minute and began discharging foam at the right engine. The Fire Commander and the Captain quickly established direct contact on the designated emergency radio frequency. On being told that the fire crews were "still trying to contain the fire", the Captain asked for confirmation that an emergency evacuation from the left side should be commenced but was told to standby by the Fire Commander who reported afterwards having been confident that the fire would not affect the fuselage and could be brought under control. After about three minutes, this had been achieved, although there was a brief flare up three minutes after that which was quickly dealt with. Disembarkation of all occupants began 20 minutes after landing using mobile stairs positioned at door 1L and took 20 minutes.
Substantial heat damage to the aircraft included to the core of the right engine and to portions of its cowlings as well as to the right wing area directly behind and outboard of the engine. There was no damage to the fuselage or left wing, nor to the right side cabin windows closest to the fire. A small area of the runway surface measuring about 2.5 metres x 1.5 metres was found to have been damaged by a fuel-fed fire beneath the right wing.
Once it was possible to begin examination of the remains of the right engine, it was quickly found that the oil tank was full of jet fuel and that fluid samples taken from various engine oil drain locations also contained jet fuel. As fuel was also found in other parts of the engine, the Main Fuel Oil Heat Exchanger (MFOHE) was removed for further testing and examination. Preliminary pressure testing confirmed that there was an internal leak between the oil and fuel flow circulation paths. A CT scan was performed on it and showed that one of the fuel tubes was cracked and displaced. The unit was then sent to the manufacturer’s facility for further examination. A test to simulate its operation with thrust set to flight idle indicated that the breach found would have leaked fuel at a rate of around 14kg/minute. When part of the outer casing of the unit was removed, the cracked tube was visible - see the illustration below.
It was found that the GE-90-115B engine had a history of leaking fuel tubes in its MFOHEs despite this component being assigned an unlimited life with no periodic inspection requirements. Investigation of the cause of cracked tubes eventually concluded that a likely cause was variation in their crimping due to the use of hand rather than machine tools which could make some tubes vulnerable to cracking caused by stress concentrations created at the support plate hole edges (again, see the MFOHE illustration below). However, on the basis that the actual damage and consequences of these occurrences, which had included a minor fire in 2014, was not 'serious', the issue of SB 79-0034 in December 2014 only required the removal of the MFOHE for modification at the next scheduled off-wing engine maintenance input. The engine involved in the investigated event had been overhauled 9 months prior to the issue of the SB and had not reached the next scheduled input when the accident occurred.
Subsequently, in 2016, GE discovered that unintended diffusion bonding - "a process whereby similar or dissimilar metals can join under high temperature and pressure through the transfer of atoms at the interface between the metals" could occur during the MFOHE manufacturing process and could weaken areas where there was close contact between the fuel tubes and the support plates thus identifying another potential origin of leaking fuel tubes.
Given the identification of the MFOHE fuel leak as causal and an understanding of the effects it would have had on the affected engine, the indication of an abnormally low oil quantity during the investigated flight without concurrent oil temperature or oil pressure indications significantly different from normal was reconsidered. It was found from data recorded during the flight that there had been "a sudden increase of engine oil quantity from 21 to 25 units about 14 minutes into the flight" but this elevated reading had only continued for about two seconds before starting to fluctuate and eventually decreasing rapidly to the 1 unit observed by the crew all within a three minute period. It was found that this could be explained by the float in the engine oil tank - used to detect oil quantity - functioning almost normally at first but then, as the fuel circulating through the engine oil system began to foam, it quickly created a mixture of foam and fuel that together had insufficient density to support the float, which then sank to the bottom of the tank causing the low quantity indication. The crew reported that prior to first noticing the low oil quantity indication, they had been focused on weather avoidance manoeuvres which it was observed would have made it easy to miss the indicated oil quantity which occurred prior to the appearance of the low level indication. It was also concluded that the slight but sustained increase in the oil temperature indication which had accompanied the low oil quantity indication could be accounted for by the fact that fuel is not as efficient as oil for engine lubrication and had leaked into the oil system at a much higher pressure (between 400 and 1,600 psi) than of the engine oil (about 100 psi) and would have quickly displaced much of the oil in the system.
It was considered that the spread of fuel throughout the engine which would have occurred once the leak became significant would have continued throughout the return to Singapore but the absence of fire whilst the aircraft was airborne could be attributed to "the high velocity of the airflow over the exterior of the engine which prevented both the ignition and sustained combustion of the leaked fuel". However, as the aircraft landed and the thrust reversers were deployed, the airflow over the core exhaust nozzle would have been significantly reduced with the area just aft of it experiencing the most significant airflow at the same time as fuel accumulated in the fan duct was distributed over a wide area of the lower wing surface. The Investigation concluded that "the disrupted airflow, the mixture of accumulated fuel on the core exhaust nozzle and fuel in the airflow would have been sufficiently heated to the point of ignition".
The absence of any fire warning on the flight deck was explained by the fact that a fire at the exterior of the engine is highly visible, and no fire detection elements are located outside the engine cowlings so that the flight crew can expect to be alerted to such a fire only when it is seen by either those in the cabin or a person outside and near enough to the aircraft - in this case the ARFF commander who "swiftly alerted the Control Tower who in turn alerted the flight crew".
In terms of any opportunity there might be to detect the early stages of a MFOHE leak, it was found that the pre-flight check specified in the AMM involved engine oil servicing. This requires removal of the engine oil tank cap "and sniffing with the nose for fuel odour". The technician who did this check prior to the accident flight stated that he had not detected any fuel odour. It was noted that an AMM alternative to sniffing with one's nose was use of a combustible gas detector. The engine manufacturer reported having conducted an informal study in which it was found that whilst "a person was generally able to detect fuel odour in the case of a 50% fuel / 50% oil mixture, the presence of fuel could be detected in a 10% fuel / 90% oil mixture when using a combustible gas detector".
Three further aspects of the event were reviewed:
- The decision of the crew not to action the FUEL LEAK checklist was inappropriate given the indicated difference in fuel on board. However it was accepted that this checklist must be "performed with both engines maintained at the same power setting" and that they had set the right engine to idle power setting "in response to the vibration felt as advised by Technical Services personnel" and that there was no applicable procedure for a fuel leak check with the engines at different thrust settings.
- The decision to return to Singapore rather than divert en route was based on the diagnosis that there was a faulty oil quantity indication. Since the right engine oil system appeared to be operating normally, it was reasonable to assume that it was safe to continue operating the aircraft for the duration of the return.
- The decision not to evacuate was considered at least questionable, although it as accepted that "making a decision to evacuate is not always straightforward". The Operators FCTM was found to recommend that "in a situation that a persistent smoke or a fire which cannot positively be confirmed to be completely extinguished, the safest course of action typically requires […] evacuation". However, it also recommends that "pilots should utilise all available sources of information in making a decision regarding evacuation" and that "key factors to be considered include the urgency of the situation (e.g. possibility of significant injury or loss of life if a significant delay occurs)" and that "in case of doubt, an evacuation should be considered". The Manual also highlights the potential dangers of evacuating into an unknown outside environment where "fire may be spreading rapidly from spilled fuel or other flammable materials, which may endanger the people who have left the aircraft or are still on the escape slides". However, it was observed that "in this occurrence, there were a number of resources that were not used by the flight crew but which could have been of help" which included turning on the taxi camera system, leaning out of the right side flight deck emergency escape window and actively seeking information from the cabin crew.
Whilst accepting the potential influence of the absence of any warning from the aircraft's fire detection system, it was noted that the crew had "depended on the Fire Commander as the sole source of information". It was noted that a narrowing of focus in stressful situations to the extent that alternatives are passively ignored is a well documented phenomenon and that since checklists cannot cover every possible emergency and abnormal situation, "it is therefore all the more critical that pilots develop the ability to always consider alternatives and other resources when they encounter a situation that is not dealt with by any checklist".
The Conclusions of the Investigation were formally summarised as follows:
- The fuel leak in the occurrence flight was a result of a cracked tube within the MFOHE of the right engine. Fuel leaked into various areas of the engine through the core of the engine and the fan duct.
- When the thrust reverser was deployed during landing, the conditions at the area aft of the turkey feather seal of the core exhaust nozzle resulted in hot surface ignition of the fuel that had leaked from the MFOHE into the various areas of the right engine.
- As the fire developed, it propagated towards the forward section of the engine and entered the core of the engine through the fan booster inlet.
- The methods that were used to detect fuel leakage into the engine system by the operator and engine manufacturer were not able to detect the fuel leak that resulted from the cracked tube within the MFOHE when it occurred in that event flight.
- The engine manufacturer issued SB 79-0034 to address the issue of possible fuel leak in the MFOHE. The deadline for incorporating the SB was determined using the (FAA process known as) Continued Airworthiness Assessment Methodologies. The action called for by the SB was not performed on the occurrence engine as the SB was issued after the engine’s last maintenance.
- In the course of the Investigation, the engine and MFOHE manufacturers have identified that diffusion bonding can potentially cause any tube in the MFOHE to crack.
- The flight crew did not execute the steps in the FUEL DISAGREE checklist correctly.
- The flight crew depended on the fire commander as their sole information source when deciding whether an evacuation was needed. Several other resources which could have aided them in making their decision were not utilised.
Safety Action taken during the conduct of and known to the Investigation included the following:
- General Electric has developed an enhanced engine monitoring algorithm that can detect a MFOHE fuel leak using post-flight data processing of engine oil system consumption, temperature and pressure. They have also developed an enhanced algorithm which is able to detect a MFOHE fuel leak in-flight based on the engine oil system temperature, pressure and quantity and this has been incorporated into the Company's overall condition monitoring system. In both cases, any operator affected will be alerted.
- Boeing has revised the Aircraft Condition Monitoring Function software in-flight procedure to incorporate the engine manufacturer’s enhanced algorithm to detect a MFOHE fuel leak in-flight. They have also developed an interim in-flight procedure to be used in the event that an in-flight MFOHE fuel leak is suspected which is intended to minimise the possibility of a fire after landing and this information has been disseminated to all affected operators.
- Singapore Airlines has completed implementation of SB 79-0034 to upgrade the affected MFOHEs on its GE90-115B engines and to date has not recorded any instances of leaks in the modified components.
- The Singapore Airport Rescue and Fire Fighting Service has made changes to its communication protocols to require its personnel use appropriate radio communication phraseology and tone in the delivery of key messages when communicating with flight crews.
A total of seventeen Safety Recommendations were made as a result of the Investigation in two batches, one issued early in the Investigation and one at its conclusion.
On 25 July 2016, 4 Safety Recommendations were issued as follows:
- that General Electric as the holder of the engine type certificate, should review the need to accelerate the implementation of the main fuel oil heat exchanger service bulletin to ensure no hazardous effect or fire can arise as a result of fuel leakage into the engine oil system. [R-2016-001]
- that the FAA should require the engine manufacturer, as holder of the engine type certificate, to review the need to accelerate the implementation of the main fuel oil heat exchanger service bulletin to ensure no hazardous effect or fire can arise as a result of fuel leakage into the engine oil system. [R-2016-002]
- that Boeing should review the need for interim operational procedures in the event a flight crew encounters a similar fuel leak situation in-flight. [R-2016-003]
- that the FAA should require the aircraft manufacturer review the need for interim operational procedures in the event a flight crew encounters a similar fuel leak situation in-flight. [R-2016-004]
On completion of the Investigation, a further 13 Safety Recommendations were made as follows:
- that Singapore Airlines should review its training programme to develop its pilots’ ability to always consider alternatives and other resources when they encounter a situation that is not dealt with by any checklist. [RA-2017-012]
- that Singapore Airlines should ensure that its pilots are able to correctly perform the actions called for in the emergency and non-normal checklists. [RA-2017-013]
- that General Electric should conduct in-depth studies to understand if cracks may develop in the crimped areas of other tubes over time. [RA-2017-0014]
- that the FAA should consider requiring the engine manufacturer to conduct further in-depth studies to better understand if there can be other ways the MFOHE can fail over the expected operating lifespan. [RA-2017-015]
- that General Electric should evaluate the need to periodically inspect the internal components of the MFOHE. [RA-2017-0016]
- that the FAA should consider requiring the engine manufacturer to evaluate the need for periodic inspection of the internal components of the MFOHE. [RA-2017-017]
- that Boeing should review the use of a combustible gas detector as the preferred means of fuel detection during engine oil servicing instead of relying on maintenance personnel’s sense of smell to detect fuel odour. [RA-2017-018]
- that the FAA should consider requiring the aircraft manufacturer review the use of combustible gas detectors as the preferred means of fuel detection during engine oil servicing instead of relying on maintenance personnel’s sense of smell to detect fuel odour. [RA-2017-019]
- that Singapore Airlines should consider the detection of fuel during engine oil servicing by using combustible gas detector as the preferred method, instead of relying on maintenance personnel’s sense of smell to detect fuel odour. [RA-2017-020]
- that the CAA Singapore should consider requiring Singapore’s airline operators to use combustible gas detector to detect the presence of fuel in their maintenance activities, instead of relying on their maintenance personnel’s sense of smell to detect fuel odour. [RA-2017-021]
- that the FAA should review its airworthiness control system to ensure that corrective actions can be implemented expeditiously to prevent the recurrence of unsafe conditions. [RA-2017-022]
- that Boeing should evaluate the need to provide guidance on how to perform a fuel leak check with the engines operated at unequal thrust. [RA-2017-023]
- that the FAA should consider requiring the aircraft manufacturer to evaluate the need for providing guidance on how to perform a fuel leak check with the engines operated at unequal thrust. [RA-2017-024]
The Final Report of the Investigation was issued on 27 February 2017.