The Joint Authorities Technical Review (JATR) - Boeing 737 MAX Flight Control System
From SKYbrary Wiki
The Review was commissioned by the U.S. Federal Aviation Administration (FAA) as a direct consequence of their recognition in the aftermath of the two fatal accidents to Boeing 737 MAX-8 aircraft in Indonesia on 29 October 2018 and in Ethiopia on 10 March 2019, that the type certification process as applied to the flight control system of the 737 MAX-8 and MAX-9 was likely to have played a significant part in their causation.
The Review Team consisted of technical representatives from the FAA, NASA, EASA and the Civil Aviation Authorities of Australia, Brazil, Canada, China, Indonesia, Japan, Singapore and the United Arab Emirates. It was Chaired by a former Chairman of the NTSB and conducted its work between May and September 2019 before presenting its completed report to the FAA on 11 October 2019. In accordance with the instructions given by the FAA, the Review was not required to produce a Report and limited itself to documenting Observations and Findings in support of a series of Recommendations. It formally defined these terms as follows:
- An Observation is “a noteworthy fact or issue gained from the JATR team’s review of the FAA’s certification of the B737 MAX flight control system and its related interfaces”.
- A Finding is “a conclusion drawn by the JATR team based on review of design details, analyses, reports, or other factual evidence”.
- A Recommendation is “a proposed action for the FAA to consider and is intended to identify ‘what’ is to be done, as opposed to ‘how’ actions are to be accomplished” and are “based on the JATR team’s Findings and Observations”.
It was noted that not all Findings and Observations had necessarily resulted in Recommendations.
A total of 12 Main Recommendations were made.
The First 5 Recommendations concerned “The Certification Process” and were as follows:
- The Changed Product Rule
Based on the JATR team’s observations and findings related to the application of the Changed Product Rule to the certification of the flight control system of the B737 MAX, JATR team members recommend that the FAA work with other civil aviation authorities to revise the harmonized approach to the certification of changed products. Changed Product Rules (e.g., 14 CFR §§ 21.19 & 21.101) and associated guidance (e.g., Advisory Circular 21.101-1B and FAA Orders 8110.4C and 8110.48A) should be revised to require a top-down approach whereby every change is evaluated from an integrated whole aircraft system perspective. These revisions should include criteria for determining when core attributes of an existing transport category aircraft design make it incapable of supporting the safety advancements introduced by the latest regulations and should drive a design change or a need for a new type certificate. The aircraft system includes the aircraft itself with all its subsystems, the flight crew, and the maintenance crew.
These Changed Product Rule revisions should take into consideration the following key principles:
- A comprehensive integrated system-level analysis recognizing that in this complex interactive system, every change could interact with other parts of the system.
- The assessment of proposed design changes on existing systems at the aircraft level includes using development assurance principles, system safety principles, and validation & verification techniques. The level of assessment should be proportional to the impact of the change at the aircraft level.
- The consideration of training and qualification of flight and maintenance personnel, as well as detailed explicit procedures for the safe operation of the aircraft. [Recommendation R1]
- Development and use of up-to-date requirements and practices
Based on the JATR team’s observations and findings related to the regulations, policy, and compliance methods applied to the B737 MAX, JATR team members recommend that the FAA update regulations and guidance that are out of date and update certification procedures to ensure that the applied requirements, issue papers, means of compliance, and policies fully address the safety issues related to state-of-the-art designs employed on new projects. JATR team members also recommend that the FAA review its processes to ensure that regulations and guidance materials are kept up to date. [Recommendation R2]
- Consistent interpretation and application of requirements
Based on the JATR team’s observations and findings related to the certification of the B737 MAX flight control system and related interfaces, JATR team members recommend that the FAA review the B737 MAX compliance to 14 CFR §§ 25.1329 (Flight Guidance System), 25.1581 (Airplane Flight Manual – General), and 25.201 (Stall Demonstration) and ensure the consistent application and interpretation of regulatory guidance material for the system safety assessment, handling qualities rating method, and conformity requirements for engineering simulators and devices. Should there be a non-compliance, the root cause should be identified and measures implemented to prevent recurrence. [Recommendation R3]
- Changes during the certification process
Based on the JATR team’s observations and findings related to the FAA type certification process, JATR team members recommend that the FAA review and update the regulatory guidance pertaining to the type certification process with particular emphasis on early FAA involvement to ensure the FAA is aware of all design assumptions, the aircraft design, and all changes to the design in cases where a changed product process is used. The FAA should consider adding feedback paths in the process to ensure that compliance, system safety, and flight deck/human factors aspects are considered for the aircraft design throughout its development and certification. [Recommendation R4]
- Delegation of Certification Authority
Based on the JATR team’s observations and findings related to FAA’s oversight by the Boeing Aviation Safety Oversight Office (BASOO), JATR team members recommend that the FAA conduct a workforce review of the BASOO engineer staffing level to ensure there is a sufficient number of experienced specialists to adequately perform certification and oversight duties, commensurate with the extent of work being performed by Boeing. The workforce levels should be such that decisions to retain responsibility for finding compliance are not constrained by a lack of experienced engineers.
The FAA should review the Boeing Organization Designation Authorization (ODA) work environment and ODA manual to ensure the Boeing ODA engineering unit members (E-UMs) are working without any undue pressure when they are making decisions on behalf of the FAA. This review should include ensuring the E-UMs have open lines of communication to FAA certification engineers without fear of punitive action or process violation. [Recommendation R5]
The next 3 Recommendations concerned the “Integrated Approach to Development and Certification” and were as follows:
- Holistic, integrated aircraft-level approach
Based on the JATR team’s observations and findings related to the design process of the flight control system and the related system safety assessments for the B737 MAX, JATR team members recommend that the FAA promote a safety culture that drives a primary focus on the creation of safe products, which in turn comply with certification requirements. Aircraft functions should be assessed, not in an incremental and fragmented manner, but holistically at the aircraft level. System function and performance, including the effects of failures, should be demonstrated and associated assumptions should be challenged to ensure robust designs are realized. The safety analysis process should be integrated with the aircraft development assurance process to ensure all safety requirements and associated assumptions are correct, complete, and verified. The FAA should encourage applicants to have a system safety function that is independent from the design organization, with the authority to impartially assess aircraft safety and influence the aircraft/system design details. Adoption of a safety management system is one way this can be achieved. [Recommendation R6]
- Human Factors
Based on the JATR team’s observations and findings related to human factors-related issues in the certification process, JATR team members recommend that the FAA integrate and emphasize human factors and human system integration throughout its certification process. Human factors-relevant policies and guidance should be expanded and clarified, and compliance with such regulatory requirements as 14 CFR §§ 25.1302 (Installed Systems and Equipment for Use by the Flightcrew), 25.1309 (Equipment, Systems, and Installations), and 25.1322 (Flightcrew Alerting) should be thoroughly verified and documented. To enable the thorough analysis and verification of compliance, the FAA should expand its aircraft certification resources in human factors and in human system integration. [Recommendation R7]
- Development Assurance
Based on the JATR team’s observations and findings related to the development assurance process applied to the design of the flight control system of the B737 MAX, JATR team members recommend that the FAA ensure applicants apply industry best practice for development assurance, including requirements management, visibility of assumptions, process assurance activities, and configuration management. The FAA should ensure achievement of the close coupling that is required between the applicant safety analysis process and the development assurance process to classify failure conditions and derive the level of rigor of design development and verification. A current example of industry best practice is SAE International’s Aerospace Recommended Practice 4754A (ARP4754A).
The FAA should review and amend Advisory Circular 20-174 to clearly articulate the principles of ARP4754A, promoting industry best practice for development assurance of aircraft and aircraft systems to address applicants’ design trend of increasing integration between aircraft functions and systems. [Recommendation R8]
The next 3 Recommendations concerned the “Impact of Design Changes on Operations and Training” and were as follows:
- Impact of product design changes on operations
Based on the JATR team’s findings and observations related to the operational design assumptions of crew response applied during the certification process for the flight control system of the B737 MAX, JATR team members recommend that the FAA require the integration of certification and operational functions during the certification process. The FAA should be provided all system differences between related aircraft in order to adequately evaluate operational impact, systems integration, and human performance. [Recommendation R9]
- Impact of product design changes on flight crew training
Based on the JATR team’s findings and observations related to flight crew training, JATR team members recommend that the FAA require a documented process to determine what information will be included in the Airplane Flight Manual, the Flight Crew Operating Manual, and the Flight Crew Training Manual. The FAA should review training programs to ensure flight crews are competent in the handling of mis-trim events. [Recommendation R10]
- Impact of product design changes on maintenance training
JATR team members recommend that the FAA conduct a study to determine the adequacy of policy, guidance, and assumptions related to maintenance and ground handling training requirements. [Recommendation R11]
The final Recommendation concerned “Post-Certification Activities” and was as follows:
- Post-certification corrective actions and data sharing
JATR team members recommend that the FAA review its policies for analyzing safety risk and implementing interim [[Airworthiness Directive|airworthiness directive] action following a fatal transport aircraft accident. The FAA should ensure that it shares post-accident safety information with the international community to the maximum extent possible. [Recommendation R12]
The Joint Authorities Technical Review (JATR) - full text
- NTSB Safety Recommendations Report Arising from the Boeing 737 MAX-8 Fatal Accidents in 2018 and 2019
- Certification of Aircraft, Design and Production
- Flight Controls
- Safety Management System
- B38M, en-route, northeast of Jakarta Indonesia, 2018
- B38M, en-route south east of Addis Ababa Ethiopia, 2019