FDAI logo   ::  Site Map  ::   
Home  |  About This Website  |  Contact Us
Home » ... » Evidence for an Issue

Evidence for an Issue 75 pieces of evidence for this issue.

understanding of automation may be inadequate (Issue #105) - Pilots may not understand the structure and function of automation or the interaction of automation devices well enough to safely perform their duties.

  1.  
  2. Evidence Type: Excerpt from Accident Report
    Evidence: "The flightcrew's situation awareness was ... compromised by a lack of information regarding the rules which governed the logic and priorities of the navigation data base in the FMS." (page 32) "... pilots are not given much information about the logic underlying much of the performance of the FMS, or shown many of the numerous options available to achieve identical goals in the FMS. This accident demonstrates that proficiency in the use of the FMS, without knowledge of the logic underlying such critical features as the design and programmed priorities of its navigation data base, can lead to its misuse. Such priorities in the system logic may result in one waypoint or fix being easily called up via the CDU by inputting simply the first letter of the name, and then selecting the nearest waypoint, at the top of the display, while another, equally important waypoint, can never be called up unless it is spelled out properly on the CDU keyboard. Such partially understood logic may partially account for the finding that use of the FMS often increases workload during periods of already high workload." (page 32)
    Strength: +2
    Aircraft: B757-233
    Equipment: FMS
    Source: Aeronautica Civil of the Republic of Colombia (1996). Controlled Flight Into Terrain, American Airlines Flight 965, Boeing 757-223, N651AA, Near Cali, Colombia, December 20, 1995. Santafe de Bogota, DC, Colombia: Aeronautica Civil of the Republic of Colombia. See Resource details

  3.  
  4. Evidence Type: Excerpt from Experiment
    Evidence: "All pilots understood that they could overpower the autopilot servos manually. A number were aware of the potential interaction between runaway pitch-trim and autopilot pitch attitude (elevator servo) inputs, whereby the autopilotdriven elevator servo masks the initial stage of the pitch trim excursion." (page 78)
    Strength: -3
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Beringer, D.B., & Harris, H.C., Jr. (1999). Automation in general aviation: Two studies of pilot responses to autopilot malfunctions. International Journal of Aviation Psychology, 9(2), 155-174. Lawrence Erlbaum Associates. See Resource details

  5.  
  6. Evidence Type: Excerpt from Survey
    Evidence: Moreover, it is essential to be able to develop a good understanding of the characteristics and functions of other collaborators (e.g. what they can do, how they do it; how they are structured; what they cannot do). Whilst the behaviour of the computer-based systems may make perfect sense to the automation logic, it does not necessarily to the pilots: … P2: “You have situations where ... you get close to the airport and the controller now vectors you ... the guy says when you’re established give me a call... [when going] from heading back to NAV ... all that happens is that the plane swings round and it is going back to where is was going before ... you run the chance of being disorientated ... before you do that you have to have moved the plane in the system onwards to the next point so you come into position. (page 4)
    Strength: +1
    Aircraft: unspecified
    Equipment: FMS & ATC
    Source: Bruseberg, A., & Johnson, P. (2004). Should Computers Function as Collaborators?. In Proceedings of HCI-Aero 2004 held in Toulouse, France September 29, 2004 to 1 October 2004. See Resource details

  7.  
  8. Evidence Type: Excerpt from Survey
    Evidence: Moreover, it is essential to be able to develop a good understanding of the characteristics and functions of other collaborators (e.g. what they can do, how they do it; how they are structured; what they cannot do). Whilst the behaviour of the computer-based systems may make perfect sense to the automation logic, it does not necessarily to the pilots: … P1: “There is one area on the 777 ... to do the VNAV approach they want you to open the speed window again. So it’s in VNAV and you open the speed window to manually set the speed, as soon as you put VNAV, the thing blanks, the speed bug jumps, it usually goes back to a lower speed ... and then you have to open it again and as soon as you open it, it’s back to whatever speed, and you watch the throttles come back. It’s messy. (page 4)
    Strength: +1
    Aircraft: B777
    Equipment: FMS VNAV
    Source: Bruseberg, A., & Johnson, P. (2004). Should Computers Function as Collaborators?. In Proceedings of HCI-Aero 2004 held in Toulouse, France September 29, 2004 to 1 October 2004. See Resource details

  9.  
  10. Evidence Type: Excerpt from Survey
    Evidence: Moreover, it is essential to be able to develop a good understanding of the characteristics and functions of other collaborators (e.g. what they can do, how they do it; how they are structured; what they cannot do). Whilst the behaviour of the computer-based systems may make perfect sense to the automation logic, it does not necessarily to the pilots: P1: “It’s understanding the technology, and if you are not doing a particular function often enough you forget ... If you can understand it, you would be able to remember it a lot longer. (page 4)
    Strength: +1
    Aircraft: unspecified
    Equipment: automation
    Source: Bruseberg, A., & Johnson, P. (2004). Should Computers Function as Collaborators?. In Proceedings of HCI-Aero 2004 held in Toulouse, France September 29, 2004 to 1 October 2004. See Resource details

  11.  
  12. Evidence Type: Excerpt from Accident Review Study
    Evidence: 4.3.1 The pilot enters 'R' to retrieve a list of waypoints. Design analysis: To communicate the waypoint to the system, the pilot is required by a procedure to type an abbreviation into the FMS, as can be found on the (paper) approach chart, which shows and legislates how to approach an airport. Due to the system logic, only inputs of the exact correct identifier can be recognised. The FMS is given the function of retrieving a list of waypoints from its database that may match the intended waypoint selection. Problem analysis: There was a mismatch between the printed approach charts and the FMS database. Since there were two waypoints with the identifier ‘R’ in same area, pilots needed to type ‘R-O-Z-O’ to get Rozo, not ‘R’ as they expected from experience and approach chart information. Collaboration analysis: The design has not allowed for the possibility that the common reference system may be faulty. Pilots needed to know what the FMS does with the instruction – they needed to understand its restriction of not truly being able to ‘guess’ from first letter, since it can only assign one meaning to one letter in a given area. Hence it could fail to match the abbreviation altogether. (page 5)
    Strength: +1
    Aircraft: B757-223
    Equipment: automation & FMS
    Source: Bruseberg, A., & Johnson, P. (not dated). Collaboration in the Flightdeck: Opportunities for Interaction Design. Department of Computer Science, University of Bath. Available at http://www.cs.bath.ac.uk/~anneb/collwn.pdf. See Resource details

  13.  
  14. Evidence Type: Excerpt from Survey
    Evidence: "About 25% or [in sic] the pilots reported experiencing some confusion, or seeing others become confused about the interaction of the autothrottles and autopilot." (page 23)
    Strength: +1
    Aircraft: B767
    Equipment: autoflight: autopilot & autoflight
    Source: Curry, R.E. (1985). The Introduction of New Cockpit Technology: A Human Factors Study. NASA Technical Memorandum 86659, 1-68. Moffett Field, CA: NASA Ames Research Center. See Resource details

  15.  
  16. Evidence Type: Excerpt from Observational Study
    Evidence: "...initial examination of the questionnaire and interview data indicated that all pilots understood they could manually overpower the AP servos [100%], and 22 were aware of the potential interaction between a runaway pitch trim motor and AP pitch-attitude (elevator servo) inputs [22 out of 29 or 75%]. Four pilots had not considered the potential interaction previously but grasped the concept immediately during the interview [4 out of 29 or 13%]." (page 165)
    Strength: -5
    Aircraft: unspecified
    Equipment: autoflight: autopilot
    Source: Damos, D.L., John, R.S., & Lyall, E.A. (1999). Changes in pilot activities with increasing automation. In R.S. Jensen, B. Cox, J.D. Callister, & R. Lavis (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology, 810-815. Columbus, OH: The Ohio State University. See Resource details

  17.  
  18. Evidence Type: Excerpt from Survey
    Evidence: "Q.33. Do you feel that you always understand your aircraft's systems and interactions?" 79.2% of the respondents answered 'No', 19.5% answered 'Yes' and 1.3% gave no response. (page 31)
    Strength: +4
    Aircraft: unspecified
    Equipment: automation
    Source: Gras, A., Moricot, C., et. al. (1994). Faced with automation. Publications de la Sorbonne. See Resource details

  19.  
  20. Evidence Type: Excerpt from Survey
    Evidence: "Q.33. Do you feel that you always understand your aircraft's systems and interactions?" 79.2% of the respondents answered 'No', 19.5% answered 'Yes' and 1.3% made no response. (page 31)
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Gras, A., Moricot, C., et. al. (1994). Faced with automation. Publications de la Sorbonne. See Resource details

  21.  
  22. Evidence Type: Excerpt from Survey
    Evidence: From the survey data: "There are still modes and features of the autoflight system that I don’t understand." On the scale in which 1= Strongly Disagree, 3=Neutral, 5=Strongly Agree, the mean pilot response was 2.48 and the standard deviation was 1.15. (page 21)
    Strength: 0
    Aircraft: B757 & B767
    Equipment: automation
    Source: Hutchins, E., Holder, B., & Hayward, M. (1999). Pilot Attitudes Toward Automation. Web published at http://hci.ucsd.edu/hutchins/attitudes/index.html. See Resource details

  23.  
  24. Evidence Type: Excerpt from Survey
    Evidence: 17 of the 30 (57%) respondents reported a 4 (= agree) or 5 (= strongly agree) with pc041 automation interaction may be misunderstood
    Strength: +3
    Aircraft: unspecified
    Equipment: automation
    Source: Lyall, E., Niemczyk, M. & Lyall, R. (1996). Evidence for flightdeck automation problems: A survey of experts. See Resource details

  25.  
  26. Evidence Type: Excerpt from Survey
    Evidence: 18 of the 30 (60%) respondents reported a 4 (= agree) or 5 (= strongly agree) with pc105 understanding of automation may be inadequate
    Strength: +3
    Aircraft: unspecified
    Equipment: automation
    Source: Lyall, E., Niemczyk, M. & Lyall, R. (1996). Evidence for flightdeck automation problems: A survey of experts. See Resource details

  27.  
  28. Evidence Type: Excerpt from Survey
    Evidence: 7 of the 30 (23%) respondents reported a 1 (=strongly disagree) or a 2 (=disagree) with pc041 automation interaction may be misunderstood
    Strength: -1
    Aircraft: unspecified
    Equipment: automation
    Source: Lyall, E., Niemczyk, M. & Lyall, R. (1996). Evidence for flightdeck automation problems: A survey of experts. See Resource details

  29.  
  30. Evidence Type: Excerpt from Survey
    Evidence: 8 of the 30 (27%) respondents reported a 1 (=strongly disagree) or a 2 (=disagree) with pc105 understanding of automation may be inadequate
    Strength: -2
    Aircraft: unspecified
    Equipment: automation
    Source: Lyall, E., Niemczyk, M. & Lyall, R. (1996). Evidence for flightdeck automation problems: A survey of experts. See Resource details

  31.  
  32. Evidence Type: Excerpt from Accident Report
    Evidence: "3. Conclusions ... 3.1 Findings ... D. Crew ... 6. The steering technique applied in the course of aircraft landing in the touchdown phase utilized the lateral bank as a countermeasure to balance lateral wind component. It resulted in touchdown on one main undercarriage leg only and in false impression on the part of the crew that touchdown was efficient. In reality the immediate start of operation of braking devices was not possible." (page 42)
    Strength: +2
    Aircraft: A320-211
    Equipment: automation
    Source: Main Commission Aircraft Accident Investigation - Poland (1994). Report on the accident to Airbus A320-211 Aircraft in Warsaw on 14 September 1993. See Resource details

  33.  
  34. Evidence Type: Excerpt from Accident Report
    Evidence: "4. CAUSES ... The AAIC determined that the following factors, as a chain or a combination thereof, caused the accident: ... 6. The CAP and F/O did not sufficiently understand the FD mode change and the AP override function. It is considered that unclear descriptions of the AFS (Automatic Flight System) in the FCOM prepared by the aircraft manufacturer contributed to this." (page 4.1)
    Strength: +5
    Aircraft: A300B4-622R
    Equipment: autoflight
    Source: Ministry of Transport Japan, Aircraft Accident Investigation Commission (1996). China Airlines Airbus Industrie A300B4-622R, B1816, Nagoya Airport, April 26, 1994. Report 96-5. Ministry of Transport. See Resource details

  35.  
  36. Evidence Type: Excerpt from Accident Report
    Evidence: "3.1.2.2 Analysis of Flight Conditions ... The procedure for performing an approach by disengaging GO AROUND mode once engaged and then engaging LAND mode is unusual in the final phase of approach. However, the fact that the crew did not change modes as intended seems to have been due to their lack of understanding of the Automatic Flight System (AFS)". (page 3.10)
    Strength: +4
    Aircraft: A300B4-600R
    Equipment: autoflight
    Source: Ministry of Transport Japan, Aircraft Accident Investigation Commission (1996). China Airlines Airbus Industrie A300B4-622R, B1816, Nagoya Airport, April 26, 1994. Report 96-5. Ministry of Transport. See Resource details

  37.  
  38. Evidence Type: Excerpt from Survey
    Evidence: "The pilots mildly rejected the comment that they have enough information about the technical complexities of the automated systems. (No 37) Although not specified on the questionnaire, these sources of information could include manuals, the conversion course and the information as presented throughout the flight management system. A number of pilot comments (in the open ended questions) reinforce this result." ... Statement 37: "I have enough information at my fingertips about the technical complexities of the automated systems." On the scale in which 1= Strongly Disagree, 25=Disagree, 50=Neither agree nor disagree, 75=Agree, and 100=Strongly Agree, the mean pilot response was 46 and the standard deviation was 24. The minimum response was 1 and the maximum was 87. (page 46-47)
    Strength: +1
    Aircraft: B767
    Equipment: automation
    Source: Morters, K. (1988). B767 Flightdeck Automation Research. Research Paper 32:420, 1-141. See Resource details

  39.  
  40. Evidence Type: Excerpt from Experiment
    Evidence: At the conclusion of the simulator session we interviewed each pilot to assess his understanding of various automation concepts. Through these interviews, we found that pilots typically could state correct expectations about common mode behavior. For example, 16 of 20 pilots indicated that they expected to see VNAV PTH as the pitch mode during cruise. However, few pilots applied this knowledge effectively during the simulator session, where they flew cruise in VNAV ALT. Also, although pilots were generally correct in the information they offered, they provided little information on subtler automation features. Detailed knowledge of VNAV SPD and VNAV ALT, in particular, was not offered. Note, however, that we recorded only what pilots volunteered, and it is possible that pilots knew these details but chose not to offer them in this setting. (page 5)
    Strength: +4
    Aircraft: B747-400
    Equipment: FMS VNAV
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. In Proceedings of the 11th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University.. See Resource details

  41.  
  42. Evidence Type: Excerpt from Experiment
    Evidence: In three separate Events (3, 7, 8), we changed the FMA artificially (and unknown to the pilot; in fact, for these cases the standard green box did not accompany the mode change). Table 3 shows the three cases where a change occurred. The second column indicates how many pilots fixated the relevant FMA while that change was in effect (note that data were not available for all pilots at this level of precision). The last column shows that in only 1 of the 32 total cases did a pilot notice that the FMA was inappropriate. Thus, even when scanning included the FMA, pilots failed to understand the implications of the FMA. (page 5)
    Strength: +1
    Aircraft: B747-400
    Equipment: automation & FMS
    Source: Mumaw, R.J., Sarter, N.B., & Wickens, C.D. (2001). Analysis of Pilots' Monitoring and Performance on an Automated Flight Deck. In Proceedings of the 11th International Symposium on Aviation Psychology. Columbus, OH: The Ohio State University.. See Resource details

  43.  
  44. Evidence Type: Excerpt from Accident Report
    Evidence: "2.2 Conclusions (a) Findings ... 7. The autopilot was utilized in basic CWS. 8. The flightcrew was unaware of the low force gradient input required to effect a change in aircraft attitude while in CWS. ..." (page 22-23)
    Strength: +5
    Aircraft: L1011
    Equipment: autoflight: autopilot
    Source: National Transportation Safety Board (1973). Eastern Airlines, Incorporated, L-1011, N31OEA, Miami, Florida, December 29, 1972. Aircraft Accident Report NTSB/AAR-73-14. Washington, DC: National Transportation Safety Board. See Resource details

  45.  
  46. Evidence Type: Excerpt from Accident Report
    Evidence: "3. CONCLUSIONS ... The flightcrew was not thoroughly knowledgeable of the aircraft's flight guidance and control system." (page 23)
    Strength: +4
    Aircraft: DC-10-30
    Equipment: autoflight
    Source: National Transportation Safety Board (1980). Aeromexico DC-10-30 over Luxembourg, November 11, 1979. Aircraft Accident Report NTSB/AAR-80-10. Washington, DC: National Transportation Safety Board. See Resource details

  47.  
  48. Evidence Type: Excerpt from Survey
    Evidence: "Further. the pilots’ expertise with the FMS, gained from having flown the Boeing 757 for several years. May have actually detracted from their ability to effectively use the FMS. The execution of the command to proceed to Romeo was consistent with the selection of the beacon that is identified first on the computer-display unit and is the beacon that the airplane is closest to. If not explicitly taught in training, the pilots would have recognized over repeated use that beacons with a common one letter identifier were presented in descending order of their proximity to the airplane. Pilots who recognized this rule could have assumed, because they were so close to Rozo, that among those beacons identified by the abbreviation "R," Rozo would be presented first. Thus, not only would the command to proceed to Romeo have been executed by a crew that had no time available to refer to a flight path display, but also by a crew that had effectively executed such commands in the past." It can be assumed that the crew’s familiarity with the FMS extended only to a portion of FMS logic, that concerning the order of presentation of the navaids, and not to another, the coding of the navaids themselves." (page 197)
    Strength: +1
    Aircraft: Boeing 757
    Equipment: automation & FMS
    Source: Noyes, J.M. & Starr, A.F. (2000). Civil aircraft warning systems: Future directions in information management and presentation. International Journal of Aviation Psychology, 10(2), 169-188. Lawrence Erlbaum Associates. See Resource details

  49.  
  50. Evidence Type: Excerpt from Incident Study
    Evidence: In our review of 282 automation-related ASRS incident reports, we found 3 reports (1%) supporting issue105 (understanding of automation may be inadequate).
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Owen, G. & Funk, K. (1997). Flight Deck Automation Issues: Incident Report Analysis. http://www.flightdeckautomation.com/incidentstudy/incident-analysis.aspx. Corvallis, OR: Oregon State University, Department of Industrial and Manufacturing Engineering. See Resource details

  51.  
  52. Evidence Type: Excerpt from resource
    Evidence: "56 pilots provided comments on Automation Surprises. These responses were coded into categories based on the primary thrust of the pilot’s response. Five categories were created with ten of the responses not fitting into one of the five categories. The categories, numbers of responses in each category, and a brief summary of the comments are provided below:...Experience (15) -- Surprises come from not understanding the system. Many errors are pilot induced and these decrease with experience. When pilots don’t know how the system works, it is easy to be surprised." (page 437)
    Strength: +1
    Aircraft: MD11
    Equipment: automation
    Source: Parasuraman, R., Mavor, A., Wickens, C.D., Danaher, J.W., & Aalfs, C. (1998). Managing the future national airspace system: Free flight or ground-based control with increased automation (panel session). In Proceedings of the 42nd Annual Meeting of the Human Factors and Ergonomic Society, 62-66. See Resource details

  53.  
  54. Evidence Type: Excerpt from resource
    Evidence: "Expedite climb. During climb-out. Pilots were cleared to climb and maintain 12,000 feet and to cross the waypoint Ventura at or below 10,000 feet. Upon reaching approximately 4000 feet, they were given the instruction to expedite their climb through 9000 feet for traffic separation. Pilots had several automation options to choose from in order to comply with this clearance. Eleven pilots used the EXPEDITE button on the FCU to engage this mode. Also, 5 pilots selected a lower airspeed on the FCU to make the airplane climb at a higher rate. The remaining 2 pilots used the vertical speed mode and dialed in a higher-than-normal rate of climb on the FCU. In the debriefing, 7 pilots were asked why they did not use the EXPEDITE mode, which was designed for this type of situation. They responded that they did not like the fact that in this mode, the automation would drastically increase the pitch angle and slow the aircraft more than they felt was necessary. In addition, some pilots knew about and disliked the fact that the EXPEDITE mode would not honor any preprogrammed constraints. Only 11 pilots (61 %) complied with the altitude constraint at the waypoint Ventura. The other 7 pilots did remember to resume “normal climb” upon reaching 9000 feet, but they selected the “open climb” mode (instead of “managed vertical navigation”), which, similar to the EXPEDITE mode, does not honor constraints programmed into the MCDU." (page 397)
    Strength: +5
    Aircraft: A320
    Equipment: autoflight FCU
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  55.  
  56. Evidence Type: Excerpt from resource
    Evidence: "Only one pilot in this study handled the go around below 100 feet AGL without any problems. He elected to stay in fully manual control of the aircraft until level-off at the acceleration altitude and then reengaged individual subsystems of the automation one after the other, each time assuring himself first that the automated system responded as expected and desired. All other pilots focused on trying to figure out why the automation did not behave as expected, and they tried to get guidance from the automation as soon as possible. For example, seven pilots 38.9%) first called for the flight directors to be turned on after initiating the go-around. even though the automation was not set up to provide any meaningful guidance Another seven pilots (38.9%) activated autothrust before selecting a target speed, and thus the approach speed became the airspeed target. The fact that most pilots hesitated to take manual control of the aircraft and instead tried to understand what the automation was doing resulted in the following problems. Six of the pilots (33.4%) exceeded 250 knots LAS (indicated air speed) below an altitude of 10,000 feet. Another two pilots (11.1%) allowed the airspeed to increase until almost reaching the maximum allowable airspeed. Another two pilots (11.1%) oversped their flaps during the go around. Finally, three pilots allowed the airspeed to increase all the way to the maximum operating speed before taking action. During the debriefing all pilots explained that they had not expected the autothrust to disengage when applying full power for the goaround. They emphasized that they were busy watching airspeed trends and altitude instead of looking at the flight mode annunciations to find out about the status and behavior of the automation." (page 398)
    Strength: +5
    Aircraft: A320
    Equipment: automation
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  57.  
  58. Evidence Type: Excerpt from resource
    Evidence: "Only one pilot in this study handled the go around below 100 feet AGL without any problems. He elected to stay in fully manual control of the aircraft until level-off at the acceleration altitude and then reengaged individual subsystems of the automation one after the other, each time assuring himself first that the automated system responded as expected and desired. All other pilots focused on trying to figure out why the automation did not behave as expected, and they tried to get guidance from the automation as soon as possible. For example, seven pilots 38.9%) first called for the flight directors to be turned on after initiating the go-around. even though the automation was not set up to provide any meaningful guidance Another seven pilots (38.9%) activated autothrust before selecting a target speed, and thus the approach speed became the airspeed target. The fact that most pilots hesitated to take manual control of the aircraft and instead tried to understand what the automation was doing resulted in the following problems. Six of the pilots (33.4%) exceeded 250 knots LAS (indicated air speed) below an altitude of 10,000 feet. Another two pilots (11.1 %) allowed the airspeed to increase until almost reaching the maximum allowable airspeed. Another two pilots (11.1%) oversped their flaps during the go around. Finally, three pilots allowed the airspeed to increase all the way to the maximum operating speed before taking action. During the debriefing all pilots explained that they had not expected the autothrust to disengage when applying full power for the goaround. They emphasized that they were busy watching airspeed trends and altitude instead of looking at the flight mode annunciations to find out about the status and behavior of the automation." (page 398)
    Strength: +4
    Aircraft: A320
    Equipment: automation & FMS
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  59.  
  60. Evidence Type: Excerpt from resource
    Evidence: "Hold present-position. Eight of the pilots (44.4%) had problems complying with the ATC clearance to hold at their present position. In this scenario, three pilots (15.2’%1)(%) tried to build the “HOLD” off of their next waypoint rather than off of the current fromwaypoint, and two pilots ( 1.1 %) entered the radial given to them by ATC under “inbound course” in the HOLD menu. In two other cases pilots failed to enter the specific HOLD parameters (distance of legs, direction of turn) requested by ATC. In addition, one pilot failed to realize that he did not have to enter the inbound course because it happened to match the default value. In the first five cases, in which the map display provided a visualization of whether or not the HOLD had been built as intended, pilots detected and recovered from their mistakes on their own. In the latter three cases, however, in which the pilots had to realize the problem based on a review of the data on the MCDU HOLD page, they needed help from the instructor to realize that a problem existed and to modify their input to resolve the problem." (page 396)
    Strength: +2
    Aircraft: A320
    Equipment: FMS & ATC
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  61.  
  62. Evidence Type: Excerpt from resource
    Evidence: "Intercepting a radial outbound. In actual line operations. very-high frequency omnirangenavigation equipment (VOR) radials are usually intercepted inbound to the station. There is a well-defined sequence of steps to program the flight management system (FMS) for such an intercept. In our scenario, however, we increased the difficulty of the task by giving pilots the unusual but possible clearance to intercept and track a radial outbound to a fix to see whether pilots understood that this clearance required a different procedure attrib-utable to the flight management system logic. Six of the pilots (33.3%) had difficulties set-ting up the automation for the intercept -- two pilots were not sure how to create the outbound fix on the radial, and in three cases pilots confused the sequence of the to- and from-way-points for the intercept. In another two cases, pilots forgot to update their current from-way-point. All pilots detected and recovered from their errors on their own based on the visualization of the programmed intercept on the map display." (page 396)
    Strength: +2
    Aircraft: A-320
    Equipment: automation & FMS
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  63.  
  64. Evidence Type: Excerpt from resource
    Evidence: "Change of runway/loss of altitude constraints. In this situation, pilots programmed the anticipated ILS approach to runway 24 L in the MCDU, including a number of ATC altitude constraints for their arrival. ATC then informed the pilot that the ILS for runway 24 L had just failed, and that they could expect an approach to runway 24 R. When the pilot changed the runway in the MCDU, his action led not only to the desired runway change but also to the loss of all altitude constraints that he entered for the originally planned approach. Four pilots never noticed that this happened and consequently failed to make their altitude constraints. In Contrast, 10 pilots realized immediately or even anticipated the loss of constraints. Another 4 pilots detected the problem when they were given the clearance to maintain 270 knots until reaching 10,000 feet. They selected 270 knots on the FCU and then looked at the map display, where they realized that the magenta indications for the programmed altitude constraints were no longer shown next to the corresponding waypoints. Of the 14 pilots who noticed the problem, only 12 recovered in time to make the constraints by reentering them. In addition, 1 of the 4 pilots who did not realize the problem still made the constraints because he was flying a descent profile that happened to lead to compliance." (page 397)
    Strength: +1
    Aircraft: A320
    Equipment: FMS & ATC
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  65.  
  66. Evidence Type: Excerpt from resource
    Evidence: "Only one pilot in this study handled the go around below 100 feet AGL without any problems. He elected to stay in fully manual control of the aircraft until level-off at the acceleration altitude and then reengaged individual subsystems of the automation one after the other, each time assuring himself first that the automated system responded as expected and desired. All other pilots focused on trying to figure out why the automation did not behave as expected, and they tried to get guidance from the automation as soon as possible. For example, seven pilots 38.9%) first called for the flight directors to be turned on after initiating the go-around. even though the automation was not set up to provide any meaningful guidance Another seven pilots (38.9%) activated autothrust before selecting a target speed, and thus the approach speed became the airspeed target. The fact that most pilots hesitated to take manual control of the aircraft and instead tried to understand what the automation was doing resulted in the following problems. Six of the pilots (33.4%) exceeded 250 knots LAS (indicated air speed) below an altitude of 10,000 feet. Another two pilots (11.1%) allowed the airspeed to increase until almost reaching the maximum allowable airspeed. Another two pilots (11.1%) oversped their flaps during the go around. Finally, three pilots allowed the airspeed to increase all the way to the maximum operating speed before taking action. During the debriefing all pilots explained that they had not expected the autothrust to disengage when applying full power for the goaround. They emphasized that they were busy watching airspeed trends and altitude instead of looking at the flight mode annunciations to find out about the status and behavior of the automation." (page 398)
    Strength: -1
    Aircraft: A320
    Equipment: automation
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  67.  
  68. Evidence Type: Excerpt from resource
    Evidence: "Expedite climb. During climb-out. Pilots were cleared to climb and maintain 12,000 feet and to cross the waypoint Ventura at or below 10,000 feet. Upon reaching approximately 4000 feet, they were given the instruction to expedite their climb through 9000 feet for traffic separation. Pilots had several automation options to choose from in order to comply with this clearance. Eleven pilots used the EXPEDITE button on the FCU to engage this mode. Also, 5 pilots selected a lower airspeed on the FCU to make the airplane climb at a higher rate. The remaining 2 pilots used the vertical speed mode and dialed in a higher-than-normal rate of climb on the FCU. In the debriefing, 7 pilots were asked why they did not use the EXPEDITE mode, which was designed for this type of situation. They responded that they did not like the fact that in this mode, the automation would drastically increase the pitch angle and slow the aircraft more than they felt was necessary. In addition, some pilots knew about and disliked the fact that the EXPEDITE mode would not honor any preprogrammed constraints. Only 11 pilots (61 %) complied with the altitude constraint at the waypoint Ventura. The other 7 pilots did remember to resume “normal climb” upon reaching 9000 feet, but they selected the “open climb” mode (instead of “managed vertical navigation”), which, similar to the EXPEDITE mode, does not honor constraints programmed into the MCDU." (page 397)
    Strength: -2
    Aircraft: A320
    Equipment: autoflight FCU
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  69.  
  70. Evidence Type: Excerpt from resource
    Evidence: "Change of runway/loss of altitude constraints. In this situation, pilots programmed the anticipated ILS approach to runway 24 L in the MCDU, including a number of ATC altitude constraints for their arrival. ATC then informed the pilot that the ILS for runway 24 L had just failed, and that they could expect an approach to runway 24 R. When the pilot changed the runway in the MCDU, his action led not only to the desired runway change but also to the loss of all altitude constraints that he entered for the originally planned approach. Four pilots never noticed that this happened and consequently failed to make their altitude constraints. In Contrast, 10 pilots realized immediately or even anticipated the loss of constraints. Another 4 pilots detected the problem when they were given the clearance to maintain 270 knots until reaching 10,000 feet. They selected 270 knots on the FCU and then looked at the map display, where they realized that the magenta indications for the programmed altitude constraints were no longer shown next to the corresponding waypoints. Of the 14 pilots who noticed the problem, only 12 recovered in time to make the constraints by reentering them. In addition, 1 of the 4 pilots who did not realize the problem still made the constraints because he was flying a descent profile that happened to lead to compliance." (page 397)
    Strength: -3
    Aircraft: A320
    Equipment: FMS & ATC
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  71.  
  72. Evidence Type: Excerpt from resource
    Evidence: "Expedite climb. During climb-out. Pilots were cleared to climb and maintain 12,000 feet and to cross the waypoint Ventura at or below 10,000 feet. Upon reaching approximately 4000 feet, they were given the instruction to expedite their climb through 9000 feet for traffic separation. Pilots had several automation options to choose from in order to comply with this clearance. Eleven pilots used the EXPEDITE button on the FCU to engage this mode. Also, 5 pilots selected a lower airspeed on the FCU to make the airplane climb at a higher rate. The remaining 2 pilots used the vertical speed mode and dialed in a higher-than-normal rate of climb on the FCU. In the debriefing, 7 pilots were asked why they did not use the EXPEDITE mode, which was designed for this type of situation. They responded that they did not like the fact that in this mode, the automation would drastically increase the pitch angle and slow the aircraft more than they felt was necessary. In addition, some pilots knew about and disliked the fact that the EXPEDITE mode would not honor any preprogrammed constraints. Only 11 pilots (61%) complied with the altitude constraint at the waypoint Ventura. The other 7 pilots did remember to resume “normal climb” upon reaching 9000 feet, but they selected the “open climb” mode (instead of “managed vertical navigation”), which, similar to the EXPEDITE mode, does not honor constraints programmed into the MCDU." (page 397)
    Strength: -3
    Aircraft: A320
    Equipment: automation & FMS
    Source: Sanchez-Ku, M.L., & Arthur, Jr. W. (2000). A dyadic protocol for training complex skills: A replication using female participants. Human Factors, 42(3), 512-520. See Resource details

  73.  
  74. Evidence Type: Excerpt from Experiment
    Evidence: "After G/S capture, a G/S signal loss was simulated at approximately 3,000 ft ... pilots were asked about the consequences of this event, and 54% of the pilots provided the correct answer. When asked whether a G/S failure at a lower altitude (<1,500 ft) would have different effects, only 15% of the pilots knew the answer. [85% did not know the answer.] Twenty-three percent of the participants did not know the answer to either question." (page 17)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  75.  
  76. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to disengage the APPR mode after localizer and glideslope had been captured, only 3 pilots (15%) could recall the three ways of accomplishing this ... Seven pilots (35%) did not know of any procedure for disengaging the APPR mode." (page 16)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autopilot
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  77.  
  78. Evidence Type: Excerpt from Experiment
    Evidence: "5 of the 6 [83%] pilots without line experience could not describe how to program an intermediate descent on the VNAV Cruise page for avoiding traffic, whereas none of the 14 experienced pilots had any problem with this task." (page 17)
    Strength: +4
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  79.  
  80. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked about the consequences of using an excessive vertical rate of climb in the V/S mode, none of the transitioning pilots could provide the correct answer, as compared to only 5 (35%) of the experienced participants [i.e., (6 + 9)/20 = 75% didn't know]." (page 17)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  81.  
  82. Evidence Type: Excerpt from Experiment
    Evidence: "The GA mode becomes available when descending below 2,000 ft radio altitude with autothrottles armed. Out of 20 pilots, only 5 [25%] recalled the altitude at which this occurs. Eight pilots (40%) knew that the availability of the mode depends on reaching a certain altitude, but they did not remember the actual height." (page 16)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  83.  
  84. Evidence Type: Excerpt from Experiment
    Evidence: Pilots were asked for their expectations concerning ADI mode indications throughout the takeoff roll ... N1 ... and THR HOLD ... Five of the pilots (25%) expected to see both these indications. Twelve subjects (60%) only mentioned either THR HOLD (15% of the pilots) or N1 (45% of the pilots) as an indications during takeoff. Another 3 pilots (15%) could not predict any of the mode indications. (page 15)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  85.  
  86. Evidence Type: Excerpt from Experiment
    Evidence: Immediately before receiving their takeoff clearance, pilots were asked what procedure they would use to abort the takeoff at 40 kts. Although it was emphasized that the takeoff had to be aborted at 40 kts -- before Throttle Hold (THR HLD) is reached at 64 kts, when the pilot can manually position the throttles -- 16 pilots (80%) described the procedure as "Throttles back, reversers, and manual brakes," They did not mention that the autothrottles would have to be disconnected to prevent the throttles from coming back up again after manual intervention. (page 15)
    Strength: +4
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  87.  
  88. Evidence Type: Excerpt from Experiment
    Evidence: "Nine out of 20 pilots knew how the FMS maintains target speed during a VNAV Path descent. Eight pilots (40%) were aware of the speed control mode during a VNAV Speed descent. With respect to the end-of-descent point of a path descent versus a speed descent, the results were similar: Twelve pilots (60%) were aware of the end of descent during a VNAV Path descent, and 9 pilots (45%) knew at what point the VNAV Speed descent would end." (page 16-17)
    Strength: +3
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  89.  
  90. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked to intercept the LAX 248 [degree] radial, all 6 of the transition pilots had difficulties carrying out the task using LNAV, as compared to only 7 of the 14 experienced pilots." (page 17)
    Strength: +3
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  91.  
  92. Evidence Type: Excerpt from Experiment
    Evidence: "[a] problem related to mode engagement was the attempt to activate a mode without the prerequistes for this activation being met: Three (50%) of the transitioning and one of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required. Three (50%) of the transitioning and 5 of the 14 experienced pilots engaged the APPR mode without lowering the MCP altitude first, and they were surprised to find that the aircraft did not start the descent." [Total number of pilots to make mistakes = 8 to 12 (66% to 88%)] (page 20)
    Strength: +3
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  93.  
  94. Evidence Type: Excerpt from Observational Study
    Evidence: "... the training observations indicate that pilots do not perceive the FMS as one large integrated system consisting of a variety of closely related, interacting subsystems such as the MCP or the CDU. ... Our data show that pilots think of and operationally use the MCP and CDU as, at least two different systems." (page 318)
    Strength: +3
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  95.  
  96. Evidence Type: Excerpt from Experiment
    Evidence: "... the overall number of 13 altitude violations is an alarming result. Six pilots [33%] failed to detect and recover in time from the loss of previously entered altitude constraints as a consequence of the runway change in the MCDU. ... The problem in these seems to be that the pilot provides an instruction to the automation without realizing the additional unintended implications of his input." (page 61-62)
    Strength: +2
    Aircraft: A320
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. CSEL Report 95-TR-01. See Resource details

  97.  
  98. Evidence Type: Excerpt from Survey
    Evidence: "The pilots' rating of the two statements on cockpit automation basically replicate Wiener's (1989) results." The following are the "percentages of pilots' responses to the second statement 'There are still modes and features of the B-737-300 FMS that I don't understand.' "Out of 135 pilots, 12% "strongly agreed", 33% "agreed", 16% were "neutral", 25% "disagreed", and 14% "strongly disagreed" with the statement. That is, 45% agreed or strongly agreed and 39% disagreed or strongly disagreed. (page 307-309)
    Strength: +2
    Aircraft: B-737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  99.  
  100. Evidence Type: Excerpt from Experiment
    Evidence: "Another interesting result refers to failures to engage or reengage a mode after entering new target values into the MCP or the CDU. This omission occurred at least once during the scenario for 5 of the 6 transitioning pilots (total number of omissions = 9). Only 2 of the 14 experienced pilots forgot to engage an appropriate mode, and this occurred only once for each of them." (page 18-20)
    Strength: +2
    Aircraft: B737-300
    Equipment: FMS & autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  101.  
  102. Evidence Type: Excerpt from Experiment
    Evidence: "... the overall number of 13 altitude violations is an alarming result. ... Seven pilots [39%] violated an assigned altitude when they reverted to an inappropriate mode from the EXPEDITE CLIMB mode. The problem in these seems to be that the pilot provides an instruction to the automation without realizing the additional unintended implications of his input." (page 61-62)
    Strength: +2
    Aircraft: A320
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1995). Strong, Silent, and Out-of-the-loop: Properties of Advanced (Cockpit) Automation and Their Impact on Human-Automation Interaction. CSEL Report 95-TR-01. See Resource details

  103.  
  104. Evidence Type: Excerpt from Experiment
    Evidence: "Another problem related to mode engagement was the attempt to activate a mode without the prerequisites for this activation being met. Fifty percent [3] of the transitioning pilots and 1 of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required. [(3+1)/20 = 4/20 = 20% failed]" (page 20)
    Strength: +1
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  105.  
  106. Evidence Type: Excerpt from Observational Study
    Evidence: "During the last three training sessions ..., pilot errors and questions focused on gaps in their understanding of the underlying functional structure of the FMS. Table 6 provides an overview of the most frequently encountered problems and questions. ... TABLE 6 Most Frequently Observed Problems ... Pilots often indicated that they were not sure whether there were other ways of achieving a goal or how to choose among multiple methods" (page 315)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  107.  
  108. Evidence Type: Excerpt from Survey
    Evidence: "Pilots were asked to describe instances where FMS behavior surprised them and to indicate modes/features of FMS operation that they did not understand. There were no sharp boundaries between the incidents elicited by the two questions. Pilot reports are categorized according to their underlying theme." ... There were 8 reports [8 / 135 = 5.9%] in the category: "VNAV Speed Descent Mode in general ... Pilots indicate that they do not understand how the VNAV Speed Descent works in terms of its targets, protections, and its operational logic." (page 307, 310)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS VNAV
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  109.  
  110. Evidence Type: Excerpt from Experiment
    Evidence: "This study ... confirms that most of the difficulties in pilot-automation interaction are related to a lack of mode awareness and to gaps in pilots' mental models of the functional structure of the automation." (page 21)
    Strength: +1
    Aircraft: B737-300
    Equipment: automation
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  111.  
  112. Evidence Type: Excerpt from Observational Study
    Evidence: "Another common factor implicated in many of the problems noted in the corpus is incomplete or buggy mental models of how various modes of the FMS work and especially how they interact with each other in different flight contexts." (page 317)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  113.  
  114. Evidence Type: Excerpt from Survey
    Evidence: "Pilots were asked to describe instances where FMS behavior surprised them and to indicate modes/features of FMS operation that they did not understand. There were no sharp boundaries between the incidents elicited by the two questions. Pilot reports are categorized according to their underlying theme." ... There were 3 reports [3 / 135 = 2.2%] in the category: "The effects of partial system failures ... These pilots report that they are unsure of the consequences of partial FMS failures. After such failures, they can not tell which subsystems are still active, which systems are available, or how the failure may interact with the active flight control mode. These reports implicate potential problems with both pilots' mental model of the FMS structure and with the indications of FMS status and behavior." (page 307-313)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  115.  
  116. Evidence Type: Excerpt from Survey
    Evidence: "Pilots were asked to describe instances where FMS behavior surprised them and to indicate modes/features of FMS operation that they did not understand. There were no sharp boundaries between the incidents elicited by the two questions. Pilot reports are categorized according to their underlying theme." ... There were 14 reports [14 / 135 = 10.4%] in the category: "Infrequently used features/modes ... Pilots report that they do not understand modes and features of the FMS that they rarely use (e.g. the Required Time of Arrival (RTA) feature). However, they also comment that they do not think of these as critically importuned features." (page 307, 311)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  117.  
  118. Evidence Type: Excerpt from Survey
    Evidence: "Pilots were asked to describe instances where FMS behavior surprised them and to indicate modes/features of FMS operation that they did not understand. There were no sharp boundaries between the incidents elicited by the two questions. Pilot reports are categorized according to their underlying theme." ... There were 6 reports [6 / 135 = 4.4%] in the category: "Disengagement of the Approach (APPR) mode ... Some pilots report that they were not able to disengage the APPR mode when required to do so." (page 307-311)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  119.  
  120. Evidence Type: Excerpt from Observational Study
    Evidence: "Frequently, pilots were able to describe FMS behavior during standard operations. ... But the same pilot would have difficulties applying this knowledge to a specific and more complicated operational situation, e.g. an aborted takeoff." (page 315)
    Strength: +1
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  121.  
  122. Evidence Type: Excerpt from Experiment
    Evidence: "The GA mode becomes available when descending below 2,000 ft radio altitude with autothrottles armed. Out of 20 pilots, only 5 [25%] recalled the altitude at which this occurs. Eight pilots (40%) knew that the availability of the mode depends on reaching a certain altitude, but they did not remember the actual height." (page 16)
    Strength: -1
    Aircraft: B737-300
    Equipment: autoflight: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  123.  
  124. Evidence Type: Excerpt from Experiment
    Evidence: "After G/S capture, a G/S signal loss was simulated at approximately 3,000 ft ... pilots were asked about the consequences of this event, and 54% of the pilots provided the correct answer. When asked whether a G/S failure at a lower altitude (<1,500 ft) would have different effects, only 15% of the pilots knew the answer. Twenty-three percent of the participants did not know the answer to either question." (page 17)
    Strength: -1
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  125.  
  126. Evidence Type: Excerpt from Survey
    Evidence: "The pilots' rating of the two statements on cockpit automation basically replicate Wiener's (1989) results." The following are the "percentages of pilots' responses to the second statement 'There are still modes and features of the B-737-300 FMS that I don't understand.' "Out of 135 pilots, 12% "strongly agreed", 33% "agreed", 16% were "neutral", 25% "disagreed", and 14% "strongly disagreed" with the statement. That is, 45% agreed or strongly agreed and 39% disagreed or strongly disagreed. (page 307-309)
    Strength: -2
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1992). Pilot interaction with cockpit automation: Operational experiences with the Flight Management System. International Journal of Aviation Psychology, 2(4), 303-321. Lawrence Erlbaum Associates. See Resource details

  127.  
  128. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked to intercept the LAX 248 [degree] radial, all 6 of the transition pilots had difficulties carrying out the task using LNAV, as compared to only 7 of the 14 experienced pilots." That is, 7 out of 20 (35%) did not have difficulties. (page 17)
    Strength: -2
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  129.  
  130. Evidence Type: Excerpt from Experiment
    Evidence: Pilots were asked for their expectations concerning ADI mode indications throughout the takeoff roll ... N1 ... and THR HOLD ... Five of the pilots (25%) expected to see both theses indications. Twelve subjects (60%) only mentioned either THR HOLD (15% of the pilots) or N1 (45% of the pilots) as an indications during takeoff. Another 3 pilots (15%) could not predict any of the mode indications. (page 15)
    Strength: -2
    Aircraft: B737-300
    Equipment: autopilot: autothrust
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  131.  
  132. Evidence Type: Excerpt from Experiment
    Evidence: "Nine [45%] out of 20 pilots knew how the FMS maintains target speed during a VNAV Path descent. Eight pilots (40%) were aware of the speed control mode during a VNAV Speed descent. With respect to the end-of-descent point of a path descent versus a speed descent, the results were similar: Twelve pilots (60%) were aware of the end of descent during a VNAV Path descent, and 9 pilots (45%) knew at what point the VNAV Speed descent would end." (page 16-17)
    Strength: -2
    Aircraft: B737-300
    Equipment: FMS
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  133.  
  134. Evidence Type: Excerpt from Experiment
    Evidence: "Another interesting result refers to failures to engage or reengage a mode after entering new target values into the MCP or the CDU. This omission occurred at least once during the scenario for 5 of the 6 transitioning pilots (total number of omissions = 9). Only 2 of the 14 experienced pilots forgot to engage an appropriate mode, and this occurred only once for each of them." That is, 13 of 20 (65%) did not forget. (page 18-20)
    Strength: -3
    Aircraft: B737-300
    Equipment: FMS & autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  135.  
  136. Evidence Type: Excerpt from Experiment
    Evidence: "When asked to disengage the APPR mode after localizer and glideslope had been captured, only 3 pilots (15%) could recall the three ways of accomplishing this ... Seven pilots (35%) did not know of any procedure for disengaging the APPR mode." This means that 50% of the pilots were able to recall at least one way to disengage the glideslope. (page 16)
    Strength: -3
    Aircraft: B737-300
    Equipment: autoflight: autopilot
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  137.  
  138. Evidence Type: Excerpt from Experiment
    Evidence: "Another problem related to mode engagement was the attempt to activate a mode without the prerequisites for this activation being met. Fifty percent of the transitioning pilots and 1 of the 14 experienced pilots tried to engage VORLOC without being in the manual radio mode as required." That is, 16 of 20 (80%) did it correctly. (page 20)
    Strength: -4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  139.  
  140. Evidence Type: Excerpt from Experiment
    Evidence: "... when asked about the consequences of using an excessive vertical rate of climb in the V/S mode, none of the transitioning pilots could provide the correct answer, as compared to only 5 (35%) of the experienced participants [i.e., (6 + 9)/20 = 75% didn't know]." (page 17)
    Strength: -4
    Aircraft: B737-300
    Equipment: autoflight
    Source: Sarter, N.B. & Woods, D.D. (1994). Pilot interaction with cockpit automation II: An experimental study of pilot's model and awareness of the Flight Management System. International Journal of Aviation Psychology, 4(1), 1-28. Lawrence Erlbaum Associates. See Resource details

  141.  
  142. Evidence Type: Excerpt from Survey
    Evidence: "Over the full range of skills that were investigated, a substantial percentage of the investigated pilot population expressed a need for extra training…ranking of seven different skill groups with respect to need/priority for extra training 1) knowledge of automation 2) decision making 3) crew resource management 4)manual flying 5)determination of appropriate SOP's 6) standard cockpit handling 7)knowledge of SOP's." Note: this information is depicted in table format in the document.
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Sherman, P.J., Helmreich, R.L., & Merritt, A. (1997). National culture and flight deck automation: Results of a multination survey. International Journal of Aviation Psychology, 7(4), 311-329. Lawrence Erlbaum Associates. See Resource details

  143.  
  144. Evidence Type: Excerpt from Survey
    Evidence: "Fifty eight structured interviews were conducted at a number of European airlines to enable pilots and training instructors to comment on current transition training practices, to give levels of understanding of various automated systems and express their views on automation and related issues… Pilots attitudes towards the automation were generally positive. Surprises caused by the automation tended to occur especially early after training, as did human errors due to negative transfer. In cases where pilots were surprised, they admitted that it did influence their trust in the aircraft. Comments suggested that a higher level of understanding of systems, better problem solving skills and prioritisation rules to avoid excessive head-down time, could mitigate negative effects of difficult situations."
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Sherman, P.J., Helmreich, R.L., & Merritt, A. (1997). National culture and flight deck automation: Results of a multination survey. International Journal of Aviation Psychology, 7(4), 311-329. Lawrence Erlbaum Associates. See Resource details

  145.  
  146. Evidence Type: Excerpt from Survey
    Evidence: "In order to identify what kinds of problems occur in automated aircraft, a review of accident and incident reports from a number of European and US sources was completed. Reports were selected on the basis of keyword searches for terms relating to human factors, training and automation, and were then classified using a taxonomy developed in ECOTTRIS to identify various operational, behavioural, design contributory and general automation factors. "Analysis of frequency of factors and linkages between factors was carried out and yielded the following results: deficiency in CRM was a contributory factor in incidents and accidents (identified in 39% of all reports) and this could be linked with incorrect settings, monitoring and vigilance, inadequate knowledge of aircraft systems, experience and flight handling. Furthermore, complacency was found in 13% of reports and improper use of systems occurred in 15% of reports. In this part of the study, mode awareness was identified as a factor in only 6% of reports."
    Strength: +1
    Aircraft: various
    Equipment: automation
    Source: Sherman, P.J., Helmreich, R.L., & Merritt, A. (1997). National culture and flight deck automation: Results of a multination survey. International Journal of Aviation Psychology, 7(4), 311-329. Lawrence Erlbaum Associates. See Resource details

  147.  
  148. Evidence Type: Excerpt from Survey
    Evidence: Statement 34: "There are still modes and features of the B-757 FMS that I don't understand." From the histograph of the responses in Phase 1 of the study, only 34% of the pilots agreed or strongly agreed with the statement and in Phase 2 of the study, only 21% of the pilots agreed or strongly agreed with the statement while 53% disagreed or strongly disagreed in Phase 1, and 64% disagreed or strongly disagreed in Phase 2. The neutral responses were 13% in Phase 1 and 15% in Phase 2. (page 58)
    Strength: +2
    Aircraft: B757
    Equipment: FMS
    Source: Wiener, E.L. (1989). Human Factors of Advanced Technology ("Glass Cockpit") Transport Aircraft. NASA Contractor Report 177528. Moffett Field, CA: NASA Ames Research Center. See Resource details

  149.  
  150. Evidence Type: Excerpt from Survey
    Evidence: Statement 34: "There are still modes and features of the B-757 FMS that I don't understand." From the histograph of the responses in Phase 1 of the study, 34% of the pilots agreed or strongly agreed with the statement and in Phase 2 of the study, 21% of the pilots agreed or strongly agreed with the statement while 53% disagreed or strongly disagreed in Phase 1, and 64% disagreed or strongly disagreed in Phase 2. The neutral responses were 13% in Phase 1 and 15% in Phase 2. (page 58)
    Strength: -3
    Aircraft: B757
    Equipment: FMS
    Source: Wiener, E.L. (1989). Human Factors of Advanced Technology ("Glass Cockpit") Transport Aircraft. NASA Contractor Report 177528. Moffett Field, CA: NASA Ames Research Center. See Resource details
Flight Deck Automation Issues Website  
© 1997-2013 Research Integrations, Inc.