ReTINA: The Pilot’s Eyes on the Ground Tyler H. Shaw, PhD Faculty Advisor
Jane H. Barrow William J. Benson Eric J. Blumberg Devon B. Kelley Brian D. Kidwell Haneen Saqer Melissa A.B. Smith Jonathan D. Strohl Graduate Students
George Mason University
Executive Summary Runway incursions happen for a variety of reasons, but pilot deviations are the leading cause. This problem will be exacerbated by the fact that air traffic is expected to increase threefold by 2025. While there are many emerging technologies focused on increasing surveillance of ground traffic, most of these systems are costly and will support ATC, but not the pilot. Currently pilots rely on paper maps and textual Notices to Airmen (NOTAMs) to familiarize themselves with airports and current conditions. Our design, ReTINA (Realtime Taxiway Interactive Navigation Aid), will improve the pilot’s awareness of the airport environment and conditions using digitized maps that provide real time information to the pilot about their current location in addition to allowing them to interactively highlight a desired taxi route. In addition to these interactive features, ReTINA overlays a graphical representation of taxiway- and runwayrelevant NOTAMs so that pilots can easily identify closed runways, incursions hotspots and construction areas. A user-centered design approach informed by user interviews, usability testing and cognitive task analyses was used to develop an application that minimizes the potential risk of introducing technology into existing cockpit systems. ReTINA also has the capability to be scaled up to incorporate new systems and capabilities as they become available. For example, other ground traffic can be displayed on the Airport Diagrams with the integration of ADS-B and similar technologies. While there is no replacement for looking out the window, ReTINA provides pilots with a user-friendly interface, containing situationally relevant information, that will improve pilots’ ability to taxi within the airport safely.
ReTINA: The Pilot’s Eye on the Ground 2
Table of Contents 1. Problem Statement and Background ...........................................................................................................5 1.1 Causes of Runway Incursions Due to Pilot Deviations .................................................................5 1.2 Emerging Technologies to Prevent Runway Incursions ..............................................................8 1.3 ReTINA: Realtime Taxiway Interactive Navigation Aid................................................................9 2. Summary of Literature Review ................................................................................................................... 10 2.1 Passive Approaches to Runway Incursion Prevention .............................................................. 10 2.2 Active Approaches to Runway Incursion Prevention................................................................. 11 2.3 iPad Usage in the Cockpit ....................................................................................................................... 13 3. Team's Approach to Problem ...................................................................................................................... 14 4. Safety Risk Assessment .................................................................................................................................. 16 5. Technical Aspects of Design ......................................................................................................................... 16 5.1 Pilot Workflow During Taxiing ............................................................................................................ 17 5.2 ReTINA iPad Application: System Overview.................................................................................. 18 5.3 ReTINA: Specific Features ..................................................................................................................... 19 5.3.1 Homepage ............................................................................................................................................ 19 5.3.2 Map Display ......................................................................................................................................... 21 5.3.3 Settings and Help Menus................................................................................................................ 23 5.4 Interaction Walkthrough - Taxiing at Ronald Reagan Washington National Airport (DCA)...................................................................................................................................................................... 23 5.5 User Testing ................................................................................................................................................. 24 5.6 Scalability of the Design .......................................................................................................................... 25 6. Interactions with Airport Operators and Industry Experts ............................................................ 27 7. Projected Impact of Design and Findings ............................................................................................... 32 Appendix A ............................................................................................................................................................... 34 Appendix B ............................................................................................................................................................... 35 Appendix C Description of Non-University Partners ............................................................................. 36 Appendix D – Signoff Form by Advisor......................................................................................................... 37 Appendix E ............................................................................................................................................................... 38 E-1
Dr. Tyler Shaw, Faculty Advisor ................................................................................................... 38
E-2
Jane H. Barrow ..................................................................................................................................... 40
E-3
William J. Benson ................................................................................................................................ 41
E-4
Eric J. Blumberg................................................................................................................................... 41
ReTINA: The Pilot’s Eye on the Ground 3
E-5
Devon B. Kelley.................................................................................................................................... 42
E-6
Brian D. Kidwell .................................................................................................................................. 43
E-7
Haneen Saqer ....................................................................................................................................... 44
E-8
Melissa A.B. Smith .............................................................................................................................. 44
E-9
Jonathan D. Strohl .............................................................................................................................. 45
Appendix F - References ...................................................................................................................................... 47 Appendix G - Figures ............................................................................................................................................ 51
ReTINA: The Pilot’s Eye on the Ground 4
1. Problem Statement and Background Flight has become the most prevalent form of long distance transport; in the United States of America alone, there is an average of over 7,000 takeoffs and landings per hour and 50,000 flights per day (FAA, 2011a). The latest Runway Safety Report indicates that in 2009, there were 52.9 million surface operations that took place (FAA, 2010a), and these numbers are set to increase threefold by the year 2025 (Joint Planning and Development Office, 2004). Of those 52.9 million operations, 951 runway incursions occurred in 2009, 12 of which fell into the serious, Category A classification for runway incursions (FAA, 2010a). While the relative infrequency of incursions based on the number of actual operations is a testament to the efficacy of the current system, the likelihood of future incursions is quite high considering that runway traffic is due to increase by three hundred percent (Joint Planning and Development Office, 2004). Additionally, the consequences of a single runway incursion can be devastating - one only has to look at the events that took place in Tenerife in 1977 to understand the loss of life that can accompany such an event (de Luchtvaart, 1979). As a result, preventing runway incursions has been at the top of both the Federal Aviation Administration and the National Transportation Safety Board most wanted lists for many years (NTSB, 2011).
1.1 Causes of Runway Incursions Due to Pilot Deviations There are three types of runway incursions: operational errors on the part of an air traffic controller (ATC), vehicle or pedestrian deviations, and pilot deviations (FAA, 2009). As of April 2012, pilot deviations are the leading cause of runway incursions, with 316 of the 455 incursions that have occurred year-to-date (FAA, 2012b). There are many possible reasons for why pilot deviations are the primary cause of runway incursions,
ReTINA: The Pilot’s Eye on the Ground 5
including distractions, fatigue, stress, impatience, and most importantly, not realizing where they are located (A. Gertsen, personal communication, February 24, 2012). This lack of situational awareness (SA) may be the most problematic aspect of the pilot experience, which is supported by the NTSB’s Most Wanted List of transportation safety improvements (NTSB, 2011). Situation awareness is generally defined as: “The perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Endsley, 1995). Figure 1 depicts the three-level model of SA as described by Endsley (1995). For example, the three levels of SA can be conceptualized as follows: level 1 SA would be a pilot orienting his current location on an airport diagram immediately after landing. Level 2 SA would be represented by the pilot understanding and mentally visualizing the taxi directions given to him by ATC, indicating an appropriate mental model of the airport surface area. Level 3 SA would be of the pilot listening to the party line and projecting ground traffic patterns and final destinations for other aircraft in his proximity demonstrating an ability to project future changes to the environment. Several issues can lead to reduced SA for pilots, preventing them from even attaining level 1 SA. One major issue is the current protocol for avoiding incursions in General Aviation (GA), and even for the majority of commercial air traffic, which has remained unchanged since the beginning of manned flight. “See-and-avoid” is the overarching procedure for incursion avoidance, subsumed by standardized airport operations and ATC guidance at non-towered and towered airports, respectively. Proficiency through training in chart symbology and interpretation of airport surface
ReTINA: The Pilot’s Eye on the Ground 6
diagrams is the current preventative measure for runway incursions. Nevertheless, a system solely reliant on pilot training and pilots’ perceptual processes is prone to breakdowns in the fail-safes of airport procedures, as evidenced in the current statistics reflecting pilot deviations as the leading cause of runway incursions. These breakdowns can occur due to lack of training, fatigue, distractions in the cockpit, and inclement weather conditions which hinder visual acuity. Moreover, the paper Airport Diagrams used by pilots currently an integral part in runway operations are particularly poor tools for aiding SA since they cannot provide the pilot with any indication of current location in relation to other structures or traffic on the runway (B. Kidwell, personal communication, March 3, 2012). Another issue regarding SA and the cockpit is the limited use that pilots make of NOtice To AirMen (NOTAMs; R. Loewen, personal communication, February 24, 2012). NOTAMs are daily updates to be accessed by the pilots (recommended at least twice daily) that provide information regarding runway closures, taxiway closures and other useful information (M. McClintock, personal communication, April 25, 2012). However, they also provide information that is not directly relevant to most pilots. NOTAMs are displayed in textual format with numerous abbreviations and they lack organization. Additionally, pilots cannot filter or sort information included in NOTAMs making it difficult to pick out relevant items, and consequently, many pilots simply ignore them (C. McCalla, personal communication, April 25, 2012). This has a direct, negative impact on SA, as NOTAMs contain pieces of highly relevant information critical to pilots’ awareness of the aerodrome environment.
ReTINA: The Pilot’s Eye on the Ground 7
As mentioned earlier, there is expected be a significant increase in the quantity of air traffic within the next 20 years. Using current technology and procedures may result in devastating effects to both controller and pilot situation awareness and workload. However, the FAA is implementing progressive programs such as Next Generation (NextGen) to actively respond to this concern.
1.2 Emerging Technologies to Prevent Runway Incursions NextGen will add increasingly accurate and capable technologies to air traffic control facilities. In turn, controllers will be better suited to prevent potential incursions, both in the air and on the ground. New systems, such as Airport Surface Detection Equipment, Model X (ASDE-X), provide increased accuracy in runway monitoring for controllers. Another example of emerging technologies are runway lighting systems that serve as external cockpit aids to instruct pilots to hold on the runway to avoid a potential incursion. However, these systems provide little to the pilot in regards to awareness of movements on the runway (Patterson, 2004; Tech Notes, 2010; FAA, 2010b). Most of these lighting systems are costly to implement, making them cost-prohibitive for smaller airports. The information gleaned from ASDE-X and other surveillance systems will greatly benefit ATC, however, current plans to incorporate this information in the cockpit are not immediately feasible. This void indicates another approach may be possible to mitigate runway incursions by aiding pilots in the cockpit. For these reasons our team recognized an opportunity to design a pilot aid that would provide realtime information in an interactive manner to improve SA for navigating on the ground.
ReTINA: The Pilot’s Eye on the Ground 8
1.3 ReTINA: Realtime Taxiway Interactive Navigation Aid The design our team produced to increase pilot situation awareness is an iPad application intended to provide easy access to critical information necessary to pilots during taxi. ReTINA was designed to provide real-time location information to the pilot specific to his movements on taxiways and, in the future, surrounding traffic as well. It was also designed to modernize paper Airport Diagrams, converting them into dynamic displays and allowing for interactive navigation. This digitization includes the addition of taxiway-relevant information, extracted from NOTAMs, as a graphical overlay on the Airport Diagrams. A brief overview of ReTINA’s key features follows. Detailed descriptions are provided in the Technical Aspects of Design Section.
Digitized Airport Diagrams (updated automatically)
Real-time updating of current location on ground
Ability to interactively plan taxi route on map
Graphical depiction of taxiway-relevant NOTAMs
Automatic retrieval of local NOTAM and weather updates
Ability to take notes on map (record taxi instructions)
Real-time updating of surrounding ground traffic movement (future)
These features were designed to complement the pilots current workflow, serve as an aid to offload cognitive resources, provide a shared mental model between pilots and ATC of ground aircraft movements, and improve pilots’ understanding of taxi instructions by providing redundant information in a visual mode in addition to the auditory instructions upon which they currently rely. The remainder of this paper will focus first on the current approaches to preventing runway incursions through technological systems and the iPad applications
ReTINA: The Pilot’s Eye on the Ground 9
that are currently available. This will lead into a discussion of the human-centered approach the design team adopted to address the issue of poor pilot SA which leads to runway incursions, and a brief assessment of the safety risk ReTINA may pose by requiring additional attention from the pilot. The technical details of the app are then provided, along with a description of how ReTINA will fit into current pilot workflow. The results of a brief usability test of the ReTINA prototype are discussed, followed by a discussion of potential scalability of the app when additional technologies become available. The paper concludes with a summary of the interactions with different individuals who contributed to our knowledge and development of our design, and a discussion of the feasibility and benefit that ReTINA offers to the aviation industry.
2. Summary of Literature Review 2.1 Passive Approaches to Runway Incursion Prevention Large-scale initiatives are currently underway to mitigate the occurrence of runway incursions through the dissemination of information. While these are not meant to replace quality pilot training or skill, sound judgment, or conservative standards of safety, they are intended to augment pilot capabilities. FAA airport diagrams include a graphical depiction of runway incursion “hot spots” (FAA, 2012a): “An ‘‘Airport surface hot spot’’ is a location on an aerodrome movement area with a history or potential risk of collision or runway incursion, and where heightened attention by pilots/drivers is necessary. A ‘‘hot spot’’ is a runway safety related problem area on an airport that presents increased risk during surface operations. Typically it is a complex or confusing taxiway/taxiway or taxiway/runway intersection. The area of increased risk has either a history of or
ReTINA: The Pilot’s Eye on the Ground 10
potential for runway incursions or surface incidents, due to a variety of causes, such as but not limited to: airport layout, traffic flow, airport marking, signage and lighting, situational awareness, and training.� This depiction is intended as a near-zero cost initiative to alert pilots to potentially confusing and/or hazardous intersections. However, these graphical annotations on paper maps are strictly passive indicators that can easily be overlooked by pilots.
2.2 Active Approaches to Runway Incursion Prevention Other methods for reducing runway incursions include a combination of surveillance systems and physical signage/lighting systems. These technological solutions have many advantages, although oftentimes at prohibitively high levels of cost. Perhaps the most widely known system for the mitigation of runway incursions is ASDEX, which represents the first dedicated system to provide ATC with positional surveillance for ground traffic. Across the United States only 35 systems are now in operation (or planned implementation) (FAA, 2010b). The system operates through a combination of ADS-B (Automatic Dependent Surveillance-Broadcast) signal, multilateration sensors, terminal automation systems, and aircraft transponders. Another system, AXSL (ASDE-X Safety Logic), uses positional data from ASDE-X systems to anticipate potential incidents of collision between aircraft and ground vehicles. Together these systems allow controllers to accurately identify aircraft on the ground, as well as maintain vigilance to issues of ground separation through the use of visual and auditory alert systems. While these technologies are proving successful in reducing runway incursions, small airports are incapable of integrating these technologies at their current costs. Again, this leaves pilots vulnerable to potential incursions. Furthermore, the system
ReTINA: The Pilot’s Eye on the Ground 11
only informs ATC, leaving the pilot out of the loop with regards to system feedback and alerts. The Low Cost Ground Surveillance (LCGS) initiative by the FAA was created to provide a low cost alternative to ASDE-X. Currently, ASDE-X systems are cost prohibitive for the majority of airports in America, costing more than 15 million dollars (DOT, 2007). LCGS systems utilize a single ground radar sensor that updates positions every second and is capable of tracking objects up to a few hundred feet in the air. This information is fed to visual displays in the tower for use by ATC. LCGS systems can detect ADS-B equipped aircraft to increase location sensitivity. Currently there are four LCGS systems being tested across the country with the hope that within the next decade 30 more LCGS systems will be implemented. While LCGS systems are considered low cost in comparison to ASDE-X, their cost remains prohibitive (capped at one million dollars) to many general aviation airports across the country. Similar to ASDE-X, these systems are designed to aid ATC rather than provide information to the pilot in the cockpit. Finally, it will take many years to fully test and implement these systems effectively, making it even more important to find a short-term pilot-centered approach to runway incursion prevention. Although many technologies focus on assisting ATC, some emerging technologies are designed to alert the pilot to potential runway incursions. The National Aeronautics and Space Administration (NASA) Runway Incursion Prevention System (RIPS) represents one such approach. The RIPS system operates via a combination of heads-down and heads-up displays that provide a moving map of the airport surface with real-time guidance to pilots. Subjective reporting and usability testing has indicated the
ReTINA: The Pilot’s Eye on the Ground 12
efficacy of the RIPS design (Jones & Prinzel III, 2006) in laboratory simulations. Even still, this technology remains impractical for GA pilots, both for its price as well as the demands of incorporating the additional displays into the flight deck (already a compact area. Mobile technologies that can be used in any cockpit will benefit a significant number of pilots until RIPS displays become standard and affordable. Another example of this technology is Searidge Technologies’™ Runway Incursion Monitoring and Collision Avoidance System (RIMCAS). Similar to RIPS, RIMCAS uses a network of camera surveillance systems, automated incursion prediction algorithms, and operational links in existing lighting systems (Searidge Technologies, 2012). However, the system is constrained by the same narrow focus as the RIPS system, operating as a solution to the occurrence of a runway incursion, rather than for the prevention of such an incident. Lighting systems represent a relatively simple solution to the problem of runway incursions. Runway Guard Lights (RGL), Runway Status Lights (RWSL), and Final Approach and Runway Occupancy Signal (FAROS) systems have all been offered as cost-effective, simple solutions overlaying existing technologies (Patterson, 2004; Tech Notes, 2010; FAA, 2010b). Generally speaking, these systems operate by directly alerting pilots via light signals as to runway occupancy. However, these systems rely on the integration of information from more costly surveillance systems currently prohibitive for general aviation airports.
2.3 iPad Usage in the Cockpit ReTINA is not the first iPad application aimed at supporting the pilot. In fact, there are over 500 applications available for the iPad and iPhone (Aviator Apps, 2012). The use of an iPad in commercial aviation has just recently become an option in the US. American Airlines, Alaska Airlines and United Airlines were approved by the FAA to ReTINA: The Pilot’s Eye on the Ground 13
use iPads in the cockpit to varying degrees. For example, American Airlines has been approved to use the iPad during all stages of flight including both takeoff and approach (Bilton, 2011). It is stated in Title 14, section 91.21 of the Electronic Code of Federal Regulations the operator of the aircraft can use portable electronic devices that will not cause interference with navigation or communication systems on the aircraft (e-CFR, 2012). Currently available iPad apps fall into four primary categories: informational, en route, terminal, and multi-purpose. Informational apps include access to digitized aircraft handbooks, flight manuals, and helpful calculation tools for take-off and descent (Brewster, 2011). En route apps contain information related to in flight tasks such as navigation charts, weather information, and trip planning aids (Digital Cyclone, 2012). Terminal apps currently feature static digital airport diagrams, fuel pricing, standard instrument departures (SIDS), standard terminal arrivals (STARS), approach plates, and visual flight rule (VFR) plates (RocketRoute LTD., 2012). Multi-purpose apps attempt to do all of this, but in so doing, fall down in terms of price and usability. These apps can exceed $1000, and have extremely poor ratings in terms of the user interface and functionality of the application. Furthermore, any poorly designed interface will lead to confusion and distraction, causing more heads down time in the cockpit. For this reason, our design team use a human-centered approach with special consideration to the cognitive limitations of the pilot and usability of the interface.
3. Team's Approach to Problem Figure 2 depicts an adaptation of the cognition-artifact-task triad introduced by Gray and Altmann (2001) which helped to frame the design team’s thinking for the
ReTINA: The Pilot’s Eye on the Ground 14
design process. Gray and Altmann (2001) suggest that an operator’s behavior is influenced by each member of the triad, culminating in the observed human-machine interaction. In addition, it is important to take the environment into account, which surrounds all aspects of the triad and can add additional influences. One of the primary reasons for poor design is that many designers focus too heavily on the artifact and ignore the constraints posed by the environment, the task, or the cognitive abilities of the user (Boehm-Davis, 2006). In an effort to avoid this pitfall, the design team repeatedly returned to this concept, ensuring that all aspects of this design framework were being assessed. A variety of techniques were used to perform an analysis of the needs for each aspect of the design framework. The design team performed a large number of interviews, both structured and unstructured. Structured interviews are when the interviewer drives the direction of questioning, whereas unstructured allow for an open dialog between the interviewer and the subject matter expert (SME; Cooke, 1999). These interviews helped to frame the design team’s understanding of the needs of the user, the task the user was performing, and the environment in which the user was operating. The design team also used task analysis techniques, such as an Operational Sequence Diagram, to understand the steps involved while taxiing, which allowed us to determine where we could incorporate our design into the process without interrupting the current workflow (Kirwan & Ainsworth, 1992). Finally, we turned our focus to developing the actual product, which involved implementing usability guidelines for software interfaces, developing a prototype for evaluation, and then conducting a usability test to elicit feedback from users of the prototype (Stone, Jarrett, Woodroffe, & Minocha, 2005).
ReTINA: The Pilot’s Eye on the Ground 15
The details and outcomes of the various techniques are included in the next section, as we describe the technical aspects of the design. By maintaining a pilotcentered approach to the design, our decisions were informed by human factors principles to offer a safe aid while minimizing distractions in the cockpit.
4. Safety Risk Assessment During usability testing of ReTINA, pilots reported that shuffling through a stack of paper maps is much less efficient and more distracting than using the map search function in the app. Pilots also reported that features provided within the application, such as the draw your own route feature, graphical NOTAMS, and realtime display would decrease the possibility of incursions by increasing situation awareness. As with any display in the cockpit, pilots must be wary not to fall victim to cognitive tunneling. There is no replacement for looking out the window; as such, we strived to use best design practices to minimize heads-down time necessary to interact with the app. Additionally, if the app does not pick up on another craft due to loss of signal or if an animal or unmarked vehicle appears on the runway, pilots may not detect the crucial events if they only rely on the app. Although our app incorporates and acknowledgement window to remind them of this fact, as well as incorporating technology to inform them when GPS reliability is less than accurate, over reliance on automation remains a serious issue.
5. Technical Aspects of Design ReTINA, which was designed to promote pilot situational awareness on the runway, is an acronym for Realtime Taxiway Interactive Navigation Aid. While ATC currently has access to multiple technologies to support visualization of runway ground ReTINA: The Pilot’s Eye on the Ground 16
traffic, pilots must solely rely on out-the-window views and auditory instructions from ATC. ReTINA has been developed to provide a low-cost, accessible aid to pilots to modernize terminal maps and provide an interactive interface to visualize taxi instructions. Below we review the technical aspects of the design in the following terms: pilot workflow during taxiing, system overview, specific design features and interactions, usability testing, and scalability potential.
5.1 Pilot Workflow During Taxiing A successfully designed application should reduce pilot workflow while minimizing “head-down” time (A. Gertsen, personal communication, February 24, 2012). To fully understand the cognitive demands that are currently experienced by the pilot during taxiing, we conducted a cognitive walkthrough with a pilot and mapped his interactions on an Operational Sequence Diagram (OSD). The results of the OSD are described in Figure 3. Upon successfully landing the aircraft, the pilot requests and receives a sequence of taxi directions from ATC. The pilot then references a paper map and either writes down the directions on paper held in place by a clipboard/kneeboard, or maintains them in working memory. The pilot then reads back the taxiing directions to the ATC controller who verbally confirms the route. During optimal flight conditions and low demand situations, this task may not require many cognitive resources for the experienced pilot. However, these ideal workload situations are seldom the case in aviation. In challenging situations including pilot fatigue, high-density traffic, inclement weather conditions, and unfamiliar airports, the task of orienting oneself on a runway and remembering taxiing instructions becomes more difficult. This may be especially difficult for inexperienced pilots. For example, as pilots become more fatigued, holding several items in memory becomes onerous and ReTINA: The Pilot’s Eye on the Ground 17
recall becomes prone to error. During late-evening and red-eye flights, when fatigue is of particular concern, it can be difficult for pilots to hold more than two items in memory (C. McCalla, personal communication, April 25, 2012). By using our app, pilots can offload the cognitive demand of remembering taxiing instructions and focus on acquiring situational awareness. Furthermore, the OSD reveals that the use of the iPad does not overlap with other critical workflow operations. Therefore, providing pilots with a tool to display their location on the runway and write down taxi instructions can enhance situation awareness.
5.2 ReTINA iPad Application: System Overview ReTINA was designed to work successfully with the Apple iPad or other comparable tablet devices. For example, the Apple iPad incorporates global positioning system (GPS) to provide current location information and real-time updating of position (it should be noted that ReTINA will incorporate Receiver Autonomous Integrity Measure (RAIM) technology to detect signal degradation and alert the pilot). Tablet computers were selected as the desired platform because it is a widely available and popular commercial technology that incorporates intuitive gestures for zooming in and out of images and other interactions. This will allow for quick and seamless adoption by pilots. Additionally, there are kneeboards and clipboard mounts for iPads and tablets available in cockpits (Figures 4 and 5, respectively) which will allow pilots to stabilize the tablet during flight and landing. By introducing ReTINA via tablet devices, it allows the FAA to test and iterate the design quickly and easily and serve as a prototype for future inflight displays. The proposed tablet app will help to reduce the number of repeats of taxiing directions from ATC. All FAA issued Airport Diagrams will be provided within the app. ReTINA: The Pilot’s Eye on the Ground 18
This will eliminate the inconvenience of pilots needing to carry paper maps and will provide for all possible maps in case of emergency landing and diverted flight situations. The application also eliminates the hassle of flipping through multiple pages to find the desired map. In the tablet app, the maps will be updated automatically without the pilot needing to purchase new paper maps multiple times a year, also reducing paper waste and eliminating production costs. ReTINA contains a digital database of maps for all airports under FAA jurisdiction. These maps are not merely imported images of Airport Diagrams, but are remastered and enhanced to contain interactive elements. The interactive features of the Airport Diagrams will allow for visualizing taxiing routes and NOTAM information. Future development and integration with aviation technology may allow for the graphical display of real-time traffic information. The specific design features of ReTINA will be detailed individually.
5.3 ReTINA: Specific Features 5.3.1 Homepage To visualize the design features of the application, a high fidelity protoype was created using Microsoft Powerpoint (TM), Adobe Photoshop (TM), and Axure RP (TM). Microsoft Powerpoint and Adobe Photoshop were used to create and manipulate images and graphics. Simulated iPad graphics were obtained from the iPad Graphical User Interface PhotoShop Data (PSD) file by Geoff Teehan (2010). The interactive components of the prototype were created using Axure RP. The interactive nature of the prototype allowed for basic user-testing for proof of design concepts. The prototype is comprised of several pages. The homepage opens with the startup of the app with a pop-up window overlayed on top with a warning to pilots that 1) ReTINA: The Pilot’s Eye on the Ground 19
eyes should be kept outside the window at all times, and 2) the GPS may not be 100% reliable (Figure 6). The user must select “I Acknowledge” on the pop-up window in order to gain access to the homepage (Figure 7). The top headers are icons representing the iPad’s wifi signal strength and battery life. Underneath this iPad header is the menubar for the homepage which contains links to the settings and help pages. The homepage is a summary page with in-frame windows that contain executable actions and current information. These in-frame windows are customizable in size and location on the homepage. Users are able to adjust these windows so that the individual’s most pertinent information can be displayed. The homepage is defaulted to display 1) an Airport Diagrams window, 2) a Favorites window, and 3) an Airport Advisories window described below. Individual maps can be opened from within the Airport Diagrams window. At the top of the Airport Diagrams window contains the individual user’s most visited diagrams. Underneath the most visited diagrams is a comprehensive list of airport diagrams. Diagrams from this comprehensive list can be manually searched using the scroll bar or can be searched by entering in an airport code in the search bar. When the user enters the first letter of the airport code, the menu jumps to that portion of the alphabetical list and the user can type via an iPad style keyboard (Figure 8). The Favorites window contains user-defined groups of Airport Diagrams. These groups can be customized so that multiple maps and NOTAM reports can be opened simultaneously in tabular format with a single action. Pilots with routine routes can access their most visited pages in this window. This allows pilots to set maps according
ReTINA: The Pilot’s Eye on the Ground 20
to routine flight patterns based on airline, day of the week, or geography. The Airport Diagrams assign up to five groups can be edited through a feature in the settings menu. The airport advisories window contains a real-time stream of information about alerts and warnings for each of the airports. Similar to the Airport Diagrams window, the first advisories displayed are for airports most proximal to current GPS location and the most frequently visited airports. Underneath these most visited airports is a comprehensive listing of all advisories. Again, users have the choice of manually scrolling through the list to select a specific airport or typing in the airport code in the search bar. The advisories and warnings consist of information pulled from the NOTAMs reports as well as from NOAA’s Aviation Weather Center. 5.3.2 Map Display Upon selecting a specific map in the Airport Diagrams menu or selecting a group of maps from the Favorites menu, the Airport Diagrams is displayed in portrait view (Figure 9). Across the header, labels of opened maps are indicated from left to right in tabular format. This allows the user to toggle between multiple windows without needing to open and close each individual diagram. The top right of the header contains 1) a link to return to the homepage, 2) a link to go to the settings menu, and 3) a link to go to the help menu. These three icons are consistent across all airport diagrams and NOTAM reports. By default, the map displays a directional arrow which indicates the aircraft’s current location and direction. The bottom of the diagram page contains two links, one to view a complete listing of the airport’s NOTAMs and another to open an onscreen notepad (Figures 10 and 11). To the right of the airport diagram is a button to open up dynamic on-map features which will be described in the next section.
ReTINA: The Pilot’s Eye on the Ground 21
By accessing the dynamic on-map features, the map is resized and a layers menu is presented on the right-hand side (Figure 12). This menu displays options for dynamic on-map layering as well as quick access to map features. The first dynamic layering feature is an overlay of airport traffic (Figure 13). While technological limitations may not currently permit for display of airport traffic, future developments may allow for this. In the future, this information may be obtained from the ADS-B, ASDE-X, and Low-Cost Ground Surveillance (LCGS) networks. It is also conceivable that airport traffic could be generated within a closed network using the iPad’s GPS signal. This graphical display of airport ground traffic may decrease incursions by providing pilots with traffic information not within eyesight. The second dynamic layering feature is an overlay of NOTAM report information. Conceivably, an algorithm could be written to extract relevant taxiway/runway information from the NOTAM reports. This would automatically generate alerts and warnings on the map itself. For example, a NOTAM alert of a runway closure can be graphically represented on the respective runway (Figure 14). Maintenance warnings, hotspots, and weather advisories could also be graphically overlayed on the map using this technique. The map menu also includes an automatic dimming feature which would adjust the brightness of the screens for optimal viewing of the screen in different lighting environments. Another on-map feature that is accessible when the layers menu is minimized is a route-drawing tool. When the route drawing mode is activated, accessible turning points (visualized as navigation pins) from the aircraft’s current location become visible (Figure 15). For example, when the aircraft is on runway 19 South, the accessible turning points
ReTINA: The Pilot’s Eye on the Ground 22
from this runway become visible. According to the ATC’s directions, the pilot selects the first turning point on the map. After this first turning point is selected, a line is drawn on the map from the aircraft’s current location to this turning point and continues for all subsequent interactions until arriving at the terminal destination (Figures 16-20). The user may reset the map directions at any time by clicking on the aircraft’s current location (Figure 21). The drawing of the map is expected to aid in confirming the ATC’s verbal directions and instructions. 5.3.3 Settings and Help Menus The settings page allows for user customization (Figure 22). One of the settings features on this page allows for the assignment of individual maps to groups in the favorites window on the homepage. By building a group of favorite maps, multiple maps can be opened with a single action. The settings page allows the user to change default on-screen map features (e.g.. dynamic layering features). Other technical settings options such as type of connection, sound, Wi-Fi settings, power saver modes, date and time display, etc. are also represented on the settings menu. The help page contains information on how to use the app as well as more explanation of its different features. Frequently Asked Questions are displayed in this location. The help menu is indexed by categories and is searchable by keywords.
5.4 Interaction Walkthrough - Taxiing at Ronald Reagan Washington National Airport (DCA) Brian Benson, a persona for a newly licensed commercial pilot (Figure 23), works for a regional airline and is still learning the layout of many of the airports. To demonstrate the application features for a typical taxiing situation, we describe Brian’s step-by-step interactions with ReTINA in the following scenario.
ReTINA: The Pilot’s Eye on the Ground 23
Brian has landed at a large airport at 10:00 pm and has continued his communication with ATC through approach. Brian has a typical flight route and has already acknowledged the app’s limitations and opened up Group 1 Airport Diagrams from the Favorites window (Figures 6 and 7). Upon opening the map, his current location is displayed and quickly orienting him. ATC verbally provides Brian with his taxiing instructions. Brian selects the route drawing mode and with his finger selects the corresponding turning points on the map. With each turning point that Brian selects a line is drawn highlighting his route (Figures 16 through 20). As the route is visualized on the map, written instructions populate on the side menu (Figure 20). By recording his instructions within the app, Brian can offload the task of remembering the taxi directions in working memory. Brian reads back these written instructions to ATC confirming his taxiing route. Brian has the Visual NOTAMs feature turned-on and is able to visualize the hotspots, construction, and closed runways at the airport. These features provide Brian with better situation awareness and allows him to focus on taxiing the aircraft.
5.5 User Testing The prototype was displayed on a Dell Inspiron Duo tablet computer with touch capability. The laptop screen can be folded down so that only the touch screen is accessible to the user. The screen size is 6 inches x 10 inches and all images of the prototype were set to the resolution dimensions for viewing on a mobile tablet device. Two pilots agreed to participate in the user testing of the ReTINA prototype. After introducing the pilots to the overall design challenge of reducing runway incursions, each pilot was instructed to locate and select various features of the application including the layer menu, turning on and off traffic and visual NOTAM display, and the notepad. The pilots were then asked to draw a taxi route on the map using the drawing function. Both ReTINA: The Pilot’s Eye on the Ground 24
pilots intuitively used tablet gestures to zoom in and out of the Airport Diagrams without instruction from the researchers. Finally, the pilots completed a survey regarding their subjective experience with the application. As part of the usability survey, the pilots were asked whether they believed the individual features of the prototype would increase or decrease runway incursions. Their responses from this portion of the survey can be found in Figure 24. Generally speaking, both pilots responded favorably to the idea of an application for displaying current aircraft location, visual display of NOTAMs, and the potential for including traffic. It was suggested that the device be used with a clipboard to stabilize it and that a stylus would not be necessary because they can be dropped or lost. When asked how likely they would be to use the graphical display of NOTAMs on the app both responded very likely (based on a 5-point likert scale ranging from very unlikely to very likely). They both also expressed that this feature would help to decrease runway incursions. Both pilots expressed some concern that interacting with the map (either via writing down the taxi instructions or by drawing the route on the map) could cause some distraction and detract from looking out the window. However, both also felt that the benefits of the application outweighed the potential risks. They were both excited at the prospect of integrating ground traffic into the application and both responded favorably to adopting mobile device technology if the application was well designed and sanctioned by the FAA.
5.6 Scalability of the Design The first iteration of ReTINA will contain only basic location information and the visualizations of NOTAMs. However, a significant benefit to designing software in a mobile application is that it can easily be modified and integrated with emerging ReTINA: The Pilot’s Eye on the Ground 25
technologies. NEXTGEN will require ADS-B transponders in all aircrafts flying in class A, B and C airspaces by 2012 (A. Gertsen, personal communication, February 24, 2012). Software applications like ReTINA should leverage this addition of location information in a visual representation for pilots. At airports with ASDE-X and LCGS systems ground surveillance can identify objects regardless of ADS-B transponders, including foreign objects like rogue vehicles and wildlife (R. Higginbothom, personal communication, March 30, 2012). Finally, ASDE-X data can be incorporated with ipads as well to improve location accuracy. Because a user-centered approach was used to develop ReTINA, the interaction foundations have been established and additional functionalities can easily be added in the future. Pilots are not the only potential users of ReTINA. Ground vehicle operators can use ReTINA on devices as small as smartphones. Once ReTINA is successfully integrated with surveillance monitoring systems that can detect objects that are not tagged with transponders, the current location and movement of all ground traffic will be available for users. In this way, ground vehicle operators can also benefit from the visual display of NOTAMs relevant to taxiways and runways. This expansion of the application will allow all stakeholders on the ground to share the same visualization of the traffic. A visual representation of the airport surface will assist the operators in accurate and safe navigation reducing runway incursion. Another technological advancement that can improve the performance of ReTINA is integration between ATC and the pilot. ATC controllers can standardize taxi routes that can be sent wirelessly to the pilot’s mobile device. The application will automatically highlight the route on the map diagram for the pilot. This can be done
ReTINA: The Pilot’s Eye on the Ground 26
either via text commands or via a voice to text conversion of ATC communication. This feature would further reduce pilot workload because the route will be automatically drawn on the map. Pilots will be able to learn their routes simultaneously via auditory and visual signals allowing for a deeper encoding of the information. In summary, smart devices are an excellent platform for interfacing with emerging technologies because they can be quickly and easily adapted and can serve as design models for future systems that will be incorporated into cockpit displays. The addition of traffic and surveillance information and potential integration with ATC communications are examples of the potential ReTINA will have to reduce runway incursions.
6. Interactions with Airport Operators and Industry Experts Alex Gertsen, President and Founder, Aviation Fury, LLC. Mr. Alex Gertsen is President and Founder of Aviation Fury, LLC, a technology company that provides technology solutions to airports, the Federal Aviation Administration, and technology developers. Alex has previously served as Director of ATC Programs at Air Traffic Control Association (ATCA) and Director of Regulatory Affairs at American Association of Airport Executives. According to Mr. Gertsen, he believes that pilot error due to distractions, fatigue, or confusion are the main cause of runway incursions, in addition to vehicle and pedestrian deviations. Specifically in general aviation, Alex believes that pilots move without proper clearance because they are often unaware of hold lines. Mr. Gertsen responded favorably to our design idea of providing digital maps to pilots via portable tablets and felt that traffic information would be helpful for pilots because it is only currently available to ATCs. His main concern regarding a digital tablet application
ReTINA: The Pilot’s Eye on the Ground 27
was that it not create overload for the pilot, but he felt that our design idea would be particularly beneficial for planes without moving map displays. Alex stressed the importance of incorporating our displays into other ground vehicles besides aircraft and the importance to incorporate all airport movement into our design. Todd A. Cox, C.M., Airport Manager, St. Lucie County International Airport. Mr. Todd A. Cox is currently the general airport manager at Port St. Lucie Airport in Port St. Lucie, Florida. Mr. Cox has over 33 years of aviation experience, including time as an air traffic controller. As a general aviation airport manager, Mr. Cox was able to provide insight into the various operational differences between general aviation and commercial airports, as well as the differences in procedures due to technology and funding disparities. Rich Burton, National Lead LCGS, San Jose Tower. Richard provided us with expert knowledge on the LCGS system along with information about the workload of ATC. He described the system in San Jose and identified how it will be used throughout the trial period. Richard went into detail about the technology behind the system. He also indicated that pilot’s situation awareness may be a leading cause of runway incursions. He believed that our ipad app will increase situation awareness and most likely lead to fewer runway incursions. Richard W. Loewen, National Air Traffic Controllers Association. Mr. Richard Loewen is a member of the National Air Traffic Controllers Associations (NACTA). Richard has been an Air Traffic Controller for 23 years and is currently based in DallasFort Worth, Texas. He is also a Runway Safety Action Team (RSAT) member. The RSAT’s purpose is to address existing and potential runway safety problems and issues.
ReTINA: The Pilot’s Eye on the Ground 28
Richard provided insight into the activities of ATC as well as feedback on our design ideas. Richard stressed the importance of maintaining ATC’s situational awareness of ground traffic. Richard emphasized the importance of heads-up displays for both controllers and pilots. Per Mr. Loewen, root cause analysis of prior incursions have indicated that flight crew time spent on programming or looking down at maps have resulted in pilot errors. Richard thought that providing pilots with an airport diagram on a tablet computer would increase pilots’ situational awareness. He was also receptive to the fact that the information could also be shared with other ground vehicle operators via the use of portable devices. He also mentioned that NOTAM alerts can be very complicated and a shared display with updated information would enhance the situational awareness of pilots. Jeff Anderson, Commercial Pilot. An interview with American Airlines pilot Jeff Anderson (6000+ hours of flight time) informed us that airlines are starting to implement iPads in the cockpit. He gave us an overview of the current technologies in the cockpit at present and other ways in which airlines are trying to mitigate runway incursions. He liked the idea of an iPad app that would consolidate the current airport maps into an electronic form. He stated that this would increase pilots’ situational awareness on the runway as long as there was no cognitive overhead in interacting with the app. Chris Stephenson, Terminal Technology Coordinator, NATCA. Chris Stevenson is a terminal technology coordinator. He worked for the safety and technology department and sat on the runway safety council. He said most runway incursions occur because of a breakdown in communications between pilots and ATC. Gave examples such as: ATC can forget to include a hold short; pilot reads back correct instructions but
ReTINA: The Pilot’s Eye on the Ground 29
forgets a hold short; pilot can be disoriented; ATC gets distracted and issues right instructions but cannot monitor the aircraft closely. Luke Basso, General Aviation Pilot. Luke Basso is a general aviation pilot with 450 hours of flight time who has recently received his commercial license. Luke let us know that some pilots use iPad apps for aviation. He liked the idea of using an iPad besides flipping through a handbook of airport maps; thought that “it would be more efficient.” Luke also told us that ATCs don’t usually have time to walk you through all of the taxiway navigation and that during long taxi routes an app would be helpful. He suggested that adding NOTAMS and other alerts would be helpful as well. He also suggested that showing other traffic could be helpful. Rob Higginbothom & Ben Marple, FAA & Veracity Engineering, LCGS Division. We toured Veracity Engineering where the LCGS systems are being monitored/evaluated. Rob and Ben gave us a presentation on the history and direction of the LCGS systems. He spoke about the expected timeline for implementing more systems. We were also able to interact with the LCGS interface. James Heath, Commercial Airline Pilot. James Heath is a first officer for a large, U.S. regional airline. He holds a commercial pilot’s license, CFI-I/MEI certificates, and CL-65 and DHC-8 SIC type ratings. He has experience in commercial and flight training environments. James provided information concerning pilot responsibilities in both commercial and general aviation domains. With his background in flight instruction, James was receptive to the idea of a navigational app for assisting low-time pilots with navigating unfamiliar airports. His opposition to the use of an app in a commercial
ReTINA: The Pilot’s Eye on the Ground 30
operation guided the group’s focus towards GA as the specific user group for which our efforts should be targeting. Jim Slate, Manager, Dulles Air Traffic Control Tower, FAA. Jim Slate is the manager of the Dulles Air Traffic Control tower. He talked about how ADS-B gives information to pilots, and how ASDE-X is helpful to controllers. He also explained that the company who works on ASDE-X is Sensis. Jim also talked about the concept of a “moving map” in NextGen, which will use information from ADS-B to show locations of other pilots, as well as show pilots their own location and terrain and weather patterns around them. He also talked about the technology of runway status lights, and how they are especially helpful in airports that consistently have poor visibility. He talked about how runway status lights incorporate data from ASDE-X to identify if there is a runway incursion, and that the lights will notify the pilots. Christopher M. McCalla, Commercial Airline Pilot. Christopher McCalla participated in a user test on April 25th, 2012 on a prototype of the iPad application. Mr. McCalla reported that he had 23 years of commercial flight aviation, and 3 in general aviation. From his usability survey, he reported that the application may be distracting to pilots that rely too much on the application. He also reported that he currently uses iPads in flight. Interestingly, while he reported that he prefers using paper maps, he thought the addition of graphical NOTAM depictions, ability to view other runway traffic, and interaction with the planning of runway routes all could be expected to decrease runway incursions. When asked to describe our application in 3 words, he used the words “pictures,” “ease,” and “potential.”
ReTINA: The Pilot’s Eye on the Ground 31
Michael McClintock, Commercial Airline Pilot. Michael McClintock was another subject during our user test on April 25th. He reported to have 4 years experience in general aviation, and less than 1 year in commercial aviation. He believed that iPad use in the cockpit with cut down in shuffling through maps and books in the cockpit. He also reported that the displays can benefit pilots in that they can identify what is occurring around them in the ground environment in quick, real time. He believed that this feature would be one of the stronger benefits of the application in terms of minimizing the prevalence of runway incursions, whereas the “draw a route” feature wouldn’t be as beneficial
7. Projected Impact of Design and Findings Oftentimes technologies are implemented as treatments to problems as opposed to preventive measures. ReTINA provides pilots with location information that leads to higher awareness of their environment. Knowing where you are and where you are going are integral pieces of information when navigating an airport surface. Incorrect movements can lead to dangerous situations such as runway incursions. Pilots will be able to identify problem locations and movements before they become dangerous situations. A visual display of the airport surface will increase pilot confidence about their movement. Navigating large airports can be extremely difficult due to the number of taxiways. ReTINA will enhance pilot navigation and possibly decrease the number times pilots ask ATC to repeat taxiing instructions. ReTINA leverages existing technologies offering a lower price point than almost any other runway incursion prevention system available. The system utilizes an off the shelf technology, the iPad along with any ground surveillance system that is available.
ReTINA: The Pilot’s Eye on the Ground 32
The main cost involved with ReTINA is the iPad. ReTINA provides an extremely low cost alternative to cockpit retrofitting and other custom technologies. ReTINA will be able to interface with future technologies providing a richer degree of information to increase pilot SA. One of the problems with new technologies is the misguided ideology that more is always better. More information or functionality does not necessarily mean better. A design based in human factors principles will continue to influence the design process with ReTINA as technologies move forward. The extra set of eyes that ReTINA provides is invaluable to aviation safety. While LCGS systems are only beginning to be tested, within the next 10 years there may be upwards of 30 live systems across the nation (R. Higginbothom, personal communication, March 30, 2012). Pilots who fly in and out of airports supporting ground surveillance technologies will be able to improve their situation awareness through the ReTINA interface. ReTINA will provide pilots with a level of awareness that will improve their performance and decrease instances of runway incursions.
ReTINA: The Pilot’s Eye on the Ground 33
Appendix A Team Contact Information
Faculty Advisor Tyler H. Shaw, PhD George Mason University Psychology Department 4400 University Drive, 3F5 Fairfax, VA 22030 P. 703-993-5187 F. 703-993-1359 Tshaw4@gmu.edu
Graduate Student Team Members Jane H. Barrow
Brian D. Kidwell
4131 Fountanside Lane #204 Fairfax, VA 22030 858-735-6088 jane.h.barrow@gmail.com
10241 Evesham Lane Fairfax, VA 22030 217-549-9305 bkidwell@gmu.edu
William J. Benson
Haneen Saqer
10200 Chase Commons Dr., APT 304 Burke, VA 22015 845-489-1480 willbenson33@gmail.com
3851 Aristotle Court #1-304 Fairfax, VA 22030 713-884-0844 hsaqer@gmu.edu
Eric J. Blumberg
Melissa A.B. Smith
7920 Sutcliffe Drive Raleigh, NC 27613 919-673-9296 ericjoshua@gmail.com
9930 Fairfax Sq APT 15 Fairfax, VA 22031 954-401-0107 mabsmith@gmail.com
Devon B. Kelley
Jonathan D. Strohl
10451 Malone Court Fairfax, VA 22032 518-935-3413 kelldb12@gmail.com
4104 Summit Heights Way #222 Fairfax, VA 22030 610-504-2752 jonstrohl@gmail.com
ReTINA: The Pilot’s Eye on the Ground 34
Appendix B Description of the University George Mason University's growing reputation as an innovative educational leader is rooted in Virginia's strong educational tradition. Since 1972, the university's development has been marked by rapid growth and innovative planning. Drawing prominent scholars from all fields, George Mason's outstanding faculty includes two Nobel laureates in Economic Science, the Robinson Professors, a group of outstanding scholars committed to undergraduate teaching and interdisciplinary scholarship, a Pulitzer Prize winner, IEEE Centennial Medalists, and recipients of numerous grants and awards, including Fulbright, National Science Foundation, and National Endowment for the Arts awards. Endowed chairs have also brought many artists and scholars to campus. The Arch Laboratory is housed in the Department of Psychology at George Mason University and is the main research and training facility of the graduate program in Human Factors and Applied Cognition. The Arch Lab has approximately 5,000 sq. ft. of dedicated space for research in human factors, cognitive psychology, cognitive neuroscience, and neuroergonomics. Agencies that have funded or are currently funding research in the Arch Lab include NIH, NSF, ONR, DARPA, FAA, NASA, NTSB, DoD, the Army Research Laboratory, and the Air Force Office of Scientific Research.
ReTINA: The Pilot’s Eye on the Ground 35
Appendix C Description of Non-University Partners This project did not incorporate the use of any non-university partners.
ReTINA: The Pilot’s Eye on the Ground 36
Appendix D – Signoff Form by Advisor
ReTINA: The Pilot’s Eye on the Ground 37
Appendix E Evaluation of Educational Experience E-1
Dr. Tyler Shaw, Faculty Advisor This was my first year serving as faculty advisor for the FAA design project. The
eight students who participated in the design project were a very diverse group, consisting of 1st year MA students and a range of 1st to 4th year doctoral students. What I was very impressed with was the ability of the group to self-organize; the more experienced doctoral students stepped into a leadership position, and every student seemed to acquire a role that played to their individual strength. Overall, the team was extremely well-balanced and communicated well about vital aspects of the project. I feel that they have learned a great deal about design, as well as team building and communication, having been involved in the FAA competition. This project was entirely student-driven. The eight students did not complete the design project as part of a course requirement-- the entirety of the project was conducted in their spare time. This also posed, in my opinion, the biggest challenge for students, as they had to leverage completion of the project with other responsibilities such as their coursework and individual research projects. Students overcame this potential obstacle through precise coordination, clearly defined roles and delegation of responsibility, and weekly meetings spanning the length of an entire 15-week semester. My role was largely one of consultation — students handled all aspects of the project, ranging from a conceptual model for the design, to the development of a workable project, and met with me to discuss ideas. This is a testament to both the student’s ability to manage time effectively and their independence as young scientists and designers.
ReTINA: The Pilot’s Eye on the Ground 38
At the outset of the project, few students had in-depth knowledge of the current technology used in aviation. They spent a substantial amount of time ensuring that they had adequate knowledge of the most relevant features, and truly did their “due diligence� in making sure that no critical piece of information was overlooked. I think students may have been a little taken aback by the tremendous strides aviation has made over the past decade, largely due to the NextGen initiative, and may not have anticipated spending so much of their 15-weeks conducting research. However, once they surveyed the existing literature and technologies, they formulated a design and executed their plan of attack flawlessly. One of the more impressive aspects of the design process was the tangible product, the ReTINA prototype, that was developed during the course of the design process. The prototype is truly remarkable, and I think it was very beneficial for students to get a sense as to the difficulties in designing a product that could potentially be used in the field. When the prototype was developed, the students conducted a small usability study with pilots to test the usability and intuitiveness of the design, and the results of that study suggest that it could be potentially well-received. This was very encouraging to myself and the students, especially since so much time and effort was allocated to building the system from the ground up. In sum, students seem to really benefit from having taken part in this experience. I personally learned a great deal more about aviation, and we all learned about each other in terms of the individual contributions we can make to these types of projects. In addition, the interaction with the very polite, yet honest, subject matter experts reemphasized to the team the importance of ecological validity and feasibility within any
ReTINA: The Pilot’s Eye on the Ground 39
design endeavor. Thus, an important lesson learned from this experience is that we can never lose sight of the importance of simplicity and intuitiveness in system design, something that can easily be overlooked by academicians. I hope to participate in the FAA design in the future and I look forward to working with the next batch of students.
E-2
Jane H. Barrow This is the second year I have participated in the FAA Design Competition for
Universities, the first being the inaugural year in 2007. This year, I led the design team, which was an entirely new experience for me. I learned a great deal about the process of leading a group towards a design goal, which I thought was hugely valuable. I have been part of a design team before, but it was a completely different challenge to lead the design process. The biggest challenge we faced as a team was completing the project in a single semester. The time it took us to develop a design idea was a bit longer than we originally anticipated, while impacted the amount of time we could spend on developing the prototype, user-testing the prototype, and then writing the paper. If we had had a few more months, I think the process might have been less stressful. Our process began with literature reviews and web searches to determine what the current solutions were and what the problems with those solutions were. We then transitioned to interviews to learn more about specific areas and get feedback about our initial thoughts on where we could make a difference in the problem space. We also interviewed pilots to better understand the task of taxiing, and to test our prototype once it was developed. Without these key industry interactions, I'm not sure we could have developed the design to the same degree. We were able to ask general and specific questions, and just about everyone that we interviewed was extremely helpful. This was one of the best aspects of the competition in my opinion - having the database of industry experts was a lifesaver. This ReTINA: The Pilot’s Eye on the Ground 40
project was learning experience in project management, as well as a way to utilize usability analysis and design techniques that we had learned about but hadn't been able to practice.
E-3
William J. Benson The FAA Design competition was a valuable learning experience that I was happy
to participate in this year. I was able to interact with professionals in the field and learn a substantial amount about ground operations and runway incursions. Not only did I learn about the field of aviation, but the experience also taught me about working in a large group with a variety of skills, on a project of a large magnitude. One of the biggest challenges our group faced was coming up with a unique idea. With the issue of runway incursions being a central focus in aviation, we were hard pressed to come up with an idea that not had already been done before. What we finally decided on was dependent on understanding pilot workflow on the ground, and thinking of ways we could minimize the mental workload that pilots often have to manage. I truly believe that the application we designed will reduce pilots workload by providing an efficient memory aid, as well as a helpful tool for gaining as much information quickly about any airport as fast as possible. Experts in the industry helped a great deal during the duration of our project, especially when determining what information we can and should present to the pilot. In the end, this project helped me determine that I would like to continue work in this field, and perhaps further into application design and development.
E-4
Eric J. Blumberg Participating in the FAA competition was a great learning experience. It was
enjoyable to identify a problem space, create a prototype, and do user evaluations. Completing the project involved a great deal of coordination and teamwork. As a group ReTINA: The Pilot’s Eye on the Ground 41
we successfully navigated a number of tense situations. We were able to logically attack the problems, enabling us to be object in our decisions. We did a good job of understanding one another's strengths and weaknesses. We utilized whiteboards and chalkboards to work out our problems. The visual information aids learning and understanding of new concepts. We did a lot of brainstorming; thinking out loud of various problem domains. Each of performed a literature review and examined the problem space extensively. We met and were able to come to a consensus upon what we thought were the greatest needs for pilots. We utilized subject matter experts extensively because they possess the true wealth of information on the topics. They provided us with invaluable information about the problem space . They also provided constructive feedback on our idea and helped us focus our product. This project was valuable to me. I improved my skills as an analytic researcher. This process also facilitated my ability to work in teams.
E-5
Devon B. Kelley The FAA Design Competition was a meaningful experience for me because it was
a project that required me to think critically about developing a system provided me the opportunity to get experience working in a large group on a long term project. Some challenges that faced our team were different personalities and individuals from different backgrounds in human factors, not just specifically aviation. Initially coming up with an idea to mitigate incursions was also a challenge. We started off with very broad ideas until we finally were able to narrow it down to the idea presented in this paper. We overcame all challenges and obstacle by working closely with each other and holding working meetings on a regular basis. Pilot awareness and communication with ATC were critical aspects in whether runway incursions occurred or not. Thus, our hypothesis was ReTINA: The Pilot’s Eye on the Ground 42
to create a system that would increase pilot’s situational awareness and perhaps enhance the communication process between the pilot and ATC. Interviews with industry professionals proved to be extremely useful to gathering information, ideas, and feedback about our design. I believe this project with provided me with an experience that will help me in the future, whether it is in the workforce or in conducting future research in aviation.
E-6
Brian D. Kidwell Participating in a group project of this order was a challenging but positive
experience. As the resulting design is intended for review by a committee with which we did not have direct contact, reliance on other group members for interpreting FAA desires was the only method of consensus regarding the ideal group focus. This necessitated group involvement and interdependence as our understanding of the aims of the competition were the sum of the group members' views of the desired result. The iterative process in which we investigated the problem of runway incursions, brainstormed solutions, and went through the steps of task analysis and prototype design effectively functioned as a level for ensuring that our group maintained project cohesion. Industry experts guided our approach at each step, tempering various untenable solutions with the realities of the National Airspace System. The largest obstacle in our design process was the process of consolidating the ideas and intentions of eight different individuals into a single, workable product. Even within a group of human factors students, each individual comes with a unique and varying background of skills, ideas, and domain knowledge. While this diversity occasionally created ideological congestion, the net effect is undoubtedly positive as group consensus hones and strengthens the ultimate result.
ReTINA: The Pilot’s Eye on the Ground 43
E-7
Haneen Saqer The FAA Design Competition was a very valuable experience for me because it
afforded me the opportunity to work in a multidisciplinary team on a specific applied program. Applying our human factors knowledge and usability skills to a real-world problem with a significant impact was very rewarding. It was exciting to work with a team whose members shared the goal of improving aviation safety by reducing runway incursions. My biggest challenge throughout this process was my lack of previous knowledge or experience with the aviation industry. To tackle this issue we interviewed as many pilots and air traffic controllers as possible. To develop our hypotheses we interviewed subject matter experts and conducted hours of research to determine current technologies available in aviation. Once our hypothesis for a mobile application was formed, we developed prototypes of the application and conducted user testing for evaluation and feedback. I learned a great deal about the intricacies of the aviation environment and the many stakeholders involved. I also taught myself how to use an interaction design software package for developing prototypes, which will be a valuable asset upon entering the workforce.
E-8
Melissa A.B. Smith The FAA Design Competition was a very rich learning experience, providing
various opportunities to learn about the pervasive problem of runway incursions, brainstorm and design various solutions, apply task analysis techniques to real-world scenarios, and collaborate with numerous industry experts and fellow graduate students. As a first year doctoral student, working on a group project of this magnitude was a novel experience, introducing me to the benefits and also the trials associated with large-scale group work. Coordinating amongst such a large group of people, from meeting times to
ReTINA: The Pilot’s Eye on the Ground 44
idea compilation, can be a challenge, but the team overcame this issue by having weekly meetings and using group collaboration technologies like a project wiki page, Google Docs, and Dropbox. The team hypothesis was developed after talking with industry experts about their perspectives on the issues surrounding runway incursions; these interviews and tours from the industry experts were invaluable to the progression of the project. This project provided a unique learning opportunity to explore the role of human factors within the area of aviation and has allowed me to utilize skills I will need throughout my graduate and future career.
E-9
Jonathan D. Strohl Previous to working on the FAA Design Competition, I had very limited
experience with human factors in the aviation domain. This competition not only increased my body of knowledge on the subject matter but also provided me with critical skills to take into the workplace. The initial creation of the design was a challenging process. A lot of time went into not only coming up with a creative and convincing idea to prevent incursions, but then molding this idea into a feasible solution. The group initially focused on general background knowledge through literature reviews on the history and current state of the problem of runway incursions. Once the problem space was appropriately defined, we focused on a variety of emerging technologies that could be implemented to address some of the most central contributing factors to runway incursions. My favorite experiences on the project involved the interviews with the industry professions. Our subject matter experts provided us with terrific insights into the problem as well as helped us to frame our design proposal. They added indispensable value to our final design. The competition particularly honed my interface prototyping skills. Not only did I get to work with graphic design software, but I was able to help in ReTINA: The Pilot’s Eye on the Ground 45
the production of a usable and testable interactive prototype. These skills will be very beneficial in my future career as a human factors professional.
ReTINA: The Pilot’s Eye on the Ground 46
Appendix F - References
Aviator Apps. (2012). Aviation Apps for for the iPhone, iPod Touch, iPad, and Mac. Retrieved from website: http://aviatorapps.com/ Bilton, N. (2011). F.A.A. Approves iPads in cockpits, but not for passengers. The New York Times. Retrieved from website: http://bits.blogs.nytimes.com/2011/12/14/fa-a-approves-ipads-in-cockpits-but-not-for-passengers/. Boehm-Davis, D. A. (2006). Improving product safety and effectiveness in the home. Reviews of Human Factors and Ergonomics 1(1): 219-250. Brewster, D. (2011). Cessnah T210 POH [iOS software]. Cooke, N. J. (1999). Knowledge Elicitation. In F.T. Durso (Ed.), Applied Cognition. Chichester, UK: John Wiley & Sons, 479-509. de Luchtvaart, R.V. (1979). Final Report and Comments of the Netherlands Aviation Safety Board of the Investigation into the Accident with the Collision of KLM Flight 4805, Boeing 747-121, N736PA at Tenerife Airport, Spain on 27 March 1977. Retrieved from website: http://www.skybrary.aero/bookshelf/books/313.pdf Department of Transportation. (2007). FAA Needs to Improve ASDE-X Management Controls to Address Cost Growth, Schedule Delays, and Safety Risks. Retrieved from website: http://www.oig.dot.gov/library-item/4973 Digital Cyclone (2012). Garmin Pilot [iOS software]. Electronic Code of Federal Regulations (e-CFR). (2012). Title 14, Aeronautics and Space. § 91.21. Retreived from website: http://ecfr.gpoaccess.gov/cgi/t/text/textidx?c=ecfr&sid=32ed53ffccb929bd13126f4654ed2363&rgn=div8&view=text&n
ReTINA: The Pilot’s Eye on the Ground 47
ode=14:2.0.1.3.10.1.4.11&idno=14 Endsley, M. R., (1995). Toward a Measurement of Situation Awareness in Dynamic Systems. Human Factors 37(1), 32-64. Federal Aviation Administration. (2009). Runway Safety - Runway Incursions. Retrieved from website: http://www.faa.gov/airports/runway_safety/news/runway_incursions/ Federal Aviation Administration. (2010a). Annual Runway Safety Report 2010. Retrieved from website: http://www.faa.gov/airports/runway_safety/news/publications/media/Annual_Run way_Safety_Report_2010.pdf Federal Aviation Administration. (2010b). Fact sheet – Airport Surface Detection Equipment, Model X (ASDE-X). Retrieved from website:http://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=6296 Federal Aviation Administration. (2012a). Airport facility directory. Retrieved from website: http://aeronav.faa.gov/pdfs/All_Hotspot.pdf Federal Aviation Administration. (2012b). Runway Incursion Totals for FY 2012. Retrieved from website: http://www.faa.gov/airports/runway_safety/statistics/regional/?fy=2012 Gabriel, J., Valovage, E., & Keller, S. (2003). Investigation of runway incursion prevention systems. Informally published manuscript, Engineering, Cornell, Ithaca, NY. Retrieved from website: http://engweb.engineering.cornell.edu/EngrWords/ingenuity/Keller_S_issue_1.pdf Gray, W.D. & Altmann, E.M. (2001). Cognitive Modeling and Human-Computer
ReTINA: The Pilot’s Eye on the Ground 48
Interaction. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors. New York: Taylor & Francis Ltd. (1), 387-391. Joint Planning and Development Office. (2004). Next generation air transportation system integrated plan. Washington, DC: Author. Jones, D. R., & Prinzel III, L. J. (2006). Runway Incursion Prevention for General Aviation Operations. Paper presented at 25th Digital Avionics Systems Conference. Kirwan, B. A., Ainsworth, L.K. (1992). A guide to task analysis. London, UK, Taylor & Francis National Transportation Safety Board. (2011). NTSB Most Wanted List. Retrieved from website: http://www.ntsb.gov/safety/mwl.html Patterson Jr., J. W. (2004). Evaluation of in-pavement runway guard lights (DOT/FAA/AR-TN04/49). Department of Transportation, Federal Aviation Administration. Retrieved from website: http://www.airporttech.tc.faa.gov/Safety/Downloads/TN04-49.pdf RocketRoute LTD. (2012). AeroPlates [iOS software]. Searidge Technologies. (2012). Runway incursion monitoring and collision avoidance system. Retrieved from website: http://searidgetech.com/ansp/rimcas Stone, D., Jarrett, C., Woodroffe, M. & Minocha, S. (2005). User interface design and evaluation. San Fransisco: Elsevier. Tech notes (2010). Lincoln Laboratory, Massachusetts Institute of Technology. Retrieved from website: http://www.ll.mit.edu/publications/technotes/TechNote_RWSL.pdf
ReTINA: The Pilot’s Eye on the Ground 49
Tehan, Geoff (2010). iPad GUI PSD. Retrieved from website: http://www.teehanlax.com/blog/ipad-gui-psd/
ReTINA: The Pilot’s Eye on the Ground 50
Appendix G - Figures Figure 1. Levels of situation awareness (adapted from Endsley, 1995)
Figure 2. Cognitive artifact task triad (adapted from Gray & Altmann, 2001)
Artifact (Product)
Interactive Behavior Cognition (User)
Task Environment
ReTINA: The Pilot’s Eye on the Ground 51
Figure 3. Operational Sequence Diagram (OSD)
ReTINA: The Pilot’s Eye on the Ground 52
Figure 4. Kneeboard placement of iPad
Figure 5. Instrument panel mounting for iPad
ReTINA: The Pilot’s Eye on the Ground 53
Figure 6. Disclaimer pop-up on homepage
ReTINA: The Pilot’s Eye on the Ground 54
Figure 7. Homepage with three customizable windows
ReTINA: The Pilot’s Eye on the Ground 55
Figure 8. Keyboard display for searching for airports
ReTINA: The Pilot’s Eye on the Ground 56
Figure 9. Airport Diagram with current location displayed
ReTINA: The Pilot’s Eye on the Ground 57
Figure 10. Notepad displayed overtop of the diagram for quick access to note taking
ReTINA: The Pilot’s Eye on the Ground 58
Figure 11. NOTAMs in complete text format displayed overtop of the diagram for quick reference
ReTINA: The Pilot’s Eye on the Ground 59
Figure 12. Layers menu sidebar for graphical overlays on the map and quick access to features
ReTINA: The Pilot’s Eye on the Ground 60
Figure 13. Future technology development and integration may allow for display of real-time traffic
ReTINA: The Pilot’s Eye on the Ground 61
Figure 14. Extraction of info. from NOTAMs allows for graphical display of runway/taxiway alerts
ReTINA: The Pilot’s Eye on the Ground 62
Figure 15. Selecting the drawing mode displays all possible turning points from current location.
ReTINA: The Pilot’s Eye on the Ground 63
Figure 16. Selecting a turning point draws a line to the point and activates the next possible turning points. Dots continue to be restricted to only accessible points to decrease screen clutter.
ReTINA: The Pilot’s Eye on the Ground 64
Figure 17. The next possible turning points are now active and the route has been updated.
ReTINA: The Pilot’s Eye on the Ground 65
Figure 18. The next possible turning points are now active and the route has been updated.
ReTINA: The Pilot’s Eye on the Ground 66
Figure 19. The user has selected the final turning point and the route has been updated.
ReTINA: The Pilot’s Eye on the Ground 67
Figure 20. Selecting the sidebar displays the route in textual format for easy read back by the pilot
ReTINA: The Pilot’s Eye on the Ground 68
Figure 21. Selecting the current location resets the route and allows the user to begin again if route has changed.
ReTINA: The Pilot’s Eye on the Ground 69
Figure 22. The settings menu allows the user to customize options and defaults. Here the user is able to assign individual maps to a group. These groups are then accessible on the homepage.
ReTINA: The Pilot’s Eye on the Ground 70
Figure 23. User persona.
ReTINA: The Pilot’s Eye on the Ground 71
Figure 24. Responses from two pilots after user-testing of the prototype.
Potential effectiveness
Likelihood of Features to Decrease Incursions
Pilot 1 Pilot 2
5 = Very likely to decrease runway incursions 4 = Likely to decrease runway incursions 3 = Neutral (will have no impact) 2 = Likely to increase runway incursions 1 = Very likely to increase runway incursions
ReTINA: The Pilot’s Eye on the Ground 72