In this white paper, we explore Mosaic ATM’s air traffic controller communications training tool, the Reconfigurable Interactive Communications Training Environment (RICTE), which leverages artificial intelligence to simulate realistic, multi-modal communication scenarios, enhancing the training effectiveness and proficiency of air traffic controllers.
Introduction to Air Traffic Controller Communications Training
“You fight like you train” and “you play like you practice” are two well-known expressions that stress the importance of learning and repeated execution to obtain a high level of proficiency at a skill or job. The more realistic the training environment, the quicker trainees will be ready to execute tasks and accomplish real-world goals. On the other hand, a poor or unrealistic training environment will only impart partial knowledge of the necessary skills. They will be forced to gain currency and proficiency on the job in real-life conditions, leading to mistakes, delays, and costly resources. Implementing better training programs using realistic training environments will greatly mitigate any risk of training shortfalls. One skillset that requires extensive training is voice control of aircraft.
In the Air Traffic Control (ATC) environment, controllers are responsible for the safe and efficient handling of aircraft within their area of responsibility (AoR). The aircraft and aircrew depend upon the air traffic controllers for situational awareness, traffic flow, separation, clearances, coordination with external entities (such as control towers and ground support teams), and direct support under emergency conditions. Controllers must communicate with several entities simultaneously via voice and text/chat.
The ability to hear, read, comprehend, and filter ongoing communications and then take appropriate action over those parallel communication channels and workplace tools is an acquired skill that takes significant training and years of experience to master. Learning this ability requires intelligent and realistic training systems and tools for both initial qualifications and ongoing proficiency. The ability to introduce these real and parallel interactions into a training system will increase the quality of instruction, lessen the time and attention burden on instructors, and allow for more realistic training sessions and scenarios.
Reconfigurable Interactive Communications Training Environment (RICTE)
Mosaic ATM’s Reconfigurable Interactive Communications Training Environment (RICTE) can easily evaluate and generate text, chat, and verbal communications and work within, alongside, or complement existing training systems using cutting edge artificial intelligence (AI) and machine learning (ML) capabilities such as domain simulation models and deep-learning (DL) -based Dialogue Agents. RICTE is a demonstrated capability designed and developed for the U.S. Navy under small business innovation research (SBIR) funding. The initial target environment of RICTE is the ATC domain, but it can also be adapted to other domains where personnel must track multiple activities and interact using multiple forms of communications in parallel.
We have brought together a set of interconnected components that work together to increase the
efficiency and realism of the training exercises and systems. This is accomplished by simulating background communications, evaluating and generating interactive communications, providing or working within existing simulation environments, accommodating instructor scoring of the trainees, and providing a playback of entire training sessions. Figure 1 shows one architecture configuration for RICTE (as a demonstration prototype system), where the simulation model drives the training scenario, the speech-to-text model and text interpreter process the trainee input, the statement generator and text-to speech model provide feedback to the trainee, the Dialogue Agents provide the domain and context appropriate dialog, and the SPOTLITE model collects the instructor’s scores and evaluations.
RICTE Components
Individual components and models seen in Figure 1 were developed from a combination of
previous Mosaic ATM research, including a recent Navy Phase I SBIR, Commercial Off-the-Shelf (COTS)/open-source services, and future tasks. Additionally, the SPOTLITE service component is a product
of Aptima, Inc., which collaborated with Mosaic ATM on Phase I work.
The RICTE components are as follows:
- Simulation Model: intermediary between RICTE components and a third-party Simulation and/or training scenario system. In the absence of a third-party simulator, the Simulation Model is fully capable of simulating the entities in the training scenario. Besides the simulation modeling capability, its role is to (1) provide aircraft state information from the simulation system and pass that onto the Dialogue Agents and (2) to interpret commands received from the Text Interpretation component and forward the commands to the simulation system. rminal airspace) and the commands represent the trainee’s commands (acting as a tower controller). Figure 2 shows a diagram of the Simulation Model component, its affiliated components, and a list of commands and simulation states it has been configured to process.
- Dialogue Agents: ML models that were derived from the DialoGPT GPT2 model developed by Microsoft and pre-trained on 147 million conversation-like exchanges. The models were then given additional training (via transfer learning techniques) using a corpus of text-based dialogue data. These models are used to determine the appropriate textual output for a simulated entity given the entity’s current state and interactions with other entities and can additionally initiate verbal communications when triggered by certain states and commands from the Simulation Model. The agents have two specific functions within the RICTE system: (1) they can interact with the trainee, providing a representative of one or more entities that directly communicates with the trainee, and (2) two or more agents that talk between themselves, providing realistic background communications that may either lead to a more direct conversation with the trainee or act as “noise” that must be ignored by the trainee.
- Text-Interpreter: Translates the text output from the Speech-to-Text component and converts it into a command or response, along with a set of identifying parameters that is then sent to the Simulation Model and the Dialogue Agents. This component is implemented by leveraging a Spacy rule-based name entity parser to extract the commands, responses, and parameters.
- Speech-to-Text: A simple pass-through and connector that utilizes Amazon’s Transcribe Web Service. The component receives recorded voice data from the trainee and ships this off to the remote web service to be transcribed into text. It then takes the returned text and passes it to the Simulation Model and the Dialogue Agents.
- Text-to-Speech: Leverages the FastSpeech 2 text-to-speech pre-trained deep learning model, which was trained with 200 male and female voices. This variety of voices allows the component to assign different voices to the various entities in the simulation to increase the realism of the training exercise.
- SPOTLITE Evaluation Training Tool: This allows the instructor to evaluate the trainee’s performance both quantitatively and qualitatively. It reduces the instructor’s workload by providing system-based performance measurements that automatically notify the instructor, through SPOTLITE, when a student fails to respond to applicable communications within a configured period. Additionally, the component can be configured to bring up scoring sheets after specific events within the training exercise so that the instructor can provide immediate feedback and smoothly continue with the training. At the end of the session, SPOTLITE will then provide the instructor with a final scoring sheet and a summary of the student’s overall performance and score.
Proposed Future Development for the Air Traffic Controller Communications Training
RICTE was initially developed as a prototype for the U.S. Navy. Several areas can be improved with additional development:
- Increase the simulation/scenario domain beyond the terminal airspace (i.e., an airport tower controller) to cover the National Airspace System (NAS) and associated use cases (e.g., an airport ground controller, departure, ATC regional centers, etc.). Additionally, RICTE can be configured to work in other domains and training environments such as military air intercept controllers (AIC) flying in USAF E-3 AWACS, USN E-2C/D Hawkeyes, and serving on board USN surface combatants.
- Develop the Training Run Data Recorder to record the trainee’s communications, commands, response times, and other metrics to provide a comprehensive record of the training sessions, allowing for playback of the training sessions.
- Continue the development of the Dialogue Agent recognition and response performance by expanding the system states input to the Dialogue Agents to accommodate additional use cases of interest to ensure a broader and tighter integration between the state of the simulation environment and the responses provided by the agents.
- Identify metrics, scoring, and reporting criteria to include in the SPOTLITE evaluation system to reduce instructor workload while simultaneously improving instructor and student performance. The initial RICTE prototype contained only a handful of evaluation events and forms as a proof of concept; further work can be derived from training requirements, instructor scoring sheets, and internal items calculated by the RITCE system itself (response times, etc.).
Department of Defense and Commercial Applications
With decades of experience working for and with the Navy, the Mosaic team understands Navy and Air Force training and education needs, policies, processes, and organization. We can refine the concept, architecture, algorithms, and performance to support appropriate programs of record or other direct needs for this technology. Mosaic already provides systems and tools that are used extensively for ATC training in multiple domains of the civilian air transportation system. Our corporate strategic direction includes further expansion of these capabilities. We can leverage RICTE as part of our existing system offerings to the FAA to support extensive ATC training capabilities, as well as commercial offerings to airlines.
Conclusion: Air Traffic Controller Communications Training
The demands placed on air traffic controllers require not only knowledge but a high degree of practiced, reflexive skill—especially in managing complex communications. RICTE stands as a transformative tool in meeting this challenge. By simulating dynamic, domain-accurate communication environments and integrating seamlessly with existing training platforms, RICTE accelerates the path from novice to proficient, reduces instructor workload, and increases training realism. As aviation systems grow more complex and the airspace becomes more congested, Mosaic ATM’s RICTE represents a forward-looking solution that prepares controllers to operate at the highest level from day one.
