Take Our Content to Go

Introduction 

Modern command and control (C2) and intelligence, surveillance, and reconnaissance (ISR) systems face the challenge of managing vast quantities of data and hundreds of contacts of interest (COI) or critical contacts of interest (CCOI) simultaneously. Operators must analyze, predict, and verify the behaviors and trajectories of vehicles in complex, dynamic environments to ensure mission success and operational safety. 

Mosaic ATM, under a Small Business Innovation Research (SBIR) effort with the Navy, is developing the Real-Time Simulation (RTS) Agent for enhanced combat and airspace awareness. This advanced artificial intelligence (AI) and machine learning (ML) tool enhances surface and air track management by providing scalable, real-time decision support and situational awareness for tactical operations. In this paper, we provide a high-level description of the approach taken to provide real-time predictions and decision support for operators monitoring complex tactical situations.   

The Problem 

In tactical environments, operators often need to answer critical questions about vehicle identity, intent, behavior, and trajectory with incomplete or noisy data. These include: 

  1. Predicting intent: Where is the vehicle likely going, and what is it trying to do? 
  2. Behavioral consistency: Is its behavior consistent with expectations given its identity? 
  3. Trajectory estimation: How will the vehicle move, and where might it be in the near future? 
  4. Verification of predictions: Is the vehicle following the expected trajectory? 
  5. Future positioning: Where will it be if position updates are unavailable? 

Scaling this process to hundreds of COI/CCOI in a cluttered C2/ISR display is beyond human capability alone. That’s where the RTS Agent comes in.  

The RTS Agent Solution 

The RTS Agent is a modular, microservices-based architecture that uses AI and ML models to analyze and predict the identity, intent, and trajectory of vehicles. Initially developed for Navy C2/ISR combat systems, the RTS Agent for enhanced combat and airspace awareness has achieved Technology Readiness Level (TRL) 6 and is being adapted for broader applications. 

Solution Architecture 

The RTS Agent consists of a scalable architecture (Figure 1) made up of several microservices. The AI and ML models have been containerized into stateless microservices to help establish the identity and intent of aircraft and surface vessels. Key components include: 

  • TargetProducer: Transcribes incoming track updates into a common internal messaging format, abstracting the system from source-specific data formats. 
  • Fuser: Orchestrates message coordination, updates track positions, and integrates model results using a stateless, in-memory REDIS database. 
  • Publisher: Converts internal messages to output formats compatible with subscribing systems. 

This modular approach ensures compatibility with diverse data sources and C2/ISR systems while maintaining operational flexibility. Within our internal message, information about each tracked object is kept, including Identification Friend or Foe (IFF) modes and codes, radar kinematics (course, speed, altitude, etc…), electronic intelligence (ELINT), communications intelligence (COMINT), and other track observations passed into the system from integrated fire control and tactical data link networks. These messages are fed through a series of AI / ML algorithms. 

Figure 1. RTS System Architecture 

The AI/ML models are trained offline using large quantities of historical data and model updates that can be pushed to the tactical edge as connectivity allows. The current RTS prototype has been built and tested using millions of lines of historical Federal Aviation Administration (FAA) data from the System-Wide Information Management (SWIM) Flight Data Publication Service (SFDPS) and US Coast Guard (USCG) Automatic Identification System (AIS) ship data. The ML models were trained using one year of data from the Norfolk, Virginia area.   

AI and ML Models for Enhanced Combat and Airspace Awareness

At the heart of the RTS Agent is a suite of AI and ML models designed to transform raw data into actionable insights. Each model addresses a critical aspect of vehicle behavior analysis, from trajectory prediction to identity verification. Together, these models work in harmony to provide operators with a comprehensive understanding of their tactical environment. 

Predicting Future Movement: The ML Trajectory Predictor (MLTP) 

One of the key challenges in-vehicle monitoring is predicting where an object is heading. The Machine Learning Trajectory Predictor (MLTP) tackles this problem by analyzing the recent movements of a vehicle and comparing them to historical trajectory patterns. Leveraging a deep learning architecture known as Long Short-Term Memory (LSTM), the MLTP predicts the most likely future path, not based solely on kinematics but on how the current trajectory aligns with historically similar routes. This model shines in scenarios where environmental or operational context influences movement patterns, such as ships navigating harbors or aircraft adhering to established airways. 

Using Physics to Guide Predictions: The Kinematic Trajectory Predictor (KTP) 

While the MLTP excels at interpreting historical patterns, the Kinematic Trajectory Predictor (KTP) focuses on immediate physical dynamics. By examining the vehicle’s current speed, course, and altitude, the KTP generates a probabilistic forecast of likely paths. This model samples statistical distributions of movement to calculate potential trajectories, providing operators with a range of possible outcomes and their likelihood. Its physics-driven approach is particularly useful for rapidly evolving scenarios where real-time kinematic changes dictate behavior. 

Detecting Anomalies: The Trajectory Anomaly Detector (TAD) 

Not all movement patterns fit expectations. The Trajectory Anomaly Detector (TAD) identifies irregularities by comparing predicted paths with actual trajectories. Using the same LSTM framework as the MLTP, the TAD examines recent track history and flags any significant deviations from predicted movement. This capability is crucial for spotting unexpected behavior, such as a vehicle diverting from its expected route, potentially signaling a mechanical issue, evasive maneuver, or hostile intent. 

Predicting the Destination: The Behavior Predictor (BP) 

Understanding where a vehicle is going is just as important as knowing how it will get there. The Behavior Predictor (BP) leverages track history to determine a vehicle’s most likely destination. Like the MLTP, this model uses an LSTM recurrent neural network, but its focus is on analyzing the sequence of movements to pinpoint probable endpoints. This insight enables operators to anticipate vehicle intentions and make informed decisions before the vehicle reaches its destination. 

Verifying Identity: The Identity Detector (ID) 

Every vehicle tells a story through its characteristics, and the Identity Detector (ID) ensures that story checks out. For positively identified objects, the ID compares observed traits—such as a hull or tail number, speed, altitude, and typical routes—with a historical profile to confirm consistent behavior. For unknown objects, the ID analyzes observed characteristics against a database of known profiles, suggesting likely matches for identity or type. This model’s use of a knowledge graph ensures a nuanced, context-aware approach to identification. 

Deployment 

The RTS Agent is trained offline using historical data and deployed at the tactical edge with updates delivered as connectivity permits. Initial testing utilized FAA System-Wide Information Management (SWIM) and USCG Automatic Identification System (AIS) datasets. 

Individually, each model provides unique insights. Together, they form a cohesive system capable of addressing the complex challenges faced by operators in cluttered C2/ISR environments. By combining historical patterns, kinematic data, anomaly detection, destination prediction, and identity verification, the RTS Agent delivers an unparalleled level of situational awareness, empowering operators to act decisively in real time.   

Applications 

The RTS Agent is a versatile tool that enhances operational efficiency and safety across a wide range of defense and commercial applications. By leveraging its AI and ML capabilities, the RTS Agent for enhanced combat and airspace awareness offers transformative solutions for both the modern battlefield and civilian airspace management. 

Revolutionizing Defense Operations 

In today’s complex and dynamic combat environments, operators face an overwhelming volume of data as they monitor hundreds of contacts simultaneously. The RTS Agent steps in to streamline this process by providing real-time insights into vehicle identity, intent, and trajectory, helping operators maintain situational awareness and make informed decisions. 

Imagine a scenario aboard an AEGIS-equipped cruiser or destroyer, where crew members must differentiate friend from foe among a sea of radar returns. The RTS Agent’s advanced models allow operators to quickly identify anomalous movements, predict likely destinations, and verify identity—all while seamlessly integrating into existing C2/ISR platforms. Whether on the deck of a P-8 Poseidon tracking submarines, or aboard the E-2D Advanced Hawkeye coordinating aerial operations, the RTS Agent empowers military personnel to focus on mission-critical tasks instead of being bogged down by data interpretation. 

This capability becomes even more critical in preventing blue-on-blue or blue-on-white engagements, where positive identification of friendly or neutral forces is paramount. The RTS Agent not only reduces cognitive load but also enhances mission success by providing clear, actionable insights in real time. Its ability to process and analyze data at scale ensures that it remains effective, even in the most data-dense environments. 

The RTS Agent’s adaptability makes it a natural fit for various defense platforms, including future surface combatants such as the CONSTELLATION-class FFG(X), as well as airborne systems like the E-3 Sentry, RC-135 RIVET JOINT, and E-8 JSTARS. By integrating the RTS Agent into these platforms, the Department of Defense (DoD) can enhance the capabilities of its fleet and ensure dominance in multi-domain operations. 

Transforming Commercial Airspace Management 

Beyond defense, the RTS Agent for enhanced combat and airspace awareness holds immense potential in civilian applications, particularly in the realm of air traffic management (ATM). As the National Airspace System (NAS) evolves to accommodate advanced air mobility (AAM) vehicles, such as air taxis and drones, maintaining safety and efficiency becomes increasingly challenging. 

The RTS Agent addresses these challenges by offering precise trajectory predictions and anomaly detection for a diverse range of air traffic. For example, the FAA can use the RTS Agent to monitor new entrants in the NAS, ensuring their operations align with established procedures while identifying any deviations that might signal safety concerns. 

Additionally, the RTS Agent supports the integration of AAM vehicles by providing detailed insights into their behavior and intent, enabling air traffic controllers to anticipate potential conflicts and manage airspace more effectively. By leveraging historical data and real-time updates, the RTS Agent ensures that even as the skies grow more crowded, they remain safe and efficient. 

Conclusion 

The RTS Agent represents a groundbreaking step in applying AI and ML to address the complexity of modern C2/ISR operations. By enabling scalable, real-time predictions and decision support, Mosaic ATM is advancing the state of combat and airspace management technology, improving operational safety, efficiency, and effectiveness for both defense and commercial applications. For further inquiries or collaboration opportunities, contact Mosaic ATM

Categories: White Papers