Take Our Content to Go
Intro
Any commercial passenger or cargo airline can save a significant operational budget by reducing delays on the ground during the departure process. Yet, predicting taxi-out time is difficult due to uncertainties associated with factors under and out of the airlines’ control. Any credible machine learning model needs to factor in airport surface traffic, downstream traffic restrictions, runway configurations, weather, and air traffic control heuristics. One of the world’s most significant commercial aviation outfits contracted Mosaic ATM, a leading aviation AI company, to design and deploy machine learning approaches to predict departure runway and taxi-out time for an upcoming departure.
Problem Definition
The taxi-out time is the duration of time from when an aircraft “pushes back” from its gate to when it takes off from the runway. Mosaic’s customer provides these predictions to pilots on a new Aviobook1 electronic flight bag tablet app that integrates capabilities and information currently spread across many different apps. Some screenshots of how the forecasts appear on the app are in the figure below.
The taxi-out time predictions will help pilots decide when to “single-engine taxi”: taxi most of the way to the runway with only a single-engine turned on, turning on the second engine just a few minutes before take-off. The single-engine taxi decision is typically made by pilots within an hour of push back. Still, our customer asked for predictions up to four hours in advance of the expected push back time. Flight dispatchers could also better anticipate taxi-out times and plan consistently with corresponding likely pilot single-engine taxi decisions.
ML Driving Sustainability
When taxi-out times are relatively long, single-engine taxiing can save fuel. The airline anticipates that the fuel savings from additional single-engine taxiing attributable to the predictions provided in the app will be on the order of $3 million per year—sufficient on their own to justify the entire expense of developing the app. Mosaic has developed models and supporting software that produces departure runway and taxi time predictions for all mainline departures at three important airports (EWR, ORD, & SFO). In the coming months, Mosaic will roll these out at 27 additional airports.
Taxi Time ML Development
Developing the real-time data and software infrastructure to support these models and establishing appropriate connectivity between Mosaic and our customer was a tremendous effort that, fortunately, could stand on the giant shoulders of ATD-22.
Mosaic needed to layout a development plan that quickly delivered value to the airline and provided quick wins along the way. First, the data science team needed to develop a taxi-out time prediction model that factored in the variability of airport surface operations. Once this model was deemed acceptable, the team designed a departure runway prediction model and deployed an interface to integrate predictions into the app, getting insights into decision-makers hands.
Early on in model development, Mosaic trained models which used features derived from weather data (LAMP forecasts). While these proved helpful to the models, our team eventually learned that this weather data would not be available in the deployed system, so these features would need to be disregarded for now.
However, we also left out an essential element during initial taxi-out time model development because we didn’t realize that it would be readily available: predicted runways. Once the predicted runway model was ready, its predictions could serve as essential features for the taxi-out time prediction model.
Typically Mosaic’s team and most data scientists like to work with data that has is structured into tables consisting of columns and rows. But as anyone knows who has designed and deployed ML in real-world settings, this is hardly the case.
Collaboration with our airline customer was crucial to the accuracy of the predictions and the project’s success. There are several factors to model, so Mosaic implemented a feedback loop with the airline to discuss which variables were important in the predictive process, aligning on the outcomes that matter to pilots.
Working through user acceptance
Scope creep arises on many projects, and this effort has been no exception. Mosaic was surprised by two ML-creeps in scope related to user trust in model predictions.
The first was that a vaguely defined “percent confidence” be provided with the predicted runway and taxi-out time for each flight. The airline has developed requirements for the new Aviobook app with the help of a working group of pilots; that group indicated that a pilot’s single-engine taxi decision should be based not only on the predicted taxi-out time but also on the trustworthiness of that particular prediction.
To address this concern, Mosaic developed confidence intervals on the predictions, using similar techniques described here3.
The second creep in scope also related to the trustworthiness or interpretation of model predictions manifested in a request that model input features be provided alongside model predictions. In particular, the airline requested that the “future demand” and “future capacity” shown in the app screenshot above be provided by Mosaic and based on the same data processing that prepares data for the model. This effort ensures that these numbers are consistent with the predicted taxi-out times and help users identify cases when the model works with forecasted traffic demand data that may not align with other forecasts.
Conclusion
The runway prediction models provide the airline with the insights they need to make more efficient and sustainable decisions. The airline reduces ground delay, providing a better customer experience. Planes are not burning off as much jet fuel, reducing carbon emissions with single-engine taxi support. The overall departure process is streamlined as the airline listens to what the data and real-time situation is telling them.
These ML models offer similar benefits to the ATD-2 project Mosaic is alternatively working on, and NASA has calculated the following environmental outputs of implementing this technology.
This effort is ongoing as we continue to deploy and update these machine learning models to serve the commercial airline best.
