21: AI in the Control Tower?
The future of air traffic control from virtual simulation to real-world innovation.
090000Z DEC 24
Greetings,
As many in the #avgeek community will know, Microsoft's Flight Simulator 2024 was released a couple of weeks ago as the most recent update to the long-running franchise. The previous edition, Flight Simulator 2020, had introduced a new level of immersion by bringing the graphics to an almost photorealistic level.
The simulated world references satellite imagery to recreate the ground exactly as it appears in reality. Live weather data and traffic are integrated into the game, meaning that if it's raining where you're flying in real life, it will also be raining in the simulator. The simulator had long allowed virtual pilots to interact with key aircraft systems and execute detailed flight plans, but these improvements in realism brought the whole experience to another level.
In the recently released 2024 edition, Microsoft and the game's developer, Asobo Studio, significantly improved the gameplay—enhancing both activities and user interface. The graphics system and flight models saw only marginal improvements, which wasn't disappointing given how advanced these systems already were. The one letdown came with the simulated air traffic control system.
In this issue, we'll examine what I expected from this update and explore how it relates to current challenges in real air traffic control. From there, we'll analyze where the industry is heading in addressing these challenges and what steps are needed to maintain safety in the skies.
Fixing Air Traffic Control
That's right—it's broken and I'm going to fix it. What a presumptuous notion. In all seriousness, it doesn't take an expert to notice the challenges facing modern air traffic control. There aren't enough controllers, the infrastructure is aging, and the skies grow more crowded with each passing year. However, the solutions are far from obvious. Before we dive into these issues, let's consider what air traffic control actually entails.
Every day, at any given time, there could be more than 5,000 aircraft in the skies. The majority of the globe has air traffic control (ATC) coverage, with varying levels of infrastructure and sophistication. As the skies stay open 24/7, most ATC facilities are staffed on this schedule. Controllers handle all phases of flight—from departure and en-route navigation to landing—and do this tens of thousands of times each day.
An airport like Hartsfield-Jackson in Atlanta (ATL) might manage over 2,500 flights in a given day. ATL is one of the world's busiest and most complex airports, where ATC staff coordinates movements across five parallel runways. This translates to a given controller issuing about 1,500 instructions during a single shift. These must be timed and ordered appropriately to avoid conflicts, and the controller is responsible for maintaining constant communication with pilots and ground operations.
That's the real world. In a game like Microsoft Flight Simulator 2024 (MSFS), we can simulate a far simpler experience—as a pilot comes in to land in the virtual world, they receive a programmed audio message from the in-game traffic controller. Air traffic control in MSFS is a non-playable character, meaning they represent an interactive part of the game but cannot be chosen by the user as their role—players can only be pilots in MSFS1. When ATC calls in each event, the message is the same, with the current aircraft, airport, and conditions merged in.
"[Your Callsign], cleared for approach, runway [runway number]".
The user does not actually speak their response as one would when dealing with ATC in real life. Instead, they are given a short list of possible responses—selecting one activates the in-game pilot's voice and sends a reply to the virtual controller over the radio. It's a fairly simple system for anyone familiar with dialogue-based video games (popular in the 1980s/90s), but it's effective for simulating the experience for beginners.
But an advanced simulator like MSFS is not necessarily for beginners. Players can spend quite a bit of money investing in hardware to operate the game—yokes, throttles, and pedals—and invest even more time learning various aircraft and performing full flights. Every minute of every mile is simulated, so if you want to fly transatlantic, get ready for about 8 hours of seat time. It's a good thing there's simulated autopilot. The point is that these enthusiasts are bought in and want to make the experience as real as possible.
In my professional life as a software developer, I have been using AI technology more and more, both as a component of the products I build and as tools that I use. My main disappointment with the new version of MSFS was that the ATC system was not using more AI technology with a large language model (LLM). An LLM is the technology that has received all of the recent hype in the news with products like ChatGPT and similar competitors from Anthropic, xAI, Meta, and Alphabet—and with good reason. Anyone who has used one of these products will attest to the almost uncanny resemblance the generated speech has to natural human language.
Pilots should try this exercise sometime—using ChatGPT, give the model this prompt:
I would like you to act as an air traffic control simulation. I am a Cessna 172 flying into KRDU. I will play my role and you will play the role of the air traffic controller. I'll start by making initial contact: 'Raleigh Tower, Cessna N123AB, 10 miles northwest of the airport at 2,500 feet, inbound for landing, with information Bravo.'
A critical analysis of the responses will reveal some issues, but this is an untrained model with very little instruction, tuned for general prosaic responses. A fine-tuned model trained on decades of ATC chatter would operate near flawlessly. In the game, the experience could be tailored to introduce variability among different controllers and airports, with different speech patterns and dialogue trees that would further immerse the user in the virtual world.
Accents could change throughout the world, and more diverse and timely disaster scenarios could confront the virtual pilot if controller behavior was driven by an LLM. Similar behavior would have required an inordinate amount of resources to implement within a software system before LLMs essentially handled the language and basic logic for us.
While this seems like a missed opportunity for MSFS 2024 today, I am sure this technology will eventually make its way into the flight simulator world. Microsoft works closely with OpenAI, the owner and producer of ChatGPT. And where this really gives me hope, and will make for a very interesting study, is using flight simulators as a test bed for AI technology—specifically LLMs—in real air traffic control.
Even though I am quite the futurist, I am not yet ready to turn operations over to ChatGPT. It will be interesting to see how the logic of air traffic management can be handled by the virtual understanding of an LLM—and I have a feeling it's far more impressive than one might guess.
What I can more clearly envision is some sort of hybrid between AI technology and human controllers managing our future skies. It's unlikely that the general public will be totally comfortable with autonomous ATC anytime soon, and it's reasonable to have this skepticism. Healthy skepticism forces a sort of divining process that will help guide the right combination of automation and manual direction in the control tower.
What is inarguable, though, is the need to start preparing for the challenges of tomorrow as best we can today. The future will bring increased traffic—more flights, more passengers, and new and more diverse types of aircraft in the skies. This is not to mention the dawn of commercial space travel, which will invariably need to be accounted for as well.
All of this is exciting but challenging at the same time. Complexity can be managed, though, and when we are serious about our work and the activities it facilitates, we can achieve incredible outcomes. I would posit that the current air traffic control system, though aging, is one of these incredible outcomes—a modern marvel of human ingenuity that allows us to rocket 5,000 building-sized tubes of metal through the air in a near-constant global juggling act. So, let's add 100,000 drones, eVTOLs, supersonic jets, satellites, rockets, flying saucers, blimps, and balloons of all types to the mix, and maintain confidence that professionals will put the proper tooling in place to get the job done.
I'm curious about your thoughts. Would you feel comfortable flying in skies managed by AI air traffic controllers? The technology may be closer than we think, and the answer might not be as simple as choosing sides. The future of aviation safety lies in finding the right partnership between human expertise and artificial intelligence.—✈
P.S. — If you’re interested in some intense flight simulator action, checkout the Flightlines account on X at @flightlines247. I go live from time to time and stream simulated flights there and will in the near future re-stream these on YouTube: @flightlines247. 🫡
✈
Thank you for reading.
Flightlines will be back next week with more insights and updates from the world of commercial aviation.
Until then, safe travels and happy flying!
There are actually related pieces of software that allow users to simulate the ATC side of flight simulation. For example, the VATSIM network connects virtual controllers with virtual pilots and allows real human-to-human interactions while flight simming.
Digging the switch to first person voice! And it seems you have a doppelganger in that first photo.
I imagine there are lots of things I trust/use every day that are actually managed by AI - I actually wouldn't be that surprised if it turned out ATC was one of them and I only found out after the fact. Sometimes ignorance is bliss :D
In the Marine Corps years ago, I had friends who were air traffic controllers at Marine Corps facilities on Okinawa and Southers California. I was stationed at Marine Corps Air Station Iwakuni Japan SHARING THE AIR FIELD WITH THE Japanese Defense Force. I suspect there and at Futima Okinawa everyone spoke English. At the giant US Air Force base Kadina Okinawa they probably spoke English too. Languages can make things complicated!