My friend Maria at Daedalean wrote this article about how they are using AI to help pilots detect gliders. I thought it was fascinating and Daedalean gave me permission to share it with you! I hope you find it interesting as well.

Training A Helicopter/Airplane AI Copilot to Spot Gliders

Noticing all fixed wing, rotercraft and drones in time

by Maria Pirson

When developing our Visual Traffic Detection technology, it’s crucial to address all kinds of potential encounters. So we collect large data sets on any possible scenario. That way, we can ensure that our software performs reliably and safely in all situations.

With a conventional aircraft, be it an airplane or helicopter, large or small, there is a costly yet well-defined process for data collection. We have consistently conducted such flights over the years, accumulating a significant number of recorded flight hours.

There are, however, less common types of potential intruders: gliders, hot air balloons, paragliders, skydivers, etc, for which our tried-and-true methodology falls short. To accumulate enough hours and create a dataset to make our algorithms failproof for these intruders, we have to be creative and invent new approaches.

Here is the story of how we solved the problem of collecting the necessary data on gliders.

Gliding into danger

Gliders pose a significant risk both to themselves and others. EASA’s annual safety reports show that they have accounted for about 25-30% of aviation midair collisions over the last 10 years.

Regulations do not mandate gliders carry transponders, such as FLARM or ADS-B, and many do not. That means pilots must rely on their ability to spot gliders with their own eyes.

Unfortunately, gliders are hard to see. They have a relatively small, narrow frontal cross-section. Consider the frontal cross-section of a Jonker JS3 Rapture glider compared to that of a Cessna 172 (which fills the skies as the most popular single-engine aircraft ever built).

Viewed head-on, the problem is obvious – the glider cuts a thin silhouette against the sky. And from a distance of 100 to 200 meters, it gets worse. The glider’s narrow wings become nearly invisible.

But the problem is that it’s really hard to collect data on gliders.

Normally, we mount our cameras on a small plane, such as a Cessna, or on a helicopter. Cessnas are, unfortunately, much less maneuverable than gliders, which make sharp turns and sudden pull-ups in taking advantage of developing weather conditions to maintain altitudes.

While helicopters are very maneuverable, they pose a different threat – prop wash. The force of the current of air from its rotors can be so powerful that it could forcefully divert a glider off course. Navigating a helicopter through a group of gliders, as a pilot vividly describes, “would be like putting an elephant in a glass store.”

An additional problem for collecting data on gliders stems from their lack of transponders. Not only does the lack of transponders hinder identifying them in the sky, as mentioned above, but it also makes collecting good data nearly impossible. That’s because good data requires ground truth.

When we collect data, we are essentially videotaping aircraft. That videotape must be analyzed by people to annotate the various objects so as to train the algorithms on what is what – an airliner, a helicopter, a glider. But to make the algorithms even more powerful, we pair the purely visual information of the videotape with information collected from transponders such as FLARM and ADS-B. That information conveys distances and GPS coordinates – the “truth” about the target aircraft’s location over the ground. Together, all this information creates algorithms offering pilots powerful situational awareness.

To approach these problems, Mark Travner, a then-intern at Daedalean and an expert glider pilot, had an idea: Why not use a glider to gather data on gliders?

Competing for safety

Gliding is a competitive sport. During competitions, there are more than 50 gliders flying for 5 to 10 days in a small area, one long flight per day. No problems with maneuverability or prop wash. And all of them are mandated to be equipped with GPS trackers (for scoring) along with FLARM (for safety). The full records for every glider are publicly available afterwards. So Mark entered some glider competition, something he has done nearly every summer for nearly a decade.

But just as one set of problems was solved, another appeared.

Mounting a camera on a glider would cause a loss of around 15-20% of flight performance. This was no small matter. Mark’s glider needed to stay competitive so that he could keep close enough to the field of competitors to record encounters. So, Mark designed and 3D-printed aero cover whose sleek teardrop shape mitigated aerodynamic disruption. (An unexpected side effect for him was to get searched at each airport on the way to the competitions because the aero cover’s shape can appear alarmingly similar to that of an explosive device.)

With the camera through security and mounted on the glider, Mark was ready to take flight. He flew in two big competitions in Central Europe known for attracting large numbers of competitors.

On each flying day, Mark started last, letting everybody go in front so that Daedalean cameras had a nice view of all the other aircraft. He captured hundreds of encounters each day. And a great variety of encounters: Distances ranged from 50 meters to 10 kilometers and included trailing, head-on, turns, dives, pull-ups–everything needed to train a machine on what to look for when scanning the sky for gliders.

This large range of encounters was far better than what we would usually achieve. Normally, we arrange to fly two aircraft to collect data. One of them is equipped with our cameras, and one is a target aircraft. We usually average between 20-30 encounters per hour. With Mark’s glider, we got around 1,000 encounters per hour. Considering that powered aircraft can cost nearly 1000 EUR per hour to operate, the unpowered glider option was extremely cost-effective.

By summer’s end, Mark took flight on

• 13 flying days
• for a total of 60 flight hours,
• during which he collected more than 32 hours of data
• while traversing more than 2,800 kilometers
• and engaging in thousands of encounters.

Our team of annotators is now hard at work analyzing this trove of data so that our Visual Traffic Detection system can help create safer skies for gliders and all those who encounter them.

But this is just the beginning. Next summer’s competitive season is just around the corner.

And beyond that, who’s to say what comes next? Drones? Paragliders? Hang gliders? Skydivers?

Maybe a hot air ballooning competition – after all, Mark already has experience as a co-pilot/navigator at the World Hot Air Ballooning Championships.

I hope you found Maria’s article as interesting as I did!

If you liked the amazing glider photos, you can see more in this Daedalean video about the glider data collection:

Please make Maria feel welcome in the comments and let her know if you have any questions!

-S.