Advancing Aviation Safety Technology with Augmented Reality

AR tools for pilots have become less of a “when” and more of a “how.”

Scottie Gardonio

In 2017, the worldwide revenue for commercial airlines sits at around $750 billion U.S. dollars. That’s just the commercial airlines. If you add in the private, corporate, military, and hobbyist sectors, the number jumps up dramatically.

So, with an industry claiming revenues most of us cannot actually fathom, it only makes sense that one of the new kids on the tech block, augmented reality, would soon find it’s way to hanging with the cool kids.

But, what does an aviation world enhanced with augmented reality look like? If you’re a consumer strolling on the outside of the aviation inner circle, you don’t have to pull out your smartphone or adorn your smartglasses just yet.

However, if you’re claiming your paycheck from a company who makes money off of circling the globe above the clouds, it’s time to make sure your seat backs and tray tables are in their full upright position and that your seat belt is correctly fastened, because we’re in for quite a ride!

Piloting Made Safer with AR

Take a moment to stick on your $2 airline headphones, activate the $10 in-flight wi-fi, hit play on Fly Like an Eagle, and imagine a world where pilots are able to utilize augmented reality to make every flight safer.

Before you get too far into the chorus, though, let’s look at a common display in cockpits. If you’re in an older plane, you have scores of dials and knobs surrounding the entirety of your space. Looking through the windshield takes your eyes off of your altimeter, and you are likely forced to waste small amounts of time to find that altitude information once again.

Pilot cockpit
Image Credit: AirSoc

Other cockpits have moved the pertinent information into what’s commonly referred to as a heads-up display. This, in what could arguably be called augmented reality, is a way of overlaying the most needed information on the windshield so pilots have an easier time searching it out.

Heads-up displays are still limited, though. They do not move around with the pilot’s head, so if the pilot looks to the left, he/she will still need to search out the centered information upon facing forward.

Instead, the ideal option would be to adorn a pilot with smartglasses that hold all of the pertinent information upon request.

As mentioned earlier, information such as altitude are important, but they are only the tipping point in what will ultimately be available for flight.

Think about what you’d want to see in AR assistance should you be using it to drive your car:

  • You’d want to see arrows showing you the roads to take to ensure you make your turns and remain in the right lanes.
  • You’d likely want something giving you an indication as to whether you are too close to something, such as a slowing car or a child jumping into the road.
  • Perhaps you’d want assistance in directing you into the perfect parallel parking spot.
  • Or maybe you just want something to tell you where to go when you enter that insanely cookie-cutter town-home suburb your in-laws moved into after retirement.

In similar ways, you can see the same types of capabilities being useful to pilots:

  • Arrows can direct you toward the runway — the right runway — even at elevations where the airport looks to be the size of a paperclip.
  • On takeoff, you see the flock of birds in the distance, but your AR glasses easily indicate how far away they are so you can better assess the potential danger.
  • When lining up for landing, it’s easier to feel confident that you’re angles and speeds are appropriate for the length and conditions of the runway.
  • And, when trying to navigate a busy, potentially unknown airport in search of a terminal or proper taxi-way, confidence can be had in the directions needed.

Now, in a slight pause, let’s address what some of you are most certainly thinking: what about autopilot? Doesn’t that do all the work?

Well, yes. Kind of. OK, no — not really at all. Not in the situations we’re discussing, anyway.

In the above examples, the takeoff and landing, the pilots are largely in control. In fact, takeoff and landing are the most dangerous times of a flight, as that is when the majority of accidents happen. So, any additional safety assistance should be welcomed.

Now, not all accidents are due to pilot error or lack of autopilot. Sometimes, in a freak accident, air traffic control has directed a pilot incorrectly. Sometimes a flock of birds cripples an engine. But wouldn’t you feel safer if you knew every option is utilized for keeping everyone on board as safe as possible?

Thankfully, the previously-mentioned fantasies are actually reality.

In an industry where billions are spent and earned, there is an emerging industry of cockpit-based AR. One such company, Aero Glass, is already showcasing smartglasses with the same capabilities we were fantasizing about just moments ago.

The future of the tools we give to pilots via AR will continue to evolve, but it has become less of a “when” and more of a “how.” While companies such as Aero Glass will be instrumental in the first stages, it is more likely for the technology to be implemented at an airplane manufacturing level.

So, in the future, when you’re buying your next Boeing jet, make sure it comes equiped with AR.

Aviation safety

The Skies Made Safer with AR

Pilots, airport personnel, and stewards only make up a small portion of the aviation industry. And, while a properly-trained pilot is instrumental in a safe flight, the unseen world is just as important.

Let’s revisit our earlier car comparison.

When you purchase a car, you care about its safety ratings. When certain vehicles have massive recalls, it makes the national news. You watch to make sure your tires are in good order to ensure travel safety.

If you’re a super-awesome car owner, you also maintain the car with regular oil changes and checkups. When you do that, it’s likely with someone you trust — whether it’s a friend, respected mechanic or certified dealer. All of this is to ensure your travel is safe. And, you likely want it done at a reasonable price too.

Now, imagine trying to do all of that to an Airbus A380 (yeah, that thing holds 853 passengers).

Advancing AR Safety - large plane

When the plane is down for maintenance, the company looses money, so efficiency is key.

If the tires are bad, off-runway crashes can happen.

If anything else breaks, we hope it’s rather minor and only causes slight inconveniences for passengers. But, if poor maintenance is done, or just simple mistakes are made, the difference is clear. It’s no longer about whether you’re broken down on the side of a road with a smoking engine. Instead, 853 passengers and the crew plummet into the Atlantic.

To ensure that doesn’t happen, manufacturers, airlines, engineers, and mechanics work hard to follow every detail and snake out every possible problem before the plane takes to the runway. Giant teams of unthanked individuals make up these dedicated teams that often work overnight to ensure such safety.

But, what if we no longer needed to ask these unsung heroes to climb through claustrophobic spaces to seek out problems in labyrinths of cables and wires only decoded with complicated maps?

What if, instead, we gave them better tools to navigate the complex mechanical waters with smartglasses that overlaid pertinent information on the area they seek to examine or repair?

By doing so, we reduce human error — and, let’s be honest, every human in every field wants a failsafe.

We can then also provide better direct visuals as to whether or not something is wrong. An overlay can easily show whether or not the intended (original) matches up to the reality (current). This won’t necessarily find all problems, but it will…

  • give a great diagnostic starting point if a problem of some sort is known to be in the vicinity, and
  • alert people to potential future issues they may not have been on the mind (such as corrosion or cracks).

By giving those in the manufacturing or maintenance industries augmented reality tools, we are giving them the tools for efficiency and optimized productivity. And that, I think we can all agree, benefits everybody.

While we’re not quite there yet, we can easily see the trajectory of how things are evolving rapidly.

The MRO-Network (Maintenance, Repair & Operations), cited a study done by an electrical engineer at Boeing during which two groups of trainees were given the same task — one group with a PDF and one with AR assistance. The AR group completed the task 30% quicker and with 90% more accuracy.*

Within roughly the same timeline, Japan Airlines developed a relationship with Microsoft’s HoloLens to provide better training, knowledge, and visual learning to those who might be confined to desks.

And those two examples both date back more than a year (which, in the tech world, we know is generations long).

More recently, we see advancements where augmented reality is being used by workers (not trainees) to reduce errors and increase efficiency. GE, for example, adopted Google’s Enterprise Glass to allow workers to see instructions or references while actually working on a part while also helping to document the work done.

When it comes to augmented reality in the aviation sphere, the potential benefits and possibilities are nearly endless. Thankfully, we’re already past the point of speculation and entering the time where practical uses are being utilized to benefit works and provide added safety to all involved.

Scottie Gardonio
Scottie Gardonio
Scottie is an award-winning designer with many years of experience in creative markets. A passionate Augmented and Virtual Reality proponent, she has been a leader implementing AR/VR in the historical tourism industry of Minnesota. Through discuss...
Scottie is an award-winning designer with many years of experience in creative markets. A passionate Augmented and Virtual Reality proponent, she has been a leader implementing AR/VR in the historical tourism industry of Minnesota. Through discuss...