Big Data, Little Team - How You Benefit from the FAA’s Surface Safety Metric

By Nick DeLotell
Source: FAA Safety Briefing, Sept/Oct 2020

Big Data: “big data” — noun, extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions. (Oxford Dictionary)

It wasn’t long into his flying career when Wilbur Wright was quoted as saying, “In flying I have learned that carelessness and overconfidence are usually far more dangerous than deliberately accepted risks.”

Merely eleven years before Wilbur and his brother made their famous flight, Sir Arthur Conan Doyle released another of his Sherlock Holmes short stories, The Adventure of the Copper Beeches. In it, when he’s frustrated at the lack of evidence, Holmes is quoted as saying, “Data! Data! Data! I can’t make bricks without clay.”

Here’s a final nugget for you; nearly 50 years before the Wright brothers packed up their flying machine and headed to Kitty Hawk, the 1854 Rulebook of the New York and Erie Railroad stated, “The road must be run safe first, and fast afterward.”

Can you imagine an aviation system that embraces the fundamental concepts of these centuries-old quotes? For comparison, here are some key words and phrases from our core ethos at the FAA:

“Safety Risk Management”

“Data-Driven Risk-Based Decision Making”

“... safest, most efficient aerospace system in the world.”

It doesn’t take Sherlock Holmes to detect some parallels between the FAA’s 21st century ethos, and these 19th and 20th century concepts. Are you surprised to know that data analysis and safety risk management pre-date manned flight? While the value of data and the concepts of safety risk management are not novel, today’s tools and technology make the FAA more and more effective at managing risk through Big Data.

Big Data, Little Team

Right now, there is a small, passionate, and professional team of FAA experts dedicated to improving safety on the surface of our nation’s airports. To be clear on that, we’re talking about all towered airports from Guam to the Virgin Islands and everything in between. The team is small (20 people), but is also diverse, with each person bringing their own unique experience and perspectives as airline pilots, general aviation pilots, air traffic controllers, and data scientists. They’re known as the FAA Runway Safety Group and they’ve taken Big Data and safety risk management concepts to the next level with their new Surface Safety Metric (SSM).

The traditional runway incursion data analysis might look purely at rates (e.g., 25 runway incursions per million flight operations) or statistics (e.g., most pilot deviations are caused by general aviation (GA) pilots). While there’s certainly a benefit to knowing rates and statistics, the numbers don’t always tell the full story. That’s where the SSM is different. The SSM goes beyond traditional data analysis by establishing certain values and algorithms within the data. The SSM also looks broadly at more data sources than ever before, such as National Transportation Safety Board (NTSB) data and data from the Aviation System Information Analysis and Sharing (ASIAS) system.

The results are impressive; the SSM has been able to objectively quantify risk. We’re not just looking at rates and statistics anymore. Now we’re able to see risk, measure it, point at it, and fix it, even in instances where no incident occurred or where, technically, no rules were broken. A key takeaway is that despite record air traffic volume (over 53 million flight operations in 2019) and a relatively steady rate of runway incursions, we are able to show that risk is continually trending down on the surface of our nation’s airports. Said another way, our airports are more and more safe.

So, how are we using the SSM? It’s really sensitive so single, “high risk” events are easy to identify and target. Events that involve injuries or fatalities on runways are examples. Alternatively, we can filter the SSM data to see trends. We look for things like individual “low risk” events but with a multitude of common factors that indicate systemic risk across the country. A good example of that is the identification of Wrong Surface Events (wrong runway or taxiway approaches, landings, or departures) as a top risk to GA pilots.

A key component to the SSM’s success has been the FAA Compliance Program. To find and fix problems, we (you and me) have to build an open and transparent exchange of information and data. If you inadvertently make a mistake, the FAA doesn’t want you to hide it because of a fear of being punished. If there is a problem, whether human or mechanical, we all need to learn from it, and we all need to make the changes necessary to prevent it from happening again. An open and transparent exchange of information requires cooperation and trust. To achieve that, we all have to understand the difference between accountability (accepting responsibility and looking forward) and blame (focusing on punishment for what’s already in the past). The Compliance Program is a critical part of the SSM because it recognizes the value of accountability, and it provides an avenue for exchange of information and data.

How You Benefit

Just like Sherlock Holmes, the Runway Safety Group collects the data, analyzes the data with the SSM, and finds the culprit. Once a culprit is identified, the group works collaboratively with aviation industry partners and other FAA offices to develop comprehensive plans either to remove hazards or to manage risks. That’s ultimately the benefit — a safer and smarter aviation system for you.

We hope certain benefits speak for themselves. The FAA publishes products like the Runway Safety Pilot Simulator, From the Flight Deck Videos, and Airport Diagram Hot Spots, to name just a few. You might see other results in the form of Advisory Circulars, InFOs or SAFOs, or changes to the various FAA handbooks and Airman Certification Standards (ACS). The FAA is also making enormous investments in predictive technologies that provide better alerts to Air Traffic Controllers, and huge airport infrastructure improvements through the Runway Incursion Mitigation (RIM) program.

So what’s on the horizon? Wrong Surface Events are still occurring at rates higher than they should, particularly within the GA community. Runway excursions by business jet operators are also a subject the Runway Safety Group continues to evaluate. Wherever the SSM takes them and whatever the solutions look like, rest assured that the Runway Safety Group is a little team that shares a big interest in keeping you safe.

Cleared for Takeoff

The lessons and concepts from the 19th and 20th centuries hold as true today as ever. You can apply them and contrib-ute to the safest and most efficient aerospace system in the world by remembering three simple things regardless of the type of aircraft you’re strapping into:

1. Wilbur said it best. Don’t be careless or overconfident just because you’re on the ground.

2. Safety is the top priority, and everything else comes later. Treat the surface just like the sky; aviate by taxiing slower, navigate by reference to an airport diagram, and communicate with ATC when you need time, clarification, or a little more assistance.

3. We’re in this together. Let’s all be accountable for our mistakes and not play the blame game. By improving our reporting culture, we’ll keep reducing risk.

Blue skies and happy landings! Taxi safely, my friends.

Learn More

From the Flight Deck - https://www.faa.gov/airports/runway_safety/videos/

Runway Safety Pilot Simulator - http://faarunwaysafetysimulator.com

FAA RIM Program - https://www.faa.gov/airports/special_programs/rim/

Nick DeLotell is an aviation safety inspector in the FAA’s Flight Standards Service in collaboration with the Runway Safety Group. He holds an airline transport pilot certificate, flight and ground instructor certificates, and is a remote pilot.