Better Safety Risk Management: Uncertainty & Decisions
A lot of us have been doing aerodrome safety risk management for a while now. In that time, I’ve seen and made a lot of mistakes and that means, hopefully, that I have learned a bit about risk management. So, I’d like to share some of my best lessons from being a regulator and practitioner of safety risk management over that time.
#1 - Risk Management is a Decision Making Tool
Before embarking on a risk management adventure, you need to ask the question, “what decision am I trying to make?” But I have seen many attempts at doing risk management where the operator has just picked up a “system” from somewhere else, plugged some details into it and now they have some “results”.
But either they don’t do anything with the results (because it was never about actually making a decision, just ticking a box) or they can’t use the results because they don’t know what decision they were trying to make in the first place.
Obviously, the decision will depend on who you are and what your role is. For most managers, it will likely be about where to assign resources or whether you have done enough to address the risk. For some aerodrome staff, the question might be a little bit more specific. It could be “which species of birds should I target?” or “will this obstacle cause an unacceptable reduction to safety?”
Once you understand this, you can then work out what hazards you should be identifying, how you might analyse the risk, what controls you may need to implement and so on.
#2 - Risk is Uncertainty
Everyone seems to have their own definition of risk. Sometimes, its not really a definition but a couple of examples of risk or the rephrasing of some problem with the cause being risk. I don’t really want to get into that whirlpool of a conversation but I do want to acknowledge that most people, including me, have a problem with understanding and articulating the probabilistic aspects of risk.
Often resulting in an argument about whether you “score” the severity of the most probable outcome or the probability of the worst case scenario?
But I do like one definition because it isn’t too rigid and can be quite powerful…
Risk: The effect of uncertainty on objectives (ISO 31000:2018)
That’s it - uncertainty. But people don’t like uncertainty and they try to use risk management as a pathway to its opposite.
People have seemed to latch on to one of the notes to this definition and define risk in terms of consequences (or impact or severity or … ) and likelihood (or probability or chance or … ). However, ISO 31000:2018 only says that risk is usually expressed in these terms.
I think the problem with these definitions is that there is actually a probability of a probability issue. If you look at the question above, the question of risk is about both “scores” in a way but I believe it is the extremes that drive action.
I tend to look only at the worst case scenario. Partly because I don’t think there is much an airport operator can do about the eventual severity of an incident/accident and partly because acknowledging the full extent of our exposure helps to focus on the problem.
#3 - The Right Tool for the Right Job
I don’t like PIGs or the risk matrix as you may call it. I think they have driven so much of our “results” focus to risk management and we often miss the decision making part. Aviation safety is a complex beast though and we do need methods and tools for analysing the risk picture around our decision making process.
One of the best examples of using a specific tool for risk management, I think, is in wildlife hazard management. There are a number of tools available to airport managers to help them decide on which species they need to take action. These tools have nothing to do with the rest of the airport’s operation.
So why don’t we have similar tools when dealing with the risk associated with airside driving or with FOD or with jet blast? Maybe because some of these issues have less perceived uncertainty about them - we know that FOD on a runway must be removed; we trust people (more than birds) not to go on the runway (not 100% though), etc..
Imagine that instead of just “scoring” the risk of airside drivers as a whole, when analysing the risk you looked into specific factors. Things like experience (likelihood), vehicle size (consequence), blind corners (likelihood), back of stand roads (a bit of likelihood and consequence), etc. The combinations of these factors might drive very specific decisions on controls. Such as, only experienced drivers get to drive bigger vehicles.
#4 - Controlling Uncertainty
There isn’t a simple equation here and, I’ll cut to the chase, I don’t believe that “residual risk” is the answer. If you have a good picture of the risk you are dealing with, you should be able to identify risk controls that will help you feel comfortable with the uncertainty.
Unfortunately, I have seen, all too often, risk registers that show a “score” in one column, a list of controls in the next and then a reduced “score”, usually colour coded to green, in a third.
I don’t blame people for doing this. This has been the accepted methodology for so long that few question it. Even when it’s results don’t make sense
But let’s think about what we are saying in these risk registers. We are saying that our analysis is so good and that our understanding of the impact of our identified risk controls is so accurate that we can calculate with greater certainty (less risk) what will happen in the future. Pretty powerful!
Honestly though, haven’t we just picked a residual score we are happy with? Haven’t we really just said that we think we are doing enough to justify the risk and we are happy to proceed, if we implement these controls?
By doing this, I think we often miss the important step of ALARP. We assume that it is embedded in the “residual risk” calculation but, depending on where you are, this question can be more specific to the decision on risk controls. ALARP is often about asking yourself what esle can we do and is it “reasonably practical?” If there is something else you can do and the answer was yes, you might find yourself legally obligated to do it.
#5 - But What if your Job is All about Controls?
Running an airport is quite interesting from an aviation safety risk point of view because nearly all of our aerodrome safety activities are risk controls in the much bigger aviation system.
Back in the early days of Safety Management Systems at Australian aerodromes, I saw quite a few risk registers as a list of aerodrome operator activities with negative adjectives next to them. Their risks were “ineffective serviceability inspection” or “poor training” or “failure to maintain aerodrome lighting”. There was quite a bit of support for this approach and in a very limited way, it’s not wrong.
But have found it better to mostly focus on aircraft safety and treat many of these activities like risk controls. This helps me do two things:
I can design the activity to address the risk it is meant to control.
I can then assess their performance against what I am trying to achieve.
I won’t go into this topic too much here as I have written about it before. Except to say that I have found this approach to be action orientated and much easier to implement and sustain.
Risk management isn’t just a tick the box exercise. It is supposed to develop understanding, solve problems and help you sleep at night.
Unfortunately, poor risk management can do the very opposite.
Image credits: Header: Taryn Elliot via Pexels, Helo: Alex Azabache via Pexels, Wreck: Francisco Echevarria via Pexels
