Risk-based Low Visibility Operations Standards Review
I may have mentioned my recent trip to Jakarta a couple of times already and this may be the last post about it but its the one I'm most excited about. The quick re-hash is that I went to Indonesia as part of Australia's Indonesian Transport Safety Assistance Package (ITSAP) to conduct a workshop on safety management principles for future members of their State Safety Program's Safety Action Groups - specifically airports and air navigation directorate members. The workshop focussed on acceptable levels of safety/safety objectives and risk management. The final two days were spent on a practical exercise which was meant to have the participants role-play as operators carrying out safety management tasks. But things rarely work out as planned and I changed the scenario the day before to something a bit more aligned with the actual role of the SAGs.
The scenario now involved the participants being divided into their operational groups - airports & air navigation - as they will be in their SAGs. Each group was set the task to review low visibility operational standards and practices. Why? Well, the preliminary report from a recent accident involving a Meparti MA-60 on approach to Kaimana recommended to the DGCA such a review. I knew I had hit the nail on the head when the workshop convenor informed me that the results of this "exercise" were going to form the start of the actual review (yes, my head swelled on that note).
Anyway, below is a run-down on the work my group (airports) completed over those two days and some subsequent thoughts I've had over the week since I got home.
Approach
In line with the workshop material, we took a risk-based approach. We decided against a straight review of the standards (the Indonesian Manual of Standards (MOS) 139) against other standards (Annex 14 etc.) as that would not necessarily cover the things that make operating in Indonesia different. Overall, we went with the ISO 31000 framework - consultation, establish context, risk assessment, risk treatment & monitoring.
Establishing the Context
The first step was to get our heads out of accident investigation mode and to take a wider view of the standards as they apply across the country. But that is a lot of ground to cover, so we needed to break the larger context down into smaller contexts. At first we went with runway approach categories - precision, non-precision and non-instrument (visual). In considering these three options, it was felt that the non-precision category was the primary concern given the minimal facilities which exist in such cases and because I was told that the trigger-accident involved such a runway.
From here we divided the context up yet again into four operational phases - approach, landing, taxi & take-off. The philosophy was that operational phases would be distinct in their events and outcomes and in the risk treatments/defences used. The group decided to tackle the landing phase first.
Risk Assessment & Treatment
Risk assessment is actually three-steps-in-one (identification, analysis & evaluation) and when actually carried out, goes around and around with a couple of goes at treatment until you are happy with the resultant or residual level of risk.
To kick off the risk identification, the group got into some brain-storming (BTW, a word which doesn't translate into Bahasa Indonesia very well!). We started well but at some point we needed a bit of structure to help us make sure we were covering the whole issue. I introduced the idea of using the SHEL model to help. It's an idea I had some time ago and hadn't had the chance to use it in a practical setting but it worked really well. As with the context, the issue is too big to just attach head-on. We needed to break it down some how. SHEL is pretty good for that and soon we had a four lists of things under the headings of Software, Hardware, Environment and Liveware.
Now it was time to try and make some sense of the mess we had just made. Somewhat organically, we moved toward the bow-tie approach (or for Indonesians - the butterfly-tie approach). We established a hard/unstable landing as our "top event" which resulted from our primary environmental low-visibility hazards of fog, rain and smoke. Our four outcomes were runway excursion, on-runway incident, passenger trauma (physical or psychological) and facilities damage. Then we plugged in our defences, our escalation factors, escalation defences and even some escalation-escalation factors and their defences. The result after half a day or so was this:
Risk Analysis - On-the-Day
In thinking about this analysis over the week after the workshop, I would move the re-scheduling recommendation out of this context and into a new "planning" context. Trying to stay true to the identified context was very difficult as the group and even I would suffer from "context creep" (thats what I call it anyway). As an example, in the question session at the end of the workshop I was asked why signage didn't appear in our risk assessment - I started talking about how we must of missed that one probably due to the time constraints on us and babbled on for a couple of minutes before one of my colleagues rightly pointed out that signage was outside the context of landing (except maybe distance to run signs) and we would have addressed it in the taxiing context. Anyway, my slight change to the final diagram would look like this:
My Changes
The next challenge was to turn this diagram into something more readable. We needed to take the above analysis another step and get it into a format that we could use to explain the situation and help us formulate a plan of action. Tables tend to be the preferred format and we discussed this and few other issues for a good couple of hours. The following issues were what we discussed:
Focus - Initially the table started out as a hazard register but this soon became unmanageable. The information became repetitive because many of the defences and higher-order hazards related to all three of the primary hazards. Two other options were identified and we went with safety issues over defences as our table's focus. By focus, I essentially mean the thing to which each row was dedicated - therefore, each row discussed some overall safety issue we had identified. This may be the identification of a new defence, a single higher-order hazard and its proposed new defence or multiple hazards combining and a range of defences put in place to address them.
Risk - This was a hard one. Again, initially we went down a pretty traditional road of deferring to a risk matrix but I, for one, wasn't convinced that we could apply a severity-x-likelihood score to our tabulated results. So on the day we went with a high/medium/low scoring system based on no real definitions. In the attachment I've changed this into an unacceptable/tolerable/acceptable system since that's what I think we were trying to say in this table. All we were saying at this stage was that in our assessment the safety issue we had identified was either acceptable as is and any further action was nice to do but not essential, tolerable meaning we should do something more or unacceptable and requiring attention and further mitigation. In the long run, further analysis and some strong assessment methodologies would be required.
Safety Goals - This was a big part of the workshop overall - safety indicators and safety performance indicators. The ones set here are fairly arbitrary and will require refinement, either through further analysis or experience. It was important for us to remember that these goals really only serve as a management guide or even just a trigger to consider the actual performance of the risk mitigation strategies described. Regardless of whether the system performs better or worse than the indicators, we will still need to examine system performance at regular intervals and adjust the strategies as required.
Conclusion
This process was just the first step. There are three more contexts to examine just for airports alone. Processes like these should be running in tandem for ATC and flying operations as well. Consultation, refinement, expansion, refinement and more consultation will be required. Its definitely not a two-day task for ten people in a hotel conference room! But it was a lot of fun and I learnt a lot in what was a fairly practical and technical exercise. I wish my Indonesian colleagues all the best in their future work and I hope to catch up with them again - either in Indonesia or in Australia or elsewhere. Terima Kasih.
Image Credit - (cc) Bill Abbott
