Better Emergency Exercises: Purpose, Planning & Failing Willingly
It takes a great deal of work to plan and execute an airport emergency exercise. They require hundreds of people-hours to organise with meetings, discussions, working groups, control plans and reviews. Approvals go up and down multiple reporting lines and resources (people, equipment and money) are committed.
And yet, I think the hardest part of this process, is that it all leads to each response organisation, and especially the aerodrome operator, opening themselves up to critique. Each exercise should be designed to push the system and to result in some kind of failure. And after all that, we ask others to tell us what we did wrong and while we know that it is better to fail in an exercise, it is never easy to openly discuss your mistakes.
With all this stress and pressure, it is not surprising that in my time overseeing and participating in exercises that I’ve seen (and perpetuated) a skewed understanding to the airport emergency exercise. To combat this, I offer the following “tips” to better exercises.
#1 - It’s All About the Plan & Not the People
A full-scale airport emergency exercise is not a training exercise. It is not designed to teach airport and other team members about emergency response or to ensure that those involved are competent in the roles they may be assigned.
Exercises are typically undertaken every two years and things like staff turnover, promotion and leave can greatly affect the emergency response team within this time frame. Surely, one cannot reasonably expect to wait for an exercise, up to two or three years, to ensure new team members are brought up to speed.
Instead, emergency exercises should be designed to test the plan. The Aerodrome Emergency Plan (AEP) will persist through personnel changes and it is this that we want to ensure is adequate and, where possible, improve.
Annex 14 is clear about this under paragraph 9.1.12 saying that the AEP itself will make arrangements for the “periodic testing of the adequacy of the plan and for reviewing the results in order to improve its effectiveness” and in paragraph 9.1.13 with the statement that “the plan shall be tested…”.
Further notes and guidance material reinforce the standards.
#2 - But Don’t Ignore the People; Plan for Development
While using full-scale exercises for training isn’t advised, using exercises as part of staff development is a great idea. Developing talented staff is a key component of good management. It is like investing in your future with that team member with both performance returns and increased retention.
As unusual as this sounds, subjecting someone to an emergency exercise can be used as an incentive and a reward. Imagine you have a senior member of your airside operations team - perhaps they have been involved in a previous exercise or perhaps they are just a good learner and a hard worker. An upcoming emergency exercise could be used to encourage them to extend their knowledge and skills by telling them that they will be in the Forward Command Post during the exercise and maybe even running it.
Such a plan could see that team member thoroughly engaged in their work for months as they prepare for the exercise. That motivation may carry through for many months after the exercise as well. Thanks to the personal sense of achievement they got from the exercise and the faith in them you showed by offering this opportunity.
These opportunities don’t always need to be taking part in the exercise. Other options include:
shadowing key position holders,
being part of the planning team,
participating as a casualty or family member,
being part of exercise control, or
being on the critique team.
Remember that development is not just about training. Some guidance on development suggests that only 10% should be education-based. Instead, we should focus on experience and exposure for our developing team members. If you are interested in this topic, I wrote a whole article on development here.
#3 - Pick What is Important & Set Some Parameters
An Aerodrome Emergency Plan is designed to address a lot of “what if” scenarios. It is impossible to test all them in a single exercise. So, we have to make some decisions and we set some parameters to assist in planning and evaluating the exercise.
ICAO talks about objectives and many exercises I have been involved in use this language but FAA guidance material outlines the entire exercise development process in a very useful format. I’ll summarise it here but I suggest a look at this section of AC150/5200-31C prior to preparing for your next exercise.
The FAA outlines the following seven step development process.
Needs Assessment - Review your obligations, previous exercises, hazards and identified difficulties.
Define the Scope - Consider the activities/procedures to be tested, the agencies involved, level of realism, target hazards, priorities and locations.
Purpose - Develop a succinct statement of the purpose of the exercise, upon which, all further actions will be aligned.
Goals & Objectives - Further refine the needs assessment and purpose to arrive at the goals and objectives of the exercise.
Scenario - Using the steps above, set the scene for the exercise in terms of a scenario or narrative. Consider complexity and complicating factors in this step of the development process.
Injects - The FAA refers to this step as message/problem statements but I am more familiar with the concept of injects. These are the sequence of events that the exercise control will inject into the scenario to guide the exercise toward the objectives and purpose..
Evaluation - Given that the overall purpose of any exercise is to test the plan, we need to establish the evaluation procedure in the preparation phase of the exercise. This will ensure that the standards against which the exercise will be critiqued align with the purpose and objectives.
#4 - Get to Know the Different Types of Exercises
Let’s return to ICAO for this discussion but I do acknowledge that the terminology might vary between jurisdictions. The Airport Services Manual (Part 7) describes three types of emergency exercises:
Table-top - A bunch of people in a meeting room, around a large map or something similar, working through the plan in response to a scenario developed beforehand. This type of exercise is good for preparing for a full-scale exercise by confirming procedures, communication lines and jurisdictions, etc.
Partial - A practical exercise looking at a limited part of the emergency plan. Limited in activities, agencies involved or complexity. This type of exercise is often used for training and currency requirements (but it needs to be practised regularly to achieve this) or to confirm that deficiencies noted in the last full-scale exercise have been addressed.
Full-Scale - Flashing lights, radios screaming, heat, sweat, stress and all that good stuff!
The language of Annex 14 also suggests that a slightly different type exercise exists under the name of “modular test”. There is little guidance provided on this idea but we can look at basic definitions and the text of the standard to derive what I think is a reasonable picture of this type of exercise.
From the dictionary definition, modular speaks of something being complete and independent in its own right but can be combined with other modules to construct something larger and more complex. Partial is different in that it need not be complete in its own right.
If the modular test option is available under your state regulations, you must plan each exercise as a complete and independent set of activities under the plan. It must be able to go together with other modular tests and to be appropriately spread out through the three-year period permitted by the standard. The final full-scale exercise must also be a conclusion to these tests.
#5 - The Feedback Loop
All this effort is for nothing without capturing what went wrong and what needs fixing or improving. To achieve this you need to plan and dedicate resources. This can be easily forgotten or left too late. It is also sometimes forgotten completely for partial and table-top exercises.
To do it right, you will need people not involved in the exercise. You will need a variety of people with some familiar with the activities being tested and some with fresh eyes. Some people will need to be assigned to specific areas and others will need to move around.
Your evaluators will need some guidance on what to assess and critique. Standardised forms help with data analysis later but they might restrict the richness of potential feedback. A balance will need to be struck.
The data and information you receive from your evaluation team will then need to be collated and analysed to arrive at a list of deficiencies or areas for improvement. This should be packaged into a report for the emergency planning team to use in the next step - because the report is not the end.
The emergency planning team needs to take this report and make changes to the aerodrome emergency plan to address the deficiencies or issues.
And then…
You need to test the plan again!
images credit: (cc) Brussels Airport (Flickr Account)
