Major Problem Reviews in 6 Easy Pieces

If you are ITIL® certified then you remember Major Problem Reviews (MPR). You may still remember that Problem Management performs a MPR after resolving a 'major problem' to identify:

• What was done right
• What was done wrong
• What could be done better next time
• How to prevent the Problem from happening again

The benefits of MPR are many, including reduced staff workload and higher quality operations. However, aside from the above, the ITIL offers no specifics on how to perform a MPR.

Luckily, under the name After Action Review (AAR), the US Army has refined a functionally identical process. Peter Senge says the US Army AAR is "...arguably one of the most successful organizational learning methods yet devised..."

Following I show how to perform an ITIL MPR using tips and techniques from the Army AAR.

A Major Problem

The ITIL indicates that Problem Management (PM) should perform a MPR after the solving a 'major problem.' A major problem is any Problem where the severity or impact was such that management decides to review the entire series of activities. The scope of an MPR includes process, actions of staff as well as tools, and the environment.

After Action Reviews

The US Army Training Circular (TC) 25-20 'A Leader's Guide to After Action Reviews' provides good detail into exactly how to plan and deliver After Action Reviews. Of course, it needs some translating for the non-military. Donald Clark, retired Sergeant First Class (E-7) in the US Army worked as an instructor and then a training developer. Later, working at Starbucks in corporate IT, he defined AAR:

The AAR is a professional discussion that includes the participants and focuses directly on the tasks and goals. It is not a critique. In fact, it has several advantages over a critique:

• It does not judge success or failure.
• It attempts to discover why things happened.
• It focuses directly on the tasks and goals that were to be accomplished.
• It encourages employees to surface important lessons in the discussion.
• More employees participate so that more of the project or activity can be recalled and more lessons can be learned and shared.

This is important -- the MPR (and the AAR) are learning activities, not punitive or troubleshooting activities. The goal of the MPR is a discussion surrounding the major problem that allows all involved to learn what happened, why it happened, what went well, what needs improvement and what lessons can be learned from the experience. The spirit of the MPR is open and learning. It is not about problem fixing or allocating blame. Problem Management explicitly documents and shares the results with all who need to know.

Major Problem Reviews

The MPR collects, organizes, evaluates, and distributes data relating the objective of the MPR. Ideally, Problem Management documents the types of information to collect before the major problem as part of the process planning activity. Key data includes:

• Immediate action plans
• Forms used by Incident Management
• Activity logs, journals and tickets
• Verbal or written messages, such as call-backs and emails
• Function or position checklists
• Any other forms of documentation required

Problem Management defines what a major problem is, and how to respond to one. The planning of how to respond is outside the scope of this article, and specific to every organization. However, there are four key steps to effective Major Problem Reviews:

• Step 1. Planning (before the major problem)
• Step 2. Preparing (before the major problem)
• Step 3. Conducting (during and after the major problem)
• Step 4. Following up (after the major problem using MPR results)

The focus of the rest of this article is on Step 3 'Conducting' a Major Problem Review.

Conducting a Major Problem Review

The basic steps for conducting an MPR are as follows:

1. Assign responsibility for the review. Problem Management should appoint a person to assume responsibility for the review as soon as possible. ITIL says to begin the MPR after resolving the Problem. The Army says to begin the process as soon the leader knows the review is required -- this includes during the response phase of a major problem. Ideally, the person assigned will have a background in Problem Management and a familiarity with the processes, systems, tools, and staff involved.

It might not exactly follow on what ITIL says, but assigning a person to this duty during a major problem, staff resources permitting, delivers several key benefits:

• Emphasizes the importance of documentation
• Allows for early identification of possible system improvements and possible immediate improvements
• Compiles before too much time has elapsed and participants disband or forget what they have done
• Documents accurate timelines and activities versus trying to recall details later
• Expedites preparation review

2. Start Documenting. Documentation begins as soon as possible. While it will be tempting to skip the documentation during the major problem, it is critical to document that what actually happens. Benefits of this include:

• Enhanced operational decision-making
• Potential legal ramifications minimized
• Possible reimbursement/refund opportunities identified

3. Gather data. Aside from the documentation developed during response to a major problem (step #2), the MPR leader must collect additional information during or after the resolution of the problem. Exactly what to gather is a function of the MPR plan, but some important data gathering methods and opportunities include:

• Interviews and forms completed as staff complete key activities or at shift change during the major problem
• Informal (or formal) meetings shortly after a major problem
• Surveys to form the basis for team discussions teams
• Team meetings/workshops to review collected data

Some key points to consider regarding the collection, structure, and organization of the data and team meetings/workshops are:

• What is the purpose of data collection?
• Who is the audience for this data?
• Have all directly involved team members been interviewed?
• Will the team meeting/workshop allow identification of internal and external improvements?
• Does team meeting/workshop include other processes like Incident, Service Level, Availability, etc.?
• Does the format align with the review report format?
• Do the individuals and teams understand their assignments?

4. Initial Meeting. As soon as possible after the major problem closes, an initial meeting of all involved is important -- this includes customers and vendors. Remember, impartiality and unbiased review with the goal of organizational learning is the key here. This meeting (and the MPR process) is not to define blame, assign guilt, or to single out individual failures.

Before any group meetings collate all data and develop an initial timeline of events and actions. Provide this initial timeline to all attendees prior to any meetings. A good agenda might include:

• Establish meeting chairperson. In most cases, a neutral and respected facilitator not involved in the major problem is the best choice for the chair.
• Establishing the tone of the meeting. Having customers and third party vendors at the meeting is important, but can also spark emotions and inhibit frank discussions. It is critical that the tone be supportive and that the meeting does not degenerate into a "blame game." Strong management control over the meeting is a must. The MPR is a learning event and not critique or performance evaluation.
• Review the major problem. Construct a timeline or flow chart of what happened, identifying events, responses, and key decision points.
• Ask 'what was done right?' Seek first to identify what went as planned or correctly. Build on best practices and learn from mistakes. Use the "5 Whys Technique" for each point raised about something that went well. This helps get to the root cause of the reason, then follow-up by asking specific and repeatable advice, tips or techniques that others can use in the future. Note the emphasis first on positive elements of the MPR.
• Ask 'What could be done better next time?' Seek next to identify what the issues or problems were. Use the "5 Whys Technique" again for each point raised to get to the root cause, and follow-up for specific and repeatable advice. Note you are not simply asking 'what went wrong?' but rather 'what could have gone better?' This technique teaches from mistakes in a positive manner since the on improvement, not failure.
• Encourage participation. It is vital that everyone is satisfied that they have made their points and been heard before leaving the meeting. Anyone leaving without satisfactorily voicing their opinion may not contribute or support the ultimate findings and actions that come from the MPR. A good technique here is making all participants rate the meeting from 1 to 5, with 5 as most accurately presenting the findings. You should ask anyone offering 4 or lower what would make it a 5 for them, and pursue as time permits.

Prepare and deliver your initial meeting findings to all participants as soon as possible after the meeting.

5. Report Preparation. A four-step process provides a nice method to prepare the MPR report:

• Make a storyboard. It is vital to have an easily understood, accurate, and engaging report. Without these key bits organizational learning will suffer and little action may result. Include tips, tricks, techniques, and guidelines for the "next time." Background information about the teams or systems involved help put the findings into an easy to relate context.
• Compile the results of all data collection (forms, surveys, interviews, meetings and workshops) and sort the information according to the areas covered.
• Review, analyze, and sort documentation according to the areas covered as appropriate to the organization.
• Prepare report drafts and distribute them for review and approval to participating individuals, departments, agencies, advisory boards, etc.
• Prepare the final report and forward it to the responsible management, copying individuals and teams involved.

6. Share the learning. Be sure to distribute the report to anyone who else could benefit from it. For example, you may know about another team using similar tools, hardware, or software. Make the report available as a document, available to anyone who might benefit from it, perhaps the IT portal, library, or in the Configuration Management Database (CMDB).

Summary

Major Problem Reviews must focus on response. It is worth repeating is that MPRs are learning events, not performance review or critiques. It is therefore vital that they seek no blame. The quality of an MPR depends totally on the open and honest discussion of events by participants. If they know punishment awaits it is very unlikely that any value will come of the MPR. Management commitment and endorsement of the process is critical.

A Major Problem Review is a Knowledge Management System. It uncovers and shares knowledge. And it's not just for Problems -- call it an After Action Review (AAR) and use the same techniques to assess any training, project, or activity. MPR and AAR both allows employees and leaders to learn what happened and why. Examples include when introducing new products, performing upgrades, after a training session, a change of process or procedure, and any time a significant event occurs and you want to capture organizational learning.

Try using MPR's and AAR's. Use the AAR for non-problems to tune your process and get the organization familiar with the benefits. Then, when you do have a major problem, the entire team will know exactly how to respond.