This article looks at some of the common scenario-testing pitfalls and how to get the most value out of your testing programme.
Balancing breadth vs depth
Both breadth and depth are necessary for a scenario testing exercise to be successful. Balancing these against the time needed to run the scenario testing programme is a key challenge for Chief Actuaries and CROs.
Scenario breadth is important to ensure the scenarios appropriately reflect the wide range of risks to which the business is exposed.
A narrow range of scenarios could lead a board to a narrow understanding of risk and a consequent failure to plan for adverse events.
Scenario depth is important to ensure the effect on the business is adequately captured, including secondary effects and management responses. For example an insurer modelling the claims impact of a cloud computing outage on its cyber portfolio should also consider the extent to which the insurer itself might be affected operationally by the same outage.
Achieving balance between breadth and depth was a topic at LCP’s recent CRO roundtable event. One approach that stood out was to consider a wide range of scenarios at a high level to achieve the requisite breadth, but then group them according to the likely way they would affect the business. This helps narrow as many as 100 scenarios down to perhaps just 10-20 different categories. Each of those categories can then be considered in depth, eg to assess secondary impacts and management responses.
Creating a joined up approach
Scenario testing is commonly part of each of the reserving, capital validation, ORSA and exposure management processes, often with each department generating scenarios independently.
Considering the various important uses of scenario testing throughout the business (eg helping Boards with strategic planning and risk management), careful planning and a joined up approach can be very valuable to ensure good coverage and to avoid duplication of effort.
In my experience, an excellent way to achieve this is through top-down planning – ie ensuring there is a high degree of coordination between scenario setting departments, ideally guided by the risk function.
Top-down planning helps ensure more joined-up scenario coverage from the relatively common (ie more frequent than 1-in-10) through to the extreme (ie 1-in-200) because the teams generating the various scenarios set them in a more coordinated way. This is important because it gives the Board a good understanding of the range of possible outcomes.
Top down planning also helps deliver more consistent messaging to the Board, so they see a more joined up illustration of key risks in reports they receive, for example Actuarial Function reports and the ORSA.
Capturing the extremes
This article opened by posing a question – have you moderated or discarded scenarios because they were implausible, extreme, or because they wouldn’t be taken seriously? Doing so may be an example of a behavioural bias called “self censorship”.
A related concept is the idea of “conformity bias”, or group-think, where people set aside their own views in order to achieve a consensus.
These biases can affect every stage of the scenario testing process from junior actuaries setting scenarios through to the board’s review.
A business can make a big difference to the effect of these biases by adjusting the approach to setting stress and scenario tests.
Three suggestions to help with this are:
• Having representation from a range of different teams and disciplines when setting the scenarios. This can help break down people’s preconceptions. For example, the actuarial team might struggle to imagine making a multi-million operational error, but someone from claims might challenge that preconception.
• Avoiding giving people pre-conceptions through the way the meeting is run. For example, starting from a blank slate each time scenarios are reviewed, and encouraging more junior team members to speak up first can lead to a wider debate than the traditional “roll forward” of scenarios, led by the senior people.
• Focusing on generating and debating a wide range of possible impacts rather than building consensus. It’s important to avoid narrowing or closing down debate and it can be useful to encourage people to play devil’s advocate to keep generating further ideas.
Summing up
Scenario testing is, in my mind, a missed opportunity for businesses. The testing most firms do is insightful and well received by Boards, but it could still be better coordinated, provide better breadth and depth of coverage, and hence add even more value.
I would encourage insurers to view all scenario testing, irrespective of the department
|
