By Julien Masselot, Principal and Head of Capital and Risk and Fearghas MacGregor, Associate and Senior Consulting Actuary at Barnett Waddingham
"While validation is an integral part of the risk and regulatory framework around internal models, the approach needs to evolve to cope with the fast changing (internal and external) environment. This will then mean valuable feedback can be provided to both first line and the Board."
Certainly from our experience, firms are beginning to ask the question as to whether the validation process is still fit for purpose.
Stakeholder engagement: quantitative versus qualitative outputs
A key discussion point was about striking the right balance of quantitative information in a validation exercise. Our roundtable attendees noted that there has been an increasing reliance on point estimates over the past ten years.
This has two effects:
It leads to anchoring to prior results but does not equip the board with the right information for effective challenge.
It increases the danger of distracting senior management from issues that are not material to their organisation and risk profile, as a point estimate does not convey information about its accuracy.
To address these issues, attendees shared strategies that have worked for them including the use of ranges when communicating results to senior management.
These provide a more realistic picture of the risks faced by the business by allowing for some uncertainty. By providing ranges rather than point estimates, the group emphasised that it gives context for stakeholders to understand and assess the results. It also allows them to temper their response to changes. For example, a change to business plans leading to changes in the solvency capital requirement (SCR) can quickly be assessed for materiality by measuring against the model output ranges used in validation.
Stakeholder interaction was also discussed as an area which needed improvement. One attendee wished for a more effective validation framework, which would require consistent and regular training for non-executive directors and other board members.
Boards have matured over time - the current members are very different to those from ten years ago and have different areas of experience and expertise. Meanwhile, the models and procedures will also have developed, so there will naturally be a need for regular realignment and training.
Improving validation quality through collaboration
The importance of building partnerships with other teams was highlighted, whether working with first line capital or with the finance team.
By creating a collaborative environment rather than operating in silos, the quality of the validation outputs can be improved, and a feeling of partnership can be developed. For example, it was shared that successful collaboration with the first line enabled the model development plan to prioritise areas of focus for future validation.
Participants mentioned that they were actively pushing back against validation as a tick box exercise. Instead, it should be viewed as a powerful tool to help the business make better decisions. But it was noted that validation must still satisfy the original requirements: providing comfort to the board, the regulator, and other stakeholders.
One proposal to unlock more benefits from the validation cycle was to work with the first line team more efficiently. This can be done by transferring some of the initial checks to first line, who are best placed to address these, freeing up validation teams to consider wider areas of focus from an earlier stage.
Key focus areas
One participant suggested leveraging historical validation to find and focus on areas that will “move the dial” – whether materially impacting the total capital requirement, individual risk areas, or any other important business metrics.
We carried out a short survey during the roundtable, asking 15 attendees to select the key themes of focus for the upcoming validation cycle. The results are shown in the graph below:
The areas identified were:
Underwriting – both profitability and prospective loss ratios. This was expected to be an area of particular regulatory focus, given the ‘Dear CEO’ letter from the Prudential Regulation Authority (PRA).
Geopolitical risks – covering both the areas from the 2023 exercise, as well as any additional areas as the current political climate develops.
Reserving - especially back year deterioration and how this contrasts with prior reliable reserve releases.
Efficiencies within the validation process
A repeated concern raised by participants was the constraints (both time and resources) of validation.
Regulatory timelines are getting tighter while the validation work remains increasingly dependent on the work of other teams. Therefore, material changes to their work can have significant knock-on impacts to validation which has implications on the output quality. For instance, triggering an unexpected major model change close to the Lloyd’s Capital Returns (LCR) submission will add a lot of pressure to the validation team.
The roundtable then moved on to discussing the challenge of creating efficiencies within validation, specifically automated validation. We have our own version of such a tool called validateR. We noted that a small number of participants had tried down this road but were often distracted by other more pressing concerns.
Automation has many benefits, including allowing the validation team to spend more time providing valuable feedback, focussing on key areas that need more attention and providing the opportunity to increase interaction with other teams, building quicker feedback loops.
Next steps
Our roundtable provided a valuable platform for insurance experts to discuss challenges and opportunities within validation. It was useful to discuss the market observations we have seen, but also to learn more about specific challenges that insurers face and share practices on how to overcome those challenges.
"As the industry continues to evolve, internal model validation remains a key component of the second line of defence of insurers, but does need to be a value adding process rather than just a regulatory box ticking exercise."
|
