By Fearghas MacGregor, Principal and Head of GI Capital Modelling & Strategy, Barnett Waddingham
Various model uses were acknowledged to encourage validation for more than just validation’s sake. By aligning the validation to business needs, our experts saw increased stakeholder buy-in. The attendees felt that the more widely a model was used, the greater the challenge and oversight it received. For example, when models were used in reinsurance purchasing, there was greater interaction with parts of the business that are not usually included in modelling committees.
Evolving test criteria for an effective challenge
Solvency II was introduced almost a decade ago and capital models are now embedded in business operations, and so our discussion touched on the need for test criteria to evolve to ensure that the validation challenge remains effective. For instance, test thresholds may have been initially broad by necessity and our experts questioned whether subsequent model improvements should lead to a narrowing of these test criteria. It was noted that while a sea of green test results may look good, it could be taken as a sign of lack of genuine challenge.
Balancing control and independence
In cases where there may be a lack of validation independence, our experts discussed the resources available to ensure all model areas are validated appropriately. The group agreed that small sections of the validation can be outsourced to others – whether an internal team like audit, or externally to consultants. This would allow for the risk team to maintain control of the validation while meeting all independence requirements. Reviews of the independence of the validation were considered important, and these reviews could be carried out periodically, again either internally or externally.
Is the current validation cycle still appropriate?
As the insurance market enters a softer part of the underwriting cycle, our panel considered how best to validate the evolving parameters of internal models.
Backtesting in a changing market
Backtesting is an important validation tool, but can these tests provide robust challenge and assurance in a changing market? Discussions revolved around whether the last five years of results are suitable to backtest models parameterised for the 2026 underwriting year. Too great a reliance on historic hard-market results could leave validators exposed to model and validation drift.
Avoiding validation fatigue
Another challenge is resource longevity. Some risk team members have been validating the same model for five or more years - how to ensure that such resource could still provide adequate challenge was debated. It was acknowledged that there was a risk of complacency and lack of external view. Proactive use of a three-year plan was seen as a way to mitigate this, and descoping some model areas for a year to two would allow for distance between reviews. This separation then facilitates a ‘fresh’ challenge each time. It is important, however, to balance the use of a three-year plan with the ever-evolving external environment and regulatory focus areas. Targeted use of external validation resource and in-team rotation were also suggested to help avoid ‘validation-capture’.
Strategic allocation of resources
How best to allocate scarce second line resource was viewed as a constant balancing act. There may be many tasks that are ‘nice to haves’ but there simply are not the resources to carry them out. Our experts put forward several options to manage the validation workloads:
reducing the strain of handle-turning tests through automation;
focussing more on top-down, value-add testing; and
reviewing test bank ahead of the validation season, with an eye to removing stale tests.
Internal capital models are costly, as is the associated onerous validation and compliance. Consideration should be given to the allocation of resource across all firm models – such as for pricing, reinsurance, and Own Risk and Solvency Assessment (ORSA). By working across such a variety of models, second line resource will be upskilled while continuing to add value across the business. Our panel recognised the need to seek experience beyond insurance, for example in banking. Bringing in new, diverse skills was hoped to provide a needed shake up the existing processes. Further links could be made by introducing second line members into first line meetings. Additional connections could extend to underwriters, for example, to explain how their return on equity metrics are derived.
Success through proactive communication
More broadly, maximising dialogue with the regulators was viewed as a key part of a successful validation. Lloyd’s especially like as much engagement as they can get, covering topics from the pre-warning of Major Model Changes (MMCs) to taking them along for the journey of a Lloyd’s Capital Returns (LCR) submission.
How is validation evolving under Solvency UK?
Solvency UK (SUK) has officially arrived, bringing changes to the profit and loss attribution (P&LA) exercise and the divergence between Lloyd’s and the PRA on the subject of analyses of change.
P&L testing has formed a key pillar validation for a number of years now. However, the requirement to perform one has been removed under SUK. Some in our panel felt that the test had value left to offer and intended to continue performing it, but now they no longer need to conform to the rigidity of Solvency II reporting, they could instead focus on the aspects of this test that would be most insightful to senior management.
An Analysis of Change (AoC) report is a formal requirement for non-Lloyd’s firms. However, Lloyd’s is focussing less on a standalone AoC report and instead has mandated detailed commentary in the existing LCR form 600, stating in Lloyd’s Capital Guidance (April 2025) ‘comments … should be sufficient on their own to explain the change in observed metrics. However, … it may be necessary to … refer to the accompanying AoC document for further detail.’ The sentiment was that in most cases, regardless of the changes to form 600, boards will want a full AoC document that can stand alone. Further, the attendees agreed that the AoC has additional value as a tool, as it enables the validators to challenge the first line and ensure that there is a good understanding of the key drivers of movements between submissions.
Key takeaways on shaping the future of validation
The roundtable provided a valuable platform for industry experts to discuss challenges and share insights on risk and validation. Key recommendations included:
reviewing tests and their criteria as models mature;
targeting validation resource across all of a firm’s models;
engaging stakeholders and expertise from all corners of the firm during a validation exercise; and
maintaining independence through deep-dive cycles and external resource.
As the industry continues to evolve, addressing these issues will be crucial for a smoother and more collaborative validation process in the future.
|
