By Tom Durkin, Head of Insurance Consulting at LCP
But I'm not reassured when someone says, "Don't worry, there will be a human in the loop." AI makes mistakes, and the speed and volume of its output can make them difficult to identify. Review and challenge have always been central to actuarial work. But if we can now build models faster than we can meaningfully review them, how do we ensure human oversight remains effective?
Three actions can help.
1. Match oversight to the risk
We need to be deliberate about where we use AI, because the appropriate level of human oversight depends on what we're asking it to do. Using AI to draft an internal report is very different from relying on it to develop a pricing model used in live business decisions. The consequences of an error, and the oversight required, may also therefore be very different.
Key questions include: How reliable does the output need to be? What happens if it's wrong? How easily could we detect an error? And how widely will the AI be used?
These answers should shape the controls and review processes. A generic checklist will only get us so far, and the nature and extent of human review should be proportionate to the risks involved.
2. Make human review meaningful
If AI can generate models in minutes that take days to review properly, the volume of work can quickly become unmanageable. We need to rethink what reviewers are being asked to do. Imagine AI creating a new Python model for each of a range of actuarial tasks. Someone still needs to establish whether the methods, implementation and results can be trusted. Repeating that exercise for every new model creates a substantial review burden.
A better approach may be to use AI to build reusable actuarial models and components. Invest in reviewing and testing those building blocks, then reuse them within defined boundaries. Their application and any changes still need checking, but we avoid repeatedly validating entirely new models.
Another opportunity is to use AI to support individual actuarial judgments. In reserving, for example, a system could highlight an unusual development pattern, present alternative assumptions and show the impact of each. The actuary can then focus on whether the judgment makes sense, rather than reviewing an entire model. They must, however, be free to challenge the options presented, not simply choose between them.
What reviewers see matters too. Too much detail obscures the important issues; too little asks them to endorse a black box. They need the right information, tools and training, together with the time and authority to disagree. Otherwise, the human in the loop risks becoming a rubber stamp.
3. Maintain the ability to challenge
Effective oversight must also be maintained over time. When AI repeatedly produces plausible results, reviewers may become less inclined to question them. We should therefore test not just the AI, but whether the review process itself works, perhaps by checking whether reviewers spot known errors in test cases.
There is a longer-term challenge too. If the next generation of actuaries spends more time reviewing AI-generated models than building models themselves, will they develop the experience needed to challenge them confidently? Training and professional development will need to adapt.
So what should the human be doing?
Simply having a human in the loop isn't enough. Asking them to check everything risks an impossible workload; asking them merely to approve AI recommendations risks a rubber stamp.
The opportunity is to design systems that let actuaries focus their expertise where it matters most: challenging assumptions, exercising judgment and understanding the consequences of decisions. Getting that right will be just as important as improving the AI itself.
|