Articles - A human in the loop – but doing what?


Anyone who has been asked to "just have a quick look" at someone else's spreadsheet knows what that can involve. Formulae to trace; assumptions to challenge; plausible-looking results that don't stand scrutiny. Now multiply that challenge to complex actuarial models produced by AI in just a few minutes. I'm optimistic about the opportunities AI brings. It is already transforming actuarial work, from identifying patterns in large datasets to accelerating model development and analysing unstructured information.

By Tom Durkin, Head of Insurance Consulting at LCP
 
But I'm not reassured when someone says, "Don't worry, there will be a human in the loop." AI makes mistakes, and the speed and volume of its output can make them difficult to identify. Review and challenge have always been central to actuarial work. But if we can now build models faster than we can meaningfully review them, how do we ensure human oversight remains effective?
 
Three actions can help.
 
1. Match oversight to the risk
We need to be deliberate about where we use AI, because the appropriate level of human oversight depends on what we're asking it to do. Using AI to draft an internal report is very different from relying on it to develop a pricing model used in live business decisions. The consequences of an error, and the oversight required, may also therefore be very different.
 
Key questions include: How reliable does the output need to be? What happens if it's wrong? How easily could we detect an error? And how widely will the AI be used?
 
These answers should shape the controls and review processes. A generic checklist will only get us so far, and the nature and extent of human review should be proportionate to the risks involved.
 
2. Make human review meaningful
If AI can generate models in minutes that take days to review properly, the volume of work can quickly become unmanageable. We need to rethink what reviewers are being asked to do. Imagine AI creating a new Python model for each of a range of actuarial tasks. Someone still needs to establish whether the methods, implementation and results can be trusted. Repeating that exercise for every new model creates a substantial review burden.
 
A better approach may be to use AI to build reusable actuarial models and components. Invest in reviewing and testing those building blocks, then reuse them within defined boundaries. Their application and any changes still need checking, but we avoid repeatedly validating entirely new models.
 
Another opportunity is to use AI to support individual actuarial judgments. In reserving, for example, a system could highlight an unusual development pattern, present alternative assumptions and show the impact of each. The actuary can then focus on whether the judgment makes sense, rather than reviewing an entire model. They must, however, be free to challenge the options presented, not simply choose between them.
 
What reviewers see matters too. Too much detail obscures the important issues; too little asks them to endorse a black box. They need the right information, tools and training, together with the time and authority to disagree. Otherwise, the human in the loop risks becoming a rubber stamp.
 
3. Maintain the ability to challenge
Effective oversight must also be maintained over time. When AI repeatedly produces plausible results, reviewers may become less inclined to question them. We should therefore test not just the AI, but whether the review process itself works, perhaps by checking whether reviewers spot known errors in test cases.
 
There is a longer-term challenge too. If the next generation of actuaries spends more time reviewing AI-generated models than building models themselves, will they develop the experience needed to challenge them confidently? Training and professional development will need to adapt.
 
So what should the human be doing?
Simply having a human in the loop isn't enough. Asking them to check everything risks an impossible workload; asking them merely to approve AI recommendations risks a rubber stamp.
 
The opportunity is to design systems that let actuaries focus their expertise where it matters most: challenging assumptions, exercising judgment and understanding the consequences of decisions. Getting that right will be just as important as improving the AI itself.

 

Back to Index


Similar News to this Story

A human in the loop – but doing what?
Anyone who has been asked to "just have a quick look" at someone else's spreadsheet knows what that can involve. Formulae to trace; assumptions t
David Fairs on innovation and choice in DB Endgames
LCP Partner Steve Hodder talks to David Fairs, Partner at LCP and former Executive Director at The Pensions Regulator, about the changing DB pensions
Guided Retirement and Behavioural Science
What can behavioural science tell us about designing retirement options and defaults? Stew Patterson considers the implications with insights from beh

Site Search

Exact   Any  

Latest Actuarial Jobs

Actuarial Login

Email
Password
 Jobseeker    Client
Reminder Logon

APA Sponsors

Actuarial Jobs & News Feeds

Jobs RSS News RSS

WikiActuary

Be the first to contribute to our definitive actuarial reference forum. Built by actuaries for actuaries.