Hello everybody and welcome to another edition of C&F Talks. Today I have with me, Paul Johnson, who's the CEO of Davies and Paul and his team are experts in the whole area of AI and agentic AI.

The Davies Group is a global professional services and technology firm specialising insurance financial services and other highly regulated industries and some of Paul's colleagues are going to be speaking at our upcoming agentic AI for leaders summit which is being held in London on the 17th of September. Paul, welcome.

Hello and thank you for having me.

Maurice

Great to have you with us. In the various sections of the programme in which Davies is involved there's this notion of the human premium and obviously when AI can generate answers in seconds, you know, what is the role of humans in practical executive terms? Is it a judgment under uncertainty, ethical accountability, contextual understanding, the ability to build trust for instance or something else and how should a leadership team recognise when it is genuinely present? 

I think that's some great questions and something that everybody's wrestling with at the moment and of course I must preface all my answers with this is based on today's situation. The world is changing so fast, 12 months time, things may have moved on and we might answer things slightly differently but I think you're absolutely right.

I think AI has expanded what is possible and the speed that things can be done massively but human judgment for me determines really what is responsible, what is valuable and what is right and I think it's probably best to maybe use an example to illustrate that and maybe if I use handling an insurance claim or making a decision about an insurance claim because very often there are slightly grey areas that might be involved in making a decision. AI can review documents, it can identify precedents, generate recommendations incredibly fast based upon its learnings, it can highlight potential areas of fraud, likely outcomes and it can package and present all of that information very very quickly but ultimately I think where a human can make a difference is in determining whether the recommendation is made is fair, proportionate and appropriate. I think a human can consider factors probably that we talk about wisdom where it requires contextual information that might not necessarily be in the immediate information that the AI has gathered so it could maybe identify customer vulnerability or exceptional circumstances or potential reputational impact associated with making some of those decisions and by applying that context and that wisdom I think we can make better decisions but I think the exciting thing is it's not thinking about AI being bad, person being good or vice versa, it's about how the two can work together to ultimately make better decisions and make our businesses perform better.

Maurice

I fully agree with that but I suppose within that then that this whole issue about where agency ends and accountability begins is relevant isn't it because if you've got an agent, there's an example given at one of our events recently where the Head of the CFTC was talking about regulating agents and he was saying how do you deal with a situation where you have an autonomous agent changing on the hoof as you like and trying to make money on a blockchain without having any regard back to the person who designed the original software. Where does agency end and accountability begin? If an agent can go out and change its pattern of behaviour and the sequence in which it's doing work, what are the boundaries? How do you hold an agent accountable or do you hold the software development of the person using that agent responsible? How does that work?

I think exactly as you say, you can delegate an action to an agent but you can't delegate accountability for the outcome and I think really when we talk about AI increasingly we talk about outcomes as opposed to inputs and it's that outcome that humans have to take and organisations have to take accountability for. And I think when designing processes and re-looking at processes with AI embedded within them that has to be the principle that you stick to.

And again I guess we kind of look here for illustrations because it's easy to maybe say these words, it's what that means in practical terms. So again if I was to take a loan application, AI can verify information, it can analyse affordability, detect anomalies, recommend approval or rejection and it can package all of that together incredibly fast. But a human can look at those unusual life circumstances, they can look at where policies might need to be flexed to accommodate specific situations and ultimately I guess anything where there's that regulatory perspective as well and ultimately the bank in that situation is accountable to the regulator.

It's not the software engineer that developed the coding, it's not the AI agent, it's the bank that is accountable and where ultimately the bank and the directors of that bank are accountable to their customers, regulators and shareholders, they have to factor into that decision-making chain, the human element, because they're accountable for that. 

Maurice

Okay and there's a great play made of the idea of AI and the genetic AI being a way of augmenting the work of humans. What changes to roles and centres and learning are necessary so that AI makes people more capable, not merely faster or dependent on a tool?

Yeah and I think that's absolutely right.

I mean there's that principle, I'm sure you've heard other people say this, you know the goal is not to accelerate yesterday's jobs, it's to design work better and I think this is really relevant here and I've touched on the concept of outcomes and the focus on outcomes already. I think increasingly the focus of business is on outcomes and in terms of humans' role in the world, the focus is on the outcomes of the things that they do. So whereas historically they might have been measured on the number of transactions they processed or in call centres, the number of calls they handled or the number of loans that were assessed very much on a productivity basis and you know a whole kind of organisational structure and personal incentives are based upon throughput and productivity.

I think increasingly that's now massively accelerated so it's less about how many things get processed, it's about the true value that they deliver and what is the outcome and I mean as an example, if you're making assessments of loans, it's not how many loans I've assessed, it's how good is the business, how valuable is that business, what's the impact on customer perception, on customer loyalty. And it's really focusing everybody on those outcomes and linking their training, the organisation and the way they're incentivised around delivering outcomes as opposed to necessarily the inputs that went behind that which historically we've been incredibly focused on.

So quality not just quantity of the work provided and the outcomes on all those structures.

Looking at the internal governments of companies deploying the genetic AI, many of them have high-level AI principles but very few have operating disciplines that can work when pace and commercial pressure rise. What are the minimum governance mechanisms you would expect to see around an agentic AI deployment such as escalation routes, decision logs, challenge processes, testing and named accountability to make human oversight meaningful? 

Yeah I mean gosh there's lots of questions within that. I think that there's a principle of proportionality which is important as well.

So it's understanding the risks and the opportunity within those decisions, how critical are they to the business and then putting proportional governance around it. I mean I did see a great story come out from Claude actually, they had some processing which as something they supported and as part of the process they were sending it out to humans to check but there were so many that were being sent to the humans they pretty much got numb to it and they were just saying yes, yes, yes, yes, yes. So it's also recognising that just pinging everything to humans and then measuring them on the number of approvals they give doesn't help either.

It's completely rethinking it and it's thinking very much in terms of accountability so it's making sure that all decisions are linked back to individuals and those individuals put proportionate governance in place to support it. I think that you touched on the control logs, evidence logs. I think particularly in financial services where we're heavily regulated actually there's an opportunity to massively improve an organisation's ability to evidence that it's following appropriate processes because the AI can provide the evidence logs that are fed into the decision that then a human can consider and they can make final decisions based upon that wisdom in the wider context.

So I think key things, I think humans have to be accountable for those key decisions, named humans and then it's about proportionality and making sure as well that you don't just swamp the human with a thousand approvals to do in a day because that's not going to help the quality either. 

Maurice

Great, and final question because I think we're running out of time. A lot of people are very keen, a lot of corporate executives are very keen to deploy AI and agentic AI but they're somewhat at sea as to what should they do first and I suppose it'd be interesting to hear your view.

What are the first steps that people should take as they assess the use of agentic AI in their companies and what evidence 12 months later would you tell them that they made the right first steps and the deployment was working?

I think that's a great question and again I don't think it's actually about choosing the technology and it's also not about the amount of AI that's been deployed either. I mean I've seen firms saying we've got 100,000 agents deployed. I mean I think for me it's all about those CEOs choosing the decisions and outcomes that they know are most critical to their business and it's choosing the two or three most critical decisions and leveraging AI to help them make better decisions on those.

I think doing a few really critical things well and making sure it gets embedded, that the organisation is changed around it, people's incentives are aligned to it and not just thinking about it as a technology project is what will help them move things forwards and I think the other thing as well is again if you try and put AI in everywhere it's very hard to track the impact and benefits and return on investment from it whereas again if you pick those two or three things that you know are most fundamental to the success of your business it's I think far easier then to track the impact of the changes that you made. 

Great, so a holistic approach is what they really need to do. It's not a separate thing, it's integral to their business.

Paul that was a really interesting discussion, thank you so much for that. For our viewers, if you'd like to hear more about this and to learn more about Agentic AI in a corporate context, please visit our website www.cityinfinancial.com where you'll find full details about the programme, the speakers and the themes and you'll be able to register to attend the event which again just to say it's being held in London on the 17th of September and it's called the Agentic AI for Leaders Summit. Paul, thank you very much indeed.

Thank you.