Following June’s NHS Confed Expo (sorry, no selfie), the HSJ have picked up on some provocative comments made by Shankar Sridharan, the CCIO at Great Ormond Street Hospital. I didn’t hear the comments in person, but based on the article, I think it’s worth exploring some counter points to what has been reported in the spirit of contributing to the discussion that I think he was trying to provoke.
His main argument seems to be made is that staff are using ‘incredible’ AI at home but at work are pushed into an underground world of shadow AI due to workplace policies and prohibitions. I agree with parts of this so will start where we have common ground.
Right about the shadows, not the label
I’ve presented the argument before that any block you can’t enforce isn’t a control. It’s at best false assurance or a policy comfort blanket. Banned doesn’t mean gone; it really means hidden. This leads to the worst outcome: all of the risks, less (or none) of the benefits, and no credible way to audit or understand when something goes wrong.
It would be incredibly naive to suggest that patient details aren’t being entered into consumer chatbots in hospitals right now. However, trying to prevent misuse isn’t the same as a blanket ban, and using the word ‘criminal’ is a mislabelling that is unhelpful. Like the tool or not, NHS England fairly point to their national rollout of Copilot to circa 500,000 staff to make the case that AI is not forbidden.
Perhaps a more balanced complaint should be that whilst AI isn’t banned in the NHS, the services that are sanctioned are too sparse and have been too slow to arrive. This feels like a fairer criticism, but calling this a ‘crime’ creates moral polarity in the discussion and positions people doing valid safety and assurance work as the wrongdoers. Whatever your view on the utility of Copilot, one of the reasons that it is being handed out at scale is because there has been effort in the enabling assurance required to move from risk to rollout.
The strongest argument is the riskiest
Although I most admire the ambition, I’m less comfortable with the urgency to move past tools that are “not that exciting” toward those that can “be the contrarian”. Using AI to present alternative diagnoses or identify treatment options could be of value is in the high-stakes clinical territory where assurance matters most, not least. The more you want AI to influence or change clinical behaviour, the more it must be checked, regulated and monitored. “Opening the front door” might work for some opportunities, but the same door that brings in a tool that supports administrative efficiency probably shouldn’t be the same door to bring in a tool that influences the care of a critically unwell patient.
The MHRA’s AI Airlock is helping to test these boundaries. To build a metaphor around this, governance isn’t building a barrier to keep us off the railway, it’s the track being put down ahead of the train. We need an environment that means when a contrarian AI arrives, clinicians will trust it enough to use it.
Pilots, proof of concept and evaluations
Another argument presented is that endless ambient voice technology (AVT) pilots are “nuts” because we don’t need to continually re-evidence benefits. Again, I half agree. It’s fair to accept there is consistency in evidence for areas such as documentation time and staff perceptions of the technology. However, some of the harder questions have more varied evidence and our understanding of accuracy, consistency across settings, product variation and the impact of these tools on our workforce is evolving. Hopefully no one would disagree that re-running another pilot to confirm what we know is daft, but extending this position to “the benefits are proven” is a false equivalence. A varied evidence base is not a settled one.
If we all agree that pilots should be limited, then one question I’d politely have asked at the panel is which digital health technologies GOSH has adopted based solely on external evidence. For anything that touches direct clinical care I suspect the list is very short – but rightly so. Someone has to go first, but GOSH didn’t adopt AVT on the basis of someone else’s experience; it ran an extensive trial to be sure it worked, was acceptable and was safe. That is the work that made their broader rollout viable, defensible and investible.
This is the part we tend to gloss over. ‘Move beyond local evaluation’ is easy to say and structurally very hard to do. The issue isn’t that NHS organisations are too timid to copy one another; it’s that careful local appraisal is real, valuable work that is hard to fund or to design in a reusable way. This is why evaluations are repeated. We’ll know we’ve made progress when Expo hands a prominent platform to the trust that did that unglamorous appraisal well, and showed others how to genuinely apply the findings, not when we keep treating the caution of the second adopter as some form of character flaw.
Good governance is infrastructure, not a barrier
It’s now somewhat of a cliché to describe digitally enabled change in the NHS as being about people and process rather than technology. Especially with a rapidly evolving area such as AI, this framing needs to expand to include trust. Clinicians will only use what they trust and can defend; assurance and engagement help to build this.
Returning to the transport metaphor, a public transport service that people trust due to its solid tracks and reliable signalling is one that can move hundreds of thousands of people each day. This doesn’t mean options with less assurance (the unlicensed taxi) don’t exist, but they remain in the shadows.
The UK’s pro-innovation framing and initiatives such as the AI Airlock should help co-design routes to market and scaled adoption. This is a model for responsible innovation that will help deliver change nationally. Here governance is an enabler, but capacity is the blocker. The freeze Shankar points to is real, but there’s a misdiagnosis about the cause. It isn’t too much governance. It’s too little, delivered too slowly. We have a good national model, but at present it’s the equivalent of a service so sparse that people have to set off on foot rather than wait for the train. Stating the obvious, the answer to a limited timetable is to run more trains, not to declare the railway itself a crime. So in short: don’t ban AI, and don’t bless it. We need to put our efforts into building the trusted service and running it often enough, and to enough places, that it becomes the obvious way to travel. For the NHS this means standards, registered tools with proper data models, real money to adopt, and the medical-device rigour that lets the clever clinical stuff arrive safely rather than not at all.
I understand the impatience. I agree that leaving people to copy patient notes into a commercial LLM is a failure. But the failure isn’t that someone said “not yet.” The cautious aren’t the criminals here and better governance is what will earn us the right to go fast.