Weeknotes 1.9
Last week, attending UK Kidney Week for an excellent clinical update gave interesting context on the proposed UK MedTech pathway (join the consultation!).
The ambition here is fantastic – creating a structure for assessing evidence (and affordability) to allow proven technologies to be more rapidly adopted into routine commissioning. However, framing this work against the established evidence pipeline for new medications highlights many of the challenges in digital health.
Evaluation of new medications follows a well defined process that builds an evidence base for safety and efficacy. Importantly this process extends after licensing with phase 4 trials and post-marketing surveillance. This work can have substantial safety benefits, evidencing potential harm and changing how future research is conducted.
Randomised controlled trials measure pre-defined outcome(s) and the impact of a defined intervention on these, ideally comparing current best practice against a credible new alternative or addition. Unlike most digital health solutions, this allows clinical teams to understand the impact of a therapeutic option that would be added into our existing model of care. This means that ‘positive’ findings have the potential to be brought into existing models of care without the need to reconfigure how care is delivered. Although the journey from basic science to proven therapy can be long with many overnight successes representing decades of careful work, the time taken to translate evidence into clinical practice is shortening.
Contrast this with many digital health interventions. It’s axiomatic that digital change, done correctly, can change the nature of a problem. This is one reason why existing models of evidence synthesis can fall short when applied to digital health technologies, with broad conclusions and few definitive answers.
I think we need to have different assessment frameworks for technologies that can be deployed into existing models of care vs those that change the how care is delivered. This second models requires us to understand success factors for delivery / operational changes as much as the evidence of benefit to be able to make informed decisions about scale and spread. Many suppliers may (rightly?) feel that we already live in an extreme version of this world with every deployment being ‘different’ and the burden of proof of value never ending. This is a barrier to getting the best tools used and a needless burden. We need better evidence not more evidence.
Building the evidence base
Finding a middle ground that respects local organisation differences whilst still allowing evidence to be acted on at broad scale is the nirvana we need to strive for and what the MedTech framework could help to achieve. But this will be hard.
Building this level of evidence in a way that is sufficiently directive without NICE having to set the ‘exam questions’ each time will be challenging. An evaluation framework that creates an environment where solutions are shaped against the “evidence needed” and not the clinical / operational problems needed to be solved will help no one!
Here digital health evaluations could learn from how pharmaceutical companies bring together expert clinicians to independently define (and defend) the outcome measures that they (as the experts) set. This model is not perfect and competing interests abound on all sides, but agencies such as the FDA place appropriate emphasis on these interactions.
Innovations should have implementation and evaluation criteria set and agreed before an initial proof of concept delivery to avoid the risk of these measures being overfit to localised challenges and needs. Openly agreeing and sharing these criteria is vital to know the outcomes shared were the ones agreed and not simply the result of a fishing expedition. These outcomes must be meaningful and break our reliance of surrogates. To build credibility, digital healthcare evaluation needs clinical reference groups to independently define evaluation criteria and something similar to a public register of clinical trials.
Understanding cost
Once benefit is shown, the next question is how value can be assessed. In economic evaluations of medications it is common to factor in a depreciating effect over time due to changes in treatment concordance. Technologies that become established and those that change how healthcare is delivered will at some point become business as usual and their initial value or ROI therefore degrades. This needs to be accounted for to ensure sustainability though not in a way that makes change impossible. The status quo is not without huge costs…
Planning for delivery
Finally, it is imperative that we don’t only focus on benefit measures and return on investment. Each year in kidney medicine we clinical journals publish more and more predictive models that can predict risk for kidney failure. What we do not see are descriptions of how to deploy these at scale or assessments of the real world change that this achieves. We continue to build incrementally better tools with no real consideration of how we can make use of them in a way that improves patient outcomes.
Taking an extreme view, Apple are often criticised for “not innovating” enough. However, watching their recent WWDC keynote it’s almost impossible not to be blown away by how well they have brought together a range of established ideas to create something incredibly impressive. I’d view Apple’s true strength as Implementation rather than Innovation. Many if not all of their products have been best in class rather than first in class.
I see a lesson here for the NHS. We have Innovation agencies, but not Implementation Agencies. Change at scale is hard and a focus on this whilst we are evaluating innovation is vital. No matter how good a new medication is, if the infrastructure needed to get it to patients does not exist it helps no one. The same is true for digital technologies. We need to closely link our understanding of how to innovate and implement.
Despite this, I’m not arguing for Implementation agencies – this is a road to ruin with innovation, iteration, interruption, implementation and other agencies all falling out of that path. What we need is meaningful agency that allows place and provider to confidently and quickly adopt new technologies. This is why, despite the MedTech framework being a good first step, I think it falls short. Healthcare is a complex adaptive system where the environment and infrastructure have a much larger impact on the success or failure of technology driven change then on new medications. We need to describe this and gladly accept that the cost of having more innovations scale successfully is understanding that not all innovations can scale equally.
Leave a Reply