The Enshittification of AI in Healthcare: An Alternative Ending
There are three things in life that are inevitable – death, taxes and technology platform providers exploiting users through monetisation techniques and ‘dark patterns’ to line the pockets of their shareholders. The lesser-known of these three evils, and the subject of this article, is what was recently coined by Cory Doctorow as ‘enshittification’.
Enshittification, more boringly known as ‘platform decay’, is how platforms die. A concept which Cory succinctly articulated in his January 2023 article, titled TikTok’s Enshittification:
Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die. I call this enshittification, and it is a seemingly inevitable consequence arising from the combination of the ease of changing how a platform allocates value, combined with the nature of a “two-sided market”, where a platform sits between buyers and sellers, hold each hostage to the other, raking off an ever-larger share of the value that passes between them.
In the eyes of fickle users and frustrated brands, this could be considered karma for consumer platforms who rinse their users’ goodwill and joy. But what happens if millions of pounds of public money are invested in such a platform? It’s not so easy to shrug it off and say ‘enshit happens’.
Enshittification is Already Happening
Omnipresent search tools, social media juggernauts, and even the latest kids on the block TikTok are in various stages of this death spiral. But what has this got to do with health? This wouldn’t possibly happen in the NHS. Would it?
Let us consider for a moment the duopoly of electronic patient record (EPR) systems in primary care. Few would argue that the two major suppliers don’t hold the crown jewels of primary care data in their hands. If you have ever tried to get access to GP data, you’ll know it’s not just a question of public good. Instead, the data is shrouded by systems that don’t want to talk to each other and consumers are charged to display the data in other systems.
The parallels between the enshittification of social media platforms, what has already been allowed to happen in other areas of health and care, and what COULD happen with AI are very troubling.
Like EPRs, in five to ten years it will be impossible for healthcare providers to function efficiently without AI. AI will be ingrained within the fabric of almost every workflow, from radiology to routine consultations and clinical decision-making. If in the future the tap is restricted or turned off completely, a hospital could grind to a halt.
If Something is too Good to be True…
At present, I see the AI model vendors and platform suppliers engaging in a ‘give everything away’ strategy. The aim being, to get NHS teams hooked on the productivity gains AI promises to deliver and secure as much of a land grab as possible with lower prices and the latest variety of models.
In my opinion, it’s woefully naive, verging on negligent, to think this will always be the case and say ‘this time it’ll be different’. Once the technology is integral to the fundamental ways of delivering care, shareholder demands will inevitably bite.
Look at TV. At first, there was free access to four channels. Then came the explosion of choice via satellite and digital freeview (which are flooded with ads), and finally streaming services. Now if you want to watch your favourite shows you have to search what platform they’re on first and hope you have a subscription to it. This has led to a resurgence in piracy, but there will be no way out for the NHS, but to pay for a captured market.
Over-reliance on Single Commercial Providers
If the NHS acts without a platform strategy, that is privately rather than publicly owned, enshittification will happen. The only way to avoid this scenario occurring is a national platform that is funded and protected as a national asset.
To be clear, I am not talking about the AI models here. Investments in AI research will reap dividends for both investors and the NHS. I am talking about the platform on which they will operate. The platform that will become as much a part of our national infrastructure as our roads are today.
If a commercial platform provider holds all the intellectual property for a platform being used by a trust, imaging network, or integrated care system, then those organisations are restricted to using the models the platform provider has a relationship with. These are likely to be those that are commercially most appealing to the platform provider, not which are most needed by the procuring organisation or present the best opportunity for improving health and care based on the latest innovations. If you want a model outside of your current platform then you will be forced into multiple subscriptions for the same service like we have seen with Premier League football rights with Sky, TNT and Amazon.
This scenario will lead to stifled innovation, a lack of competition in the platform market, and ultimately inflated costs for the NHS.
The Alternative is an Open Platform Approach
AI is moving fast. But there’s an alternative option to ensure the NHS is not locked out, or priced out of future innovation. That is an open platform approach.
A platform already exists that is built on open-source technology developed by Medical Open Network for AI (MONAI), a first-of-its-kind set of open-source, freely available collaborative frameworks built for accelerating AI research and clinical AI deployment in medical imaging.
The platform is being continually developed, for the community by the community, side-by-side with global tech leaders and world-renowned academic and clinical institutions such as UCL, Stanford University, Mayo Clinic, and Guy’s and St Thomas’ and King’s College Hospital NHS Foundation trusts.
So, why is this option the only viable alternative to avoid enshittification? AI is an emerging technology for healthcare. This means new systems deployed into clinical workflows must be thoroughly evaluated before they are used in direct patient care.
An open AI deployment platform approach will facilitate future innovation, rather than a closed proprietary platform which will restrict innovations based on what is most commercially appealing.
The existing open deployment platform enables an organisation to run multiple AI models in shadow mode before moving to full commission. Each model can first be tested on fictional data, followed by retrospective data before moving to a live clinical environment.
AI is such a big concept for healthcare to imagine, and at the moment the vast benefits are not yet all conceivable. The only way the NHS can avoid restricting its future ambitions to the whim of companies who wish to own all the intellectual property and data, is to own the deployment platform itself. With open-source AI deployment technology, this is possible.
By Rich Pugmire, CEO, Answer Digital
