I keep thinking about the future of AI through the lens of a hospital: the cleverest technology starts in the specialist centres, then slowly moves closer and closer to the patient. I think OpenAI and Anthropic don’t have a future the way things are going. Inspired by a presentation by Pat Grady from Sequoia.
Tests that once had to be sent to an outside high-tech lab can now be done in the hospital, on the ward, at the bedside or even on a device attached to a patient. Big specialist labs have not disappeared, they’re still needed for some cases. But as tech gets cheaper, smaller and better, less and less needs to be sent away to an outside lab or even to a lab in the hospital itself.
Will this happen with AI? OpenAI, Anthropic and the other AI powerhouses are building the equivalent of extraordinarily sophisticated diagnostic machines. These machines can do things that smaller models cannot yet do. But they are also in a breathless race. While they spend enormous amounts making the next machine more capable, cheaper and faster models are coming up behind them, FAST.
Some processing can already happen on devices. Apple is really into this. Its newer systems can run AI models directly on the device, and developers can build apps that use those local models without sending every request off to a cloud service. More complex jobs can then be routed to larger cloud models when needed. Apple is also making it easier for developers to bring their own models onto devices. So we could end up with AI sitting inside the app itself, doing the everyday work locally, and only calling out to a much larger model when it actually needs one.
The same thing can happen at enterprise level. More can happen in private clouds. Enterprises can build or tune specialised models for their own work. Thomson Reuters is already doing something interesting here. It has built Thomson, its own model, starting with an open-source foundation and training it around its own legal, tax and regulatory knowledge. Its argument is that owning more of the AI stack gives it more control over performance, cost, deployment and data sovereignty, instead of depending entirely on what an outside model provider decides to build next.
In the Sequoia talk that started me thinking about this, Pat Grady describes JEV as handling a particular class of decisions around 100 times more cheaply than an LLM. Is OpenAI going to have a Kodak future?
So perhaps the big AI crowds gradually become the equivalent of the specialist labs, only used when the job genuinely requires them. Routine work gets pushed closer to where it is needed: into the enterprise, the application, the private cloud or the device itself. Today’s difficult problem becomes tomorrow’s bedside test. That leaves the big AI companies with a terrible problem: they have to keep moving the needle because everything behind them is steadily becoming commoditised. They may keep winning the technical race while losing more and more of the everyday traffic.
And another problem for them! Most of us don’t use technology anywhere near its limits. We carry supercomputers in our pockets and mostly use them to check email, send WhatsApps, look at photos and doomscroll. Every year the phones get faster, cameras get better, storage greater, yet we’re still doing the same boring shite. AI may be no different. OpenAI and Anthropic can keep adding extraordinary new abilities, but that doesn’t mean most people or businesses will need them.
If smaller, cheaper AI tech can do the boring everyday work perfectly well, what happens to the economics of constantly building the biggest and most expensive machine? Perhaps BIG AIs become specialist providers, perhaps they become the routing layer, perhaps they merge or disappear. I think Copilot has a future: massive installed base and another boring Microsoft app!