By now it’s likely you’ve heard of the whole concept of the AI paperclip maximizer problem — a hypothetical first put forth by Nick Bostrom — suggesting that a misaligned AI system that looks to optimize on the wrong single thing (in this case creating paperclips) could lead to the end of humanity. Specifically, the thought experiment is that if you have a sufficiently powerful AI tool that is told to “maximize” the creation of paperclips, it might decide the best way to do that is to seize control of all resources and dedicate them solely to making paperclips. And, naturally, that might include grinding up humans, either as raw material for paperclips or as fuel to power the paperclip factory.
The supposed “lesson” of this is to be cautious about these AI tools, though I think the actual lessons are twofold: (1) thought experiments can only take you so far and (2) never over optimize on a single metric. Indeed, this was part of what our recent series, The Metric is Not the Mission, was all about.
Anyway, everyone in the AI world knows about the whole paperclip thing. I guarantee you Sam Altman knows about it. He has praised the book where Bostrom put forth the hypothetical, claiming that it was “the best thing I’ve seen” on the topic of “will AI kill us all.” And he’s joked about it in a memey way:
But it’s not clear that he truly has internalized that lesson regarding not optimizing on just a single thing. Last week he went on John Collison’s (co-founder of Stripe) “Cheeky Pint” podcast to talk about the state of AI.
A bunch of the headlines focused on one specific thing that he said, however, that it’s “quite terrible” for people to be wasting RAM in home machines over having them working full steam in a data center somewhere. The context here matters a bit. Collison catches Altman a bit off guard by asking “how’s the compute shortage going?” just as Altman takes a big gulp from his pint of beer, and you see Altman try not to laugh/spit out his beer, before there’s an obvious cut in the video, where he recomposes himself.
They then have a discussion about the increasing demand for compute, Altman talks about how they’re working on having robots build “self-replicating data centers” (what?!?) to which Collison rightly interjects that the problem with data centers right now isn’t the lack of labor available to build them. Collison then shifts the conversation back to the stuff that most people care about: the fact that RAM prices have increased dramatically recently, which is jacking up the price of everything from computers to phones.
Collison notes that people keep saying the market should take care of this: if prices keep going up, shouldn’t Adam Smith’s “invisible hand” motivate more companies to make more RAM and ease the shortage? Altman shows that he’s not a big believer in the invisible hand of the market concept:
Collison: My observation, the compute stuff is they say: “The solution to high prices is high prices.” And it’s like, okay.
Step 1: We have the high prices for memory, and GPUs, and everything. The solutions should be here any moment now. How are we doing on the solutions front?Altman: Bad.
Collison: And I’m not saying this is OpenAI’s job. I feel this is the invisible hand’s job, right? Memory’s gotten really expensive. So, new memory makers are going to flood the market any moment now.
Altman: I think the invisible hand is failing right now. I mean, the invisible hand… It has like a checkered track record at best. But, I think it’s not quite working here.
He then points out that, for decades, DRAM was a commodity product with razor-thin margins and barely any profits. His argument is that memory makers aren’t willing to expand much because they’re afraid that the bubble might burst and they’d have overbuilt (which… might not be wrong?).
Of course, Altman leaves out of this the fact that OpenAI has basically tried to corner the market on RAM, trying to lock up 40% of the world’s supply without even guaranteeing that it will follow through on the actual purchase orders. I mean, I think that alone would have at least some impact on the current price for RAM, wouldn’t it?
Collison pushes a bit further, and still wants to know why others aren’t getting into the memory market, and even (half jokingly?) suggests Intel could get into the market — a reference to one of the key parts of Intel’s lore and mystique: the Andy Grove decision in 1985 to get completely out of the memory chip business to focus on processors, which got a bunch of recent attention when Intel’s current CEO suggested the (struggling) company might reverse that decision 40 years later.
That brings Altman around to the critical quote regarding local RAM, in which he dismisses the argument that local inference is important, as a way of suggesting that the more efficient use of memory would be in large datacenters where it can be utilized at a much higher rate:
Oh, I think there are all of these ways where the supply chain—it’s just super non-obvious.
An example of this is people…A lot of people want local inference. And somehow, they assume that the problem is solved now. Because, we are going to have open-source models. We’re going to be able to run them on our computers. And so we’re not going to need all the stuff in the data center.
But from a global allocation perspective, actually, it’s quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer. And, being used—Mm-hmm. 5% of the time. You want maximum utilization in the cloud if you’re trying to deal with this very supply constrained environment.
And somehow everyone’s… many people’s intuition is the opposite of that.
So he’s correct… in one very, very, very narrow sense of this quote. If you are only interested in dealing with the current “world’s limited supply of DRAM” he’s not wrong that it is more “efficient” to have that RAM be in a data center and then rented out to you. And he’s not exactly wrong that if everyone just paid per token for remote compute, the cost of RAM in your own PC would matter a lot less.
But… he’s kinda missing the larger point: that people are not just focused on optimizing for the most efficient use of memory. They’re not just optimizing for what’s best for “the global supply chain of RAM makers.” They have other concerns — and other uses beyond AI — as well, such as their own sovereignty and control over their own tools and systems. There’s a reason my essay on the need for decentralization (including in AI) went so viral a few months back.
And it’s not because people think they’re single-handedly going to solve the global supply crisis in DRAM by running everything locally. It’s because we’ve seen what happens when a central provider controls key parts of our stack, and the results are very, very, very bad.
Using open weight models on local hardware may not be the most efficient for the exact price of RAM (which, again, Altman tried to corner the market on), but it certainly leads to much more actual user empowerment by people who want to use these tools, but without outsourcing control to someone like Altman.
In a pure “efficiency” model where you are solely optimizing on what’s the most cost efficient way to use memory the answer would be to have it in datacenters where you’d benefit from both economies of scale as well as flexible utilization. But that has consequences. It’s the same logic by which a fascist dictator may seem “more efficient” to those sick of democratic quirks and bureaucratic red tape. It only sounds good if your mental model is completely screwed up and simplistic.
In both cases, you’re talking about a very simplified mental model, optimizing on a suboptimal point. Personal autonomy over our tools and our systems is more important than making your data center more efficient, Sam. The paperclip problem comes from focusing on a narrow metric and over optimizing on it. And in this clip, Sam is paperclipping RAM.
