The intelligence explosion, re-examined
Twenty years after the argument was written, the debate has moved from whether the curve is steep to what constrains it.
The argument, as it was made
The original case was simple enough to fit on a napkin: intelligence is the ability to solve problems, including the problem of building better intelligence, therefore a sufficiently capable system improves itself, therefore takeoff is fast and possibly discontinuous.
Two decades of argument have not refuted it. They have moved the disagreement somewhere more useful.
What the debate moved to
Almost nobody serious now argues about whether the curve could be steep. The argument is about what limits it:
- Compute and energy. Training and inference both scale with power. Grid interconnection queues are measured in years.
- Verification. Being able to check a claim is a different capability from making one, and it lags.
- Coordination. The number of organisations that can act on a breakthrough is small, and their incentives are not aligned.
- Physical lead times. Fabs, data centres, and supply chains move on industrial time, not software time.
Each of these is a reason the transition might be slower than the napkin suggests. None of them is a reason it does not happen.
The version that actually bites
A slow explosion is more dangerous than a fast one in one specific way: it gives institutions just enough time to believe they are in control. Rules get written for the previous generation of the technology. Oversight bodies are staffed for a world where a human can read the output.
The most likely failure is not a system that arrives too fast to regulate. It is a system that arrives slowly enough that nobody notices the regulator stopped being able to read the work.
What follows from this
If the constraint is physical, then the near-term story is about infrastructure: power, packaging, and the politics of building at scale. If the constraint is epistemic, the near-term story is about verification: who can independently check what a system claims.
Both readings point the same way. The scarce resource is not intelligence. It is the ability to trust a result you did not personally produce.
