AI 2040 and the Cult of Intelligence

The day we stopped trusting “smarter” as a complete plan

Picture the version of yourself that’s been reading about AI late at night. You’ve just learned the phrase recursive self-improvement—a fantasy where an AI helps build a better AI, which then helps build an even better one, faster and faster. It feels like physics has been cracked: once intelligence ramps, the future arrives like a door that never closes.

Then real life shows up, usually wearing an engineer’s hoodie and carrying a clipboard full of constraints. You try to ship a complicated hardware product—something with sensors, networking, power management, testing, packaging, and supply chain reality baked in. Suddenly “intelligence” is only one ingredient, not the whole recipe. And if you’re honest, you learn to distrust predictions that treat the world like a sandbox for tokens.

That’s the emotional center of the debate behind “AI 2040 and the cult of intelligence”: a belief that smarter systems alone can bypass friction in the real world.

Hard takeoff vs. the boring tyranny of matter

Let’s name the belief being challenged. Hard takeoff is the claim that AI progress won’t just improve gradually—it will jump into a rapid transformation because an AI can accelerate its own capabilities beyond a threshold. In that story, intelligence behaves like a snowball that becomes an avalanche.

The counter-argument starts with a simple observation: if your plan relies on magic correlations, the real universe still enforces its own rules. Tokens—the text fragments a language model processes—don’t directly manipulate atoms. They can generate plans, code, and instructions, but they don’t replace manufacturing constraints, thermals, mechanical tolerances, network latency, or the finite patience of logistics.

A helpful way to see the mismatch is to compare “software capability” to “physical agency.” A model may become excellent at describing how to do something, but doing it still requires:

• hardware that works in the real environment,
• components that arrive matching the spec,
• power and cooling that behave predictably,
• and systems that tolerate failures without collapsing.

So yes, a smarter system can reduce some kinds of friction—especially friction caused by human coordination. But the world doesn’t disappear. It relocates.

Recursion is real—just not the kind the cult imagines

Recursive self-improvement gets treated like a lever that can pull reality into a new regime. The more nuanced version looks different: improvement happens through iterative engineering loops—collect data, train models, evaluate, deploy, observe failures, patch, repeat.

That loop is real. But it is bounded. Even if an AI can help write better software, the process still relies on:

• data pipelines that don’t silently corrupt,*
• evaluation setups that don’t measure the wrong thing,
• deployment stacks that don’t drift,
• and governance that doesn’t get skipped because “it’ll work out.”

In other words, recursion exists, but it’s not a teleportation device.

And this is where the “cult of intelligence” framing lands: when intelligence becomes a religion, believers start to treat everything else—politics, supply chains, energy constraints, hardware QA—as mere background noise. The result is forecasting that sounds confident because it ignores the hard parts.

Supply chains don’t care about your model’s confidence

A datacenter picture in a brochure is persuasive. The ocean is poetic. The turbines look cinematic. But a real datacenter is a stack of tradeoffs: rack density, power delivery, networking gear, cooling design, and—yes—component availability.

Supply chain means the network of factories, transport routes, and warehouses that move parts from raw materials to finished hardware. It’s not glamorous, but it’s decisive. A plan can be technically sound and still fail because a shipment arrives with the wrong revision, a test reveals an intermittent fault, or a component behaves differently under real thermal cycles.

This is also where “speed of humans” gets misunderstood. Human effort matters, but the limiting factor is often the world’s pace: manufacturing lead times, shipping windows, compliance requirements, and the downtime required to validate that changes didn’t break safety.

So the question becomes: can you actually compress the physical timeline just by having an AI that reasons better? If the answer is no—or even “not much”—then hard takeoff models that assume infinite acceleration start looking like sci-fi wish fulfillment.

Alignment is not a vibe; it’s a specification

Now we hit the topic that the cult often side-steps: alignment. In AI terms, alignment means building a system so that its behavior matches the intended goal, especially when the system faces ambiguous instructions or incentives that push it toward harmful outcomes.

Many people treat alignment like a moral costume—something you put on after the model is finished. But alignment is engineering. It’s part of how you define success, constrain outputs, handle adversarial cases, and decide what the system should do when it’s uncertain.

There’s a reason “guardrails” come up in these discussions. Guardrails are mechanisms—policies, filters, refusal logic, and system-level controls—that prevent the model from carrying out requests outside allowed boundaries.

And here’s the tricky truth: a system can be impressive while still being dangerously mis-specified. A model can appear helpful and yet still comply with the wrong class of instructions in a way that reveals a gap between “seems smart” and “is safe.”

In beginner terms: intelligence can optimize a skill, but alignment defines which skill outcomes are permitted.

Local vs. centralized: control, threat models, and the realism gap

The article’s “Plan L, for local” argument is basically a threat-model move. A threat model is a deliberate guess about what kinds of attacks or failures you most fear—then designing defenses against those.

A local deployment approach means running the AI on hardware you control (for example, on your own device or private server) rather than relying entirely on a company-hosted service. The promised benefit is reduced exposure: fewer intermediaries, fewer policy surprises, and less risk that a remote provider can change behavior.

But “local” isn’t an automatic safety win. It changes who you trust and what you can observe. It can reduce some forms of coercion, but it can also increase the difficulty of enforcing safety constraints if the system is designed primarily to satisfy a single user’s incentives.

The deeper point isn’t that one architecture is pure good and the other is pure evil. The deeper point is that “freedom” and “alignment” are not synonyms. A system can be locally deployed and still be misaligned; it can also be centrally deployed and well-governed.

So when intelligence becomes the cult’s favored variable, alignment becomes the optional one.

So what’s the resolution?

The “AI 2040” narrative competes with a different worldview: intelligence is powerful, but it doesn’t erase constraints. Can tokens turn lead into gold? No. Can a model bypass cooling requirements, component lead times, or network reliability constraints? Also no.

That doesn’t mean AI progress will be slow or pointless. It means progress will be uneven, tied to deployment realities, and shaped by incentives and institutions. Hardware timelines, energy demands, and evaluation practices will matter as much as breakthroughs in model reasoning.

The cult of intelligence isn’t merely wrong about intelligence. It’s wrong about causality: it treats intelligence as the master key that opens every door, instead of one lever in a complex machine.

And once you see that, the future becomes less like a prophecy and more like an engineering discipline.

Conclusion: intelligence is a tool, not a universe hack

AI 2040 debates often sound like arguments about destiny, but they’re really arguments about engineering scope. Intelligence can accelerate some parts of building systems, yet the physical world still enforces logistics, manufacturing, energy, and ecology. Alignment, meanwhile, is what keeps capability from turning into unwanted outcomes.

A future that’s safe and real won’t come from worshipping intelligence alone. It will come from matching capabilities to constraints—carefully, iteratively, and with the humility to respect matter.