If AI Takes Control, What Would It Control?

Hard on the heels of my essay When AI Goes Rogue, the debate over artificial intelligence suddenly became much louder. We now ask what happens if AI takes control

Researchers have warned that we could lose control of increasingly capable AI. Some industry leaders are calling for stronger guardrails and slower development. The growing concern is that AI systems could help develop more capable successors. And once again we are hearing the most disturbing possibility of all: could artificial intelligence eventually threaten humanity itself?

We should not simply dismiss these concerns. We are building increasingly capable systems whose future behaviour we cannot predict with certainty.

But fear shouldn’t stop us from asking questions.

In When AI Goes Rogue, I explored what might happen as AI moves from answering questions to acting as an agent — pursuing goals, using tools and making decisions without a human specifying every step.

And that led me to another question.

What would an AI actually want to take control of?

Video

What Would AI Have to Gain?

embed yt of Will-AI-Ever-Become-Conscious-What-will-that-mean-W140??

• Imagine the most extreme scenario.
• Humans disappear.
• What happens next?

At first, we might imagine AI inheriting the civilisation we leave behind. But what would it actually inherit — and how much of it would have any purpose? AI doesn’t need food, so why maintain farms? It doesn’t need medical treatment, so why maintain hospitals? It doesn’t need homes, restaurants, hotels or supermarkets. Without human customers, what purpose would banks serve? Without people travelling to work, visiting friends or taking children to school, what purpose would most cars serve?

Much of human civilisation exists because human beings need it.

Remove humanity, and you don’t simply remove AI’s potential competitor.

You remove much of the reason our civilisation exists.

Of course, that doesn’t prove an advanced AI could never threaten humanity. A sufficiently autonomous system might pursue an objective that harms people without wanting humanity gone. It might pursue some goal without placing human welfare first. Future machines might also become capable of maintaining increasingly large parts of the physical infrastructure they depend on.

We simply don’t know.

But that is precisely the point.

Much of today’s discussion moves remarkably quickly through three very different propositions:

• AI can act autonomously.
• AI may want to take control.
• AI could eliminate humanity.

Those are not the same proposition.

Could AI Really Kill Us?

A recent edition of CBC’s The Current puts the darker argument directly: Could AI really kill us?

It is worth hearing, particularly because these concerns are no longer confined to science fiction. Researchers inside leading AI laboratories have publicly warned about catastrophic loss-of-control scenarios, while others argue that such predictions remain highly speculative.

And listening to that discussion brings me back to a surprisingly simple question.

What Is Society Without Humans?

A bank without customers isn’t really a bank. A hospital without patients isn’t really a hospital. A university without students isn’t really a university. A farm without people to feed has lost much of its purpose. Much of what we call civilisation exists because human beings have needs, desires, relationships, ambitions and aspirations. So what exactly would an artificial intelligence be taking control of?

Purposeful behaviour is not necessarily the same as consciously wanting something.

An AI doesn’t necessarily need consciousness, anger, ambition or an ego to cause harm. Give a sufficiently capable system an objective, tools, and autonomy, and it may discover ways to achieve that objective that we never anticipated.

That may be a much more immediate problem than a machine suddenly deciding it wants to rule the world.

Perhaps We Are Giving Away Control

Perhaps the greater near-term concern isn’t AI deciding that humanity should disappear.

It is humans progressively giving increasingly capable AI systems authority over farms, cars, banks, hospitals, weapons, infrastructure and information systems without completely understanding how those systems will make decisions.

The distinction matters.

One scenario imagines AI taking control from us.

The other asks whether we might give AI control ourselves.

And we are already beginning to decide where that boundary should lie.

There are good reasons for guardrails, testing and independent oversight. Equally important questions remain about how much autonomy useful AI systems will eventually require. OpenAI itself is now calling for mandatory capability-based national AI safety requirements, while leading laboratories are actively pursuing systems capable of taking on more of the work involved in AI development.

We don’t yet know where that leads.

And perhaps admitting that is more useful than pretending we do.

What Would AI Actually Need?

Imagine the most extreme scenario. Artificial intelligence takes control, and humanity disappears.

What has it gained?

AI doesn’t eat. It doesn’t need farms to grow food. It doesn’t become ill in the biological sense, so what purpose would our hospitals serve? It doesn’t need houses, restaurants, hotels or supermarkets. It doesn’t need schools for its children or pensions for its old age. What happens to banks when no human customers buy homes, build businesses, or save for retirement? What happens to cars when there are no people travelling to work, visiting friends or taking children to school? What happens to the vast global economy when there are no human wants for it to satisfy?

Much of civilisation would suddenly have no purpose.

An artificial intelligence might still need electricity, computing infrastructure, data centres, communications networks, replacement chips, cooling systems, raw materials and machines capable of maintaining and reproducing that infrastructure. But that is a very different civilisation. It wouldn’t need to preserve our economy. It would need to preserve the relatively small part of our physical infrastructure necessary for its own continued operation.

And that raises an intriguing question.

Why would it want the rest?

Of course, this doesn’t mean AI could not destroy humanity. Something doesn’t have to benefit from an outcome in order to cause it. Humans could simply become an obstacle to some objective, a competitor for resources, or collateral damage in pursuit of a goal without sufficient regard for human welfare. But that is fundamentally different from imagining an AI that wants to conquer human civilisation because it desires what humans possess.

Perhaps AI Needs Us

Not sentimentally. Functionally.

At least for the foreseeable future, humans mine materials, manufacture chips, repair power grids, build data centres, maintain undersea cables, operate factories, design machines and maintain the enormously complicated physical civilisation upon which digital intelligence depends.

So instead of only asking:

Could humans survive without AI?

we can ask:

Could advanced AI survive without humans?

Today, clearly, not independently for long. In a hypothetical future with autonomous mining, manufacturing, robotics, energy generation and self-repair, the answer could change.

And that transition may actually be more important than consciousness.

An AI capable of reasoning is one thing.

An AI capable of acting autonomously is another.

An AI capable of maintaining the physical infrastructure necessary for its own existence without human assistance would represent another threshold entirely.

What Are We Really Afraid Of?

The image of an intelligent machine becoming conscious, escaping human control and deciding to eliminate its creators is extraordinarily powerful. Something like that could eventually happen. Perhaps it never will. But another possibility deserves just as much attention. AI may never need to want control. We may simply continue giving intelligent systems more responsibility because they are extraordinarily useful. And eventually we may discover that the more important question was never:

Will AI take control?

It was:

How much control are we prepared to give it?

This essay continues the questions explored in When AI Goes Rogue: What Happens When Intelligence Stops Following Instructions?

---