I want my ChatGPT

Strap yourselves in, DD is getting philosophical with today’s blog post.

But before he gives you his views on AI, the universe and everything …

Time has created an arresting cover for its latest issue.

I love it.

As for the comments that it generated on Facebook, my favourite was this one …

Oh Cory, Cory, Cory.

What is it with people blithely revealing their total, dazzling ignorance on social media? Does she actually believe a hose is going to protect her?

AI isn’t a machine that can be disabled by giving its circuits a vigorous hosing and voila you’re safe.

But then again, we are talking about a nation where people thought it was funny to wear these matching sweatshirts on a trip to Canada and smirk about it on social media …

As you can imagine, the LOL was on Lisa and she quickly regretted her life decisions when the internet piled on.

OK, now it’s over to DD. Strap yourselves in …

Yesterday’s guest blog post ended with the words “technology is a wonderful thing or is it?”  

That’s because it’s becoming clear that – as humans and human nature is involved – all technology including AI can be used for good or bad. Not sure if you saw that viral footage of a Ukrainian stall vendor being stalked and chased by a Russian drone? Horrifying.   

Back to the good. While cruising at 35,000 feet yesterday I was able to access the internet thanks to free and relatively fast inflight wifi. I had an old school Lonely Planet Mexico guidebook in my lap and anything that caught my fancy was put into ChatGPT, which then advised me on reputable tour operators and the optimal scheduling of various activities taking the weather and other variables into account.   

For example: “I recommend snorkelling on Sunday when the swell is almost non existent and the winds are light”. Or “Schedule the cave trip on Tuesday when the weather will be overcast”. Great practical advice given in less than a minute.  

I find it incredible how in just seconds I receive (on the whole) excellent advice, recommendations and costings that would have taken me hours of web research.  

At work I can point ChatGPT at scientific literature and get it to summarise topics that I am unfamiliar with. Last year when I was on a very tight deadline ChatGPT saved me by reading several complex scientific articles and drafting a properly referenced summary paragraph. While it needed some tweaking to match my style it saved me several hours I didn’t have. 

However, in counterpoint – back on the plane while dinner was being served – I started watching the latest Mission Impossible movie, with a plot revolving around an evil AI called “The Entity” that was quietly taking control of every nation’s nuclear weapons systems with plans to destroy humanity.   

Then I turned to reading the Sydney Morning Herald, ABC News and finally the Australian Financial Review. Each had various articles discussing – with varying degrees of alacrity – how AI could lead to the death of humanity quite quickly if better controls and boundaries were not put in place soon.   

Very not good – especially given Ms HouseGoesHome has scheduled some exciting holidays for us over the next two years – so I need to be alive for those.

Are these AI concerns overblown? Who to ask? ChatGPT and Claude of course and below are their perspectives. 

DD: What is your opinion on the claims that AI developments and lack of boundaries could threaten the survival of humanity? 

ChatGPT: I think the claim deserves to be taken seriously, but the strongest version — “uncontrolled AI is likely to make humanity extinct” — goes substantially beyond what we can currently demonstrate.

The key distinction is between a plausible catastrophic-risk mechanism and a reliable estimate that catastrophe will actually occur.

Why the concern isn’t just science fiction

The core argument does not require an AI to become conscious, malicious, or “hate humans.” Imagine a future system that is considerably better than humans at scientific research, software engineering, persuasion, strategic planning and perhaps designing other AI systems.

If such a system is given substantial autonomy and pursues objectives that are even subtly inconsistent with human intentions, several problems arise.

One is instrumental convergence. For many objectives, acquiring resources, avoiding being switched off, influencing people and preserving one’s ability to act can be useful intermediate strategies.

A sufficiently capable system therefore might behave in apparently self-preserving ways without possessing anything resembling a human desire for survival.

A second is the control/alignment problem. We are already remarkably poor at specifying exactly what we want from comparatively simple systems.

With a system operating much faster and more effectively than its supervisors, discovering that its objective was incorrectly specified after deployment could be too late.

And a third, which I regard as more immediately compelling, is human misuse. AI can lower the expertise and cost required for cyberattacks, biological research, automated propaganda, surveillance and potentially weapons development.

You don’t need a rogue superintelligence for AI to create catastrophic risks; humans using very capable AI systems badly may be sufficient.

Where I think some arguments become overstated

There is an enormous inferential jump between today’s models and an autonomous system capable of defeating essentially all attempts by humanity to control it.

Current AI systems remain dependent upon human-built infrastructure: electricity, data centres, networks, semiconductor fabrication, physical supply chains and people. They also make significant errors and don’t possess unlimited agency simply because they’re highly intelligent.

Nor do we know that increasing AI capability necessarily produces the kind of strategic, autonomous, self-improving superintelligence assumed in some extinction scenarios. That is a hypothesis about future technology rather than an established consequence of scaling AI.

Consequently, statements such as “AI extinction is inevitable” or assigning extremely precise probabilities to extinction should be treated sceptically.

The part that concerns me most: the race dynamic

This is where the “lack of boundaries” argument becomes particularly persuasive.

Suppose Company A discovers that giving an AI greater autonomy makes it substantially more commercially useful. Company B has an incentive to do the same. Governments face a similar problem because no major power wants another country to obtain a decisive AI advantage.

You can therefore end up with something analogous to an arms race: capability improves → economic/strategic value rises → competition accelerates deployment → safety margins shrink → increasingly consequential decisions are delegated to AI.

No participant needs to be irresponsible for the collective outcome to become dangerous.

This phenomenon is familiar from drug development, aviation and nuclear engineering: some risks can’t adequately be controlled simply by expecting individual organisations to behave responsibly. You eventually need agreed standards, independent verification and limits on certain activities.

How I would rank the risks

If we’re talking about the next couple of decades, I’d separate them roughly like this:

Risk	My assessment
AI-enabled fraud, cybercrime and misinformation	Very high / already occurring
Employment and economic disruption	High
Concentration of political/economic power	High
AI-assisted biological or cyber catastrophe	Meaningful and increasing
Autonomous weapons/escalation	Meaningful
Loss of control of extremely capable autonomous AI	Highly uncertain but potentially catastrophic
Literal human extinction from AI	Low-confidence probability, but not negligible enough to ignore

The final two are unusual risks because probability × consequence matters. Even if you believed there were only, say, a few-percent probability of genuinely uncontrollable AI emerging this century, extinction or permanent loss of human control is such an extraordinary consequence that spending substantial resources reducing that probability would be rational.

It’s similar to designing a pivotal Phase III trial: you don’t have to believe a severe adverse event will happen before establishing stopping rules and independent safety monitoring.

What “boundaries” should actually mean

I wouldn’t advocate trying to stop AI development altogether. Apart from losing enormous potential benefits in medicine, science and productivity, a global moratorium would probably be unenforceable.

More credible boundaries would involve mandatory testing before deployment of exceptionally capable systems; restrictions on autonomous access to weapons, critical infrastructure and certain biological capabilities; strong cybersecurity around frontier models; independent evaluations of deception, autonomous replication and cyber/biological capabilities; human authorization for genuinely consequential actions; and international agreements concerning the most powerful systems.

Importantly, regulation should probably be capability-based rather than regulating “AI” generally. A diagnostic assistant and an autonomous system capable of conducting sophisticated cyber operations clearly shouldn’t face identical restrictions.

My overall view

I would place myself between the two loudest camps.

I don’t think we have good evidence that AI extinction is inevitable, imminent, or even the most likely outcome. But I also think dismissing the possibility because today’s systems sometimes hallucinate or can’t reliably perform mundane tasks misunderstands the argument. The concern is primarily about what happens if capabilities continue improving rapidly while autonomy and access to the real world increase.

And there is an important asymmetry: if the pessimists are substantially wrong, excessive safety precautions impose economic costs. If the optimists are substantially wrong, the downside could be irreversible.

And for some balance here is the response from Claude at Anthropic to the same question: “My own honest take: I don’t think it’s wise for me to project confidence either way — genuine uncertainty here isn’t a dodge, it’s the accurate epistemic state.

“I’d say the risk isn’t zero and the ‘let’s figure out safety as we go’ approach has real critics for good reason, but I’m also wary of narratives that treat any speed bump as existential. If you want, I can go deeper into a specific angle — alignment research, regulatory proposals, or the ‘AI safety vs AI ethics’ split within the field.”

So with those two perspectives, I am really hoping that the AI tech bro’s and all the other relevant powers that be act decisively to sort this out because to steal a line from Dire Straits “I want my ChatGPTeeee” as – in my hands – it is incredibly useful.  

Song of the day: Dire Straits “Money for nothing”