The AI you are using for your business right now is not what keeps the world's most serious AI researchers awake at night.

That sentence is worth sitting with. Because most people, when they think about AI risk, are picturing the tools they already use. ChatGPT writing their emails, Claude answering questions, or an automation bot handling customer inquiries on WhatsApp. These tools are already changing how African businesses operate, and the concern is understandable: will AI take jobs? Will it replace my team? Will it disrupt my industry?

Those are real concerns, and they matter. But they are not what Eliezer Yudkowsky and Nate Soares, two of the longest-standing voices on machine intelligence, are warning about.

Their book, "If Anyone Builds It, Everyone Dies" (Little, Brown and Company, 2025), draws a clear and deliberate line between the AI that exists today and the AI that is coming. Understanding that line is one of the most important things any business owner, parent, or citizen can do in 2026.

What the Experts Actually Said

In early 2023, hundreds of AI scientists signed an open letter with a single sentence: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." [1, p. 11]

Among the signatories were Geoffrey Hinton, Nobel laureate in Physics, and Yoshua Bengio, who shared the Turing Award for the foundational work on deep learning that underlies every modern AI system. [1, p. 11]

Yudkowsky and Soares also signed, but they considered it "a severe understatement." [1, p. 11]

Here is what they said about the AI that existed at the time of signing: "It wasn't the AIs of 2023 that worried us or the other signatories. Nor are we worried about the AIs that exist as we write this, in early 2025. Today's AIs still feel shallow, in some deep sense that's hard to describe. They have limitations, such as an inability to form new long-term memories. These shortcomings have been enough to prevent those AIs from doing substantial scientific research or replacing all that many human jobs." [1, p. 11]

Read that again. The people warning about AI extinction risk are not talking about the tools you are using today. They are explicitly saying those tools are not the concern. Their concern is for what comes after.

A Documented Pattern of Jumps

To understand why experts are concerned about what comes next, you need to understand the pattern of how AI has actually developed.

Yudkowsky and Soares document five major capability jumps in the book's footnotes [1, p. 22]:

2012: AlexNet cracked open the problem of recognising objects in images, launching the deep learning era
2016: AlphaGo defeated the top human Go player, a milestone experts had predicted was decades away
2020: GPT-3 demonstrated that a purely predictive language model could perform surprisingly well across many tasks
2022: ChatGPT arrived as a widely useful, conversational AI that the public could access directly
2024: Reasoning models began solving mathematics, coding problems, and visual puzzles that required multi-step thinking

Each of these jumps was, to some degree, a surprise. Yudkowsky and Soares note that "most computer scientists in 2015 would have told you that ChatGPT-level artificial conversation wouldn't be in reach for another thirty or fifty years." [1, p. 12]

The pattern is not smooth, linear improvement. It is jumps. Periods of slower progress, then sudden breakthroughs that move the frontier further and faster than even experts predicted.

Nobody knows how many jumps are left before AI crosses the threshold the authors are warning about. "We don't know whether progress will peter out, causing these jumps to halt for a time until new methods and technologies are invented. We don't know how many jumps are left before AI becomes the extinction-level threat that the letter's signatories warned about. But history has shown time and time again that AI researchers invent new methods and overcome old obstacles." [1, p. 12]

What Machines Have That Humans Don't

Current AI is still described by Yudkowsky and Soares as "shallow" relative to human intelligence. OpenAI's o1, for example, is capable of reasoning across physics and biology in the same conversation without switching between two separate databases, something that would have been unimaginable in early AI systems. Yet the authors write that "o1 is less intelligent than even the humans who don't make big scientific breakthroughs. It is increasingly hard to pin down exactly what it's missing, but we nevertheless have the sense that, although o1 knows and remembers more than any single human, it is still in some important sense 'shallow' compared to a human twelve-year-old." [1, p. 29]

That shallowness, though, will not hold forever. The book outlines five structural advantages that machines have over biological brains, advantages that become increasingly relevant as AI systems grow more capable [1, pp. 29-31]:

Speed. Transistors can switch on and off billions of times per second. Even the fastest neurons spike only around a hundred times per second. The authors calculate that "human-quality thinking could be emulated 10,000 times faster on a machine." A mind operating at that speed would experience human communication as though each word takes roughly an hour to arrive. [1, p. 29]

Copy-and-paste ability. It takes twenty or more years to grow a single human and transfer into them a fraction of human knowledge. Albert Einstein's genius died with him. An AI system, by contrast, operates in a world "where genius could be replicated on demand." [1, p. 30]

Faster self-improvement. Human brain development ran into a biological bottleneck when baby heads became too large to fit through the birth canal. AI hardware and algorithms, by contrast, improve "much much much faster than the human species can evolve." [1, p. 30]

Larger memory. A modern data centre "can have 400 quadrillion bytes within five milliseconds' reach, over a thousand times more storage than a human brain." Modern AI systems are trained on a significant portion of all human knowledge, something no individual human could match. [1, p. 30]

Higher-quality thinking. Human minds are subject to well-documented systematic errors, including motivated skepticism: the tendency to look for arguments against conclusions you dislike but not against ones you favour. The authors ask readers to "imagine an AI that never did that." [1, p. 31]

None of this describes AI today. It describes the structural trajectory: what becomes possible as the capability jumps continue.

The Moment Nobody Can Predict

The authors are honest about what they do not know. They do not know when the next capability jump will happen. They do not know how many jumps separate current AI from superintelligent AI. They do not know exactly what threshold would make an AI system capable of the kind of autonomous, self-improving agency that concerns them.

"Nobody knows exactly when all hell will break loose. Nobody knows exactly how advanced an AI would need to be, in order to end up with the motive and capability to secretly copy itself onto the internet. Nobody knows what year or month some company will build a superhuman AI researcher that can create a new, more powerful generation of artificial intelligences." [1, p. 201]

What the authors do know is the incentive structure that is pushing the development forward regardless. They use a striking analogy: "Imagine that every competing AI company is climbing a ladder in the dark. At every rung but the top one, they get five times as much money: 10 billion, 50 billion, 250 billion, 1.25 trillion dollars. But if anyone reaches the top rung, the ladder explodes and kills everyone. Also, nobody knows where the ladder ends." [1, p. 201]

No company wants to stop climbing while its competitors do not. No country wants to regulate while other countries do not. The result is a race up a ladder nobody fully understands, toward a top rung nobody can locate.

There Is Still Time

The book does not end in despair. Yudkowsky and Soares draw a comparison with nuclear weapons: many people in the 1950s expected nuclear war between the major powers. Given the history of human conflict, pessimism was reasonable. Yet to date, nuclear war has not happened. "That's not because nuclear bombs turned out to be pure science fiction that could never happen in real life; it's because people have worked hard to build resilient systems around not starting nuclear wars." [1, p. 21]

The same kind of deliberate, coordinated effort could still be applied to AI. "Artificial superintelligence doesn't exist yet. Humanity could still decide not to build it." [1, p. 21]

The window is open. It will not stay open indefinitely.

What This Means for African Businesses Right Now

None of this means you should stop using AI tools. Yudkowsky and Soares are explicit: the tools available today are not the concern. The concern is the trajectory they are on.

For African businesses, two things are simultaneously true:

First: Current AI tools are genuinely valuable and underused. WhatsApp automation, AI-assisted content creation, image generation, customer service bots, business analytics: all of these are accessible, affordable, and practical right now. African businesses that build AI fluency in this window will have a real advantage over those that wait.

Second: The same tools that help you run your business today are funding the development of systems that could fundamentally reshape the world within a decade. That is not a reason to stop using them. It is a reason to pay attention to the larger story, to understand that the current AI moment is a chapter in a much longer arc, and to advocate for the kind of international coordination on AI safety that the world managed to build around nuclear weapons.

The authors put it plainly. Their goal is to rally "enough key decision-makers and regular people to take AI seriously." [1, p. 15] Ordinary people, business owners included, are part of that audience.

Where Uhuru AI Stands

The debate about AI risk is often framed as a binary: either you use AI aggressively and dismiss the concerns, or you fear AI and avoid it entirely. We reject that framing.

Our position at Uhuru AI is specific: the AI tools that exist today are not just adequate, they are extraordinary. Claude, ChatGPT, Gemini, and the automation infrastructure built on top of them represent more intelligence augmentation and business leverage than most organisations have even begun to use. A small business in Lesotho can now run marketing, customer service, content creation, and sales automation with tools that cost less per month than a single employee works in a day. That is not a gap that needs filling. That is a transformation still in progress.

There is no pressing business need, and no societal need, for AI that exceeds human intelligence. The race toward superintelligence is not being driven by a gap in what people actually require. It is being driven by the incentive structure Yudkowsky describes so precisely: a ladder in the dark where every rung pays more money and no company wants to stop climbing while competitors don't.

We believe the right call is to pause the push toward superintelligent AI until the alignment problem has a credible, verified solution. Not slow it down. Pause it. The world built functional restrictions around nuclear weapons without losing access to electricity. We can build functional restrictions around AI capability escalation without losing access to the tools already transforming business and daily life.

Current LLMs can write, reason, automate, design, analyse, and build. They can handle the marketing, customer service, operations, and growth functions of nearly every business on the planet. We do not need to add capabilities that nobody fully understands and nobody can safely control.

Use what exists. Build with it. And advocate loudly for a pause on what comes next.

That is where we stand.

Citations

[1] Yudkowsky, E. & Soares, N. "If Anyone Builds It, Everyone Dies: AI Would Kill Us All." Little, Brown and Company, September 2025.

p. 11: Current AI described as "shallow," open letter quote, Hinton and Bengio cited
p. 12: Capability jump history, 2015 expert prediction quote
p. 13: MIRI history, Yudkowsky background
p. 15: "Default outcome is lethal, but the situation is not hopeless"
p. 21: Nuclear war comparison, "Humanity could still decide not to build it"
p. 22 (footnotes): The five documented AI capability jumps (2012, 2016, 2020, 2022, 2024)
p. 29: o1 described as still "shallow" compared to a human twelve-year-old
pp. 29-31: Five structural machine advantages over biological brains
p. 201: The "ladder in the dark" analogy, "nobody knows when"