Chapter One: A Blast from the Future CHAPTER ONE A BLAST FROM THE FUTURE
In 1983, while researching an article about artificial intelligence that I was writing for an obscure journal called The Wilson Quarterly, I interviewed an obscure computer scientist named Geoffrey Hinton. Hinton advocated an approach to artificial intelligence that was outside the mainstream but that, as I put it in the article, “some tout as the new wave in AI.”

Four decades after my conversation with Hinton, I came across an article about him in The New York Times that said he was known in some circles as “The Godfather of AI.” Apparently the approach he championed had worked.

In particular: This approach—after much evolution via various innovations, some of which Hinton figured prominently in—had led to dramatic progress in “generative AI,” AI that creates images and sentences and other things traditionally created by human beings. In the months before the Times article came out, generative AI, most notably in the form of the chatbot ChatGPT, had taken the world by storm, wowing some people and freaking some people out.

Hinton, it turned out, was among the freaked out. The headline atop that New York Times story read “?‘The Godfather of A.I.’ Leaves Google and Warns of Danger Ahead.” AI had over the past few years been getting smarter much faster than he had anticipated, and he was worried about the consequences.

The specific dangers cited by Hinton in the Times article are by now familiar. AI could (a) take lots of people’s jobs; (b) make it harder to tell the real from the fake; (c) behave in unpredictably dangerous ways; and (d) be weaponized, in both literal and figurative senses, by bad actors.

And then there’s (e): AI could actually become a bad actor—at least, a bad actor from a human point of view. It might become so smart and powerful that it could sustain itself without human assistance. At which point it might decide it had no use for human beings.

Here’s how Hinton framed that prospect shortly after the New York Times story broke: It could turn out that “the good news is we figured out how to build beings that are immortal” and the bad news is that maybe “that immortality is not for us.”

With the publication of the Times piece, Hinton joined the group of people known as AI doomers. And in floating that last vision of doom—the demise of our species at the hands of brainy but heartless supermachines—he joined a select subset of AI doomers that you might call “sci-fi AI doomers.” They imagine AI becoming so smart and powerful as to be a kind of God—and, unfortunately, not a God that wishes us well. They imagine the human species surrendering its freedom, if not its very existence, to a ruthless superintelligence.

At the other end of the spectrum from the sci-fi doomers are people, sometimes called “accelerationists,” who want AI to move along as fast as possible. They envision a world full of AI-provided wonder, a world where disease and poverty and other human afflictions are in the process of disappearing.

There’s one thing that many of the more fervent accelerationists have in common with many of the more fervent sci-fi doomers (aside from fervor, I mean). And that’s a belief in the coming “singularity.” The singularity is the point where AI starts improving itself in a very hands-on way—amending its own algorithms to become smarter and smarter and hence better and better at amending its own algorithms. Once this “recursive self-improvement” kicks in, the rate of advance grows so rapidly that soon the term “rate of advance” doesn’t do it justice; there is an “intelligence explosion,” and life on Earth is forever transformed, for better or worse.

So which is it: better or worse? Who is right, the accelerationists or the doomers? The word “singularity,” actually, suggests that maybe there’s no way of knowing. The term comes from physics, where it is applied to black holes and denotes an impenetrably opaque boundary—a point beyond which existing theories lose their predictive value. Maybe technological change of a certain velocity could have such wildly transformative effects that there’s no telling what lies ahead.

Maybe. But both accelerationists and hard-core sci-fi doomers are undaunted; when they gaze into the singularity, they think they see the future’s likely contours.
WHICH AWE IS IN ORDER?
One way to get a finer feel for what they see is via the history of the word “awe.”

When the word came into use, more than a millennium ago, it meant, the Oxford English Dictionary tells us, “immediate and active fear; terror, dread.” But, the OED explains, “from its use in reference to the Divine Being,” the word gradually came to mean “dread mingled with veneration, reverential or respectful fear; the attitude of a mind subdued to profound reverence in the presence of supreme authority, moral greatness or sublimity, or mysterious sacredness.” By the mid-eighteenth century, the word meant “the feeling of solemn and reverential wonder, tinged with latent fear, inspired by what is terribly sublime and majestic in nature.” Today the word, as typically used, doesn’t carry even a tinge of latent fear. To stand in awe is to stand in wonder at the greatness or power or magnitude of something, period.

Sci-fi AI doomers stand near one end of this spectrum of meaning, the end dominated by fear and dread. Accelerationists stand at the other end, contemplating the power of AI with wonder and no trace of fear. And some people stand somewhere in between—maybe cautiously optimistic, maybe somewhat pessimistic, maybe right on the border between positive and negative expectation, but in any event convinced that something momentous and transformative is unfolding.

This book has four purposes:

1. I want to convince you that if you aren’t somewhere on the awe spectrum—if you don’t feel something of great magnitude and power approaching—you aren’t getting the picture. Before the artificial intelligence revolution is over, I believe, it will be the most abruptly dramatic transformation of human experience and human society in the history of our species.
2. I want to convince you that there’s a way to steer the human future toward the better end of the awe spectrum—to create a world where AI’s upside greatly outweighs its downside, a world where amazing things happen, a world where humans flourish—where they can find not just happiness but meaning and purpose, where they can live in freedom even amid awesomely smart and powerful machines.
3. But I also want to convince you that pulling this off—reconciling the coming transformation with our happiness and fulfillment and freedom—will require a big reorientation of human thought. Politicians and corporate leaders and other powerful people need to reckon with the magnitude of what’s coming and the specific nature of the challenge it poses. So do the rest of us, in part because so few of those powerful people seem inclined to do that reckoning.

Now here comes the weird part.

1. I want to convince you that, however cosmic the worldviews of the AI accelerationists and the sci-fi AI doomers may sound, there’s a sense in which these worldviews aren’t cosmic enough. In assessing the significance of the AI revolution, these people tend to put it in the context of the past century of technological change or the past several centuries of technological change. Some of them take a broader view, and locate the current moment in the whole history of technological change—which, construed broadly, encompasses more than a million years. And a few of them go even further than that. I’m with the “go even further” crowd, to put it mildly. I think we need to widen the lens by a few billion years. I think the coming of artificial intelligence marks a major threshold not just in the history of technology, and not just in the history of our species, but in the history of our planet.

Seeing AI in this context can help us envision, and shape, the future. We’re definitely entering a phase of accelerating change, and it may deserve a label as dramatic as “the singularity.” But one view I share with both the sci-fi doomers and the accelerationists is that this term’s connotation of opaqueness is misleading. The alternative paths that lie ahead are discernible, and the choices we need to make are clear.

Perhaps not surprisingly, given the magnitude of the coming transformation, these choices cover a lot of ground. The challenge we face is not just a political challenge—a matter of getting the laws and regulations right. And it’s not just a challenge of governance more broadly—though it is definitely that. It’s a challenge with a moral dimension, even what some would call a spiritual dimension.

The AI challenge will reach into our everyday lives. The decisions that each of us makes about how we use artificial intelligence, and how we think about it, are, inevitably, decisions about the role we will play in the collective human reckoning with the AI revolution. Happily, playing a constructive role in this reckoning can dovetail with using AI constructively in our own lives. Approaching the age of AI in a way that’s good for you can be good for the world.

I call this whole challenge—personal, moral, spiritual, political—The God Test, for several reasons.

One of them—the sheer cosmic scale of the perspective I’m viewing the AI revolution from—will get fleshed out in chapter 3. But first, in the chapter that follows, I’ll start to explore the question of why Geoffrey Hinton went from AI Evangelist to AI Cassandra. What exactly happened in artificial intelligence between 1983 and now that shocked even the person who is probably more responsible than anyone else for it happening? I’ll continue that exploration in chapters 4 and 5, explaining why, for all the progress that’s been made in artificial intelligence lately, we can be sure that there’s much more to come.

The next four chapters, in short, are meant to convince you that you belong somewhere on the awe spectrum. Later chapters will explain how we—humankind collectively—can determine which part of the awe spectrum our species eventually winds up on.

As for the other reasons I call the coming challenge The God Test—well, these will take a while to unfold fully, but I can say this much now:

Oddly, the phase of technological evolution we’re entering—a phase of ultrarapid advance that will make the futuristic feel familiar and make the familiar seem archaic—will also give new life to some ancient questions. In particular, this one: Is there some larger purpose unfolding on this planet?

In fact, this very old question is already drawing new life from the currents of technology. It is embedded in a question that has been popping up a lot in recent years: Are we living in a giant computer simulation? The question is often thrown out lightheartedly, as a kind of joke—a comment on how weird life sometimes seems. But the reason the simulation scenario gained currency in the first place is that a number of people in the tech world and its intellectual orbit, including some very smart and reflective people, started taking it seriously.

Why? Partly because of our growing power to digitally simulate complex worlds—and the recognition that there’s no obvious limit to how complex such worlds could be. But I think the simulation question has also been given prominence by something subtler: a sense that our ability to simulate complex worlds, and our ability to perform other technological wonders that are coming within reach, was in the cards all along.

In other words, there’s a clearer and clearer appreciation of how profoundly directional technological evolution is—a sense that our technologically driven history has a kind of storyline, and that we’re watching the story unfold, even move toward some kind of climax. There’s a sense that we’re being carried on a trajectory that, whether you find it terrifying or exhilarating or something in between, is too interesting not to be the product of some sort of programming, some sort of authorship, some sort of primordial design.

Indeed, the whole idea of the singularity—the idea taken seriously at the two ends of the awe spectrum—carries an unspoken sense that there’s a kind of narrative arrow in technological evolution, a dramatic impetus that has been pushing us toward something momentous, for better or worse.

I have no idea if we’re living in a simulation, and I’m not one to earnestly invoke the word “singularity.” But I do share the underlying sense that we’re part of a story that’s been unfolding for a long time and is now approaching a kind of climax. And I mean a long time: not just since the origin of human technology, but since the origin of life on Earth. And I mean a big climax: like, so big that, as it comes into view, it is casting our politics and our culture and our personal lives in a new light and giving them new dimensions of meaning.

The question of higher purpose is a question you can make sound new (What kind of superintelligent thing built the simulation, and what did it have in mind?) or make sound old (Is the purpose unfolding via this cosmic storyline a divine purpose, imbued by a God?). But however you frame the question, you wind up examining the same set of clues: Are we moving toward something better? Something worse? Is it within our power to shape that outcome? How?

That last question—the “How do we create a happy ending” question—can be put another way: What is technological evolution asking of us? What is our calling? Is it possible that the future, like gods of the past, is demanding that we move toward spiritual progress, toward a kind of enlightenment—that we become better versions of ourselves? And will we, as some of those gods mandated, pay a steep price, even an apocalyptic price, if we fail at that mission?

Among the reasons this book is called The God Test is that I believe the answer to those last two questions is yes. Whether or not we live in a universe imbued with purpose, we face the kind of epic, maybe even existential, challenge that you might expect from such a universe. Understanding the logic behind this challenge—seeing the choices technological evolution is now placing before us—is, I think, the first step toward a happy ending.