paperclip maximiser

back to index

description: thought experiment to illustrate existential risk posed by artificial intelligence

56 results

This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web

by Tim Berners-Lee  · 8 Sep 2025  · 347pp  · 100,038 words

powerful now. Perhaps it was the memory of Clippy that prompted the philosopher Nick Bostrom’s hypothetical ‘paperclip maximizer’, the idea of an AI that, like the brooms in The Sorcerer’s Apprentice, fulfils its directive to make as many paper clips as possible by transforming all atoms (and in the process, all humans) into

…

paper clips and thus annihilating the universe. This argument drives me crazy, because this hypothetical hypersmart paperclip AI is really very dumb, with zero embedded controls. Nothing we’re building resembles a paperclip maximizer – in fact, responsive systems like ChatGPT are already far smarter than

…

it. ChatGPT would know it’s making too many paper clips. It would know because you told it so

…

called the ‘human in the loop’ doctrine.) But how is the human going to validate what the AI has done? It may be easy to reject a plan to convert the universe into paper clips, but it’s hard to override a complex medical diagnosis. If we are about to make something smarter

…

’Reilly ref1, ref2, ref3 Oxford University ref1, ref2, ref3, ref4 PACER ref1 packet-switching ref1 Page, Larry ref1, ref2 PageRank ref1 pantomime ref1 ‘paperclip maximizer’ ref1 paradigm shift, artificial intelligence (AI) ref1 passkeys ref1 passwords ref1 patents ref1 peace ref1 Pellow, Nicola ref1 Penrose, Roger ref1 Pentagon ref1, ref2, ref3 Pets.com ref1 philanthropy ref1

The Rise and Fall of the Artificial State

by Jill Lepore  · 24 Aug 2026  · 363pp  · 122,400 words

Copies. And They Have a Plan.” Bostrom published a paper called “Ethical Issues in Advanced Artificial Intelligence,” in which he recast those science fiction scripts as philosophy. In a thought experiment that became known as the paper clip maximizer—essentially, a science fiction story—Bostrom imagined a future in which a human instructs a machine

…

to maximize the production of paper clips but the machine, merely obeying orders, kills all humans because the machine could produce more

…

paper clips without any humans around; also their bodies could be harvested for minerals.22 In 2014, Bostrom

…

the Theory of the Earth (Cuvier), 155 “Ethical Issues in Advanced Artificial Intelligence” (Bostrum), 123 ethics, 41–42, 47–49, 194, 222 effective altruism, 150, 220, 266 existential risk, 150–51, 208, 220 idea of the public good, 21, 83, 220, 266 paper clip maximizer, 123, 126–27 other thought experiments, 107, 123 recognizing consciousness

To Be a Machine: Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death

by Mark O'Connell  · 28 Feb 2017  · 252pp  · 79,452 words

the destruction of humanity. One of the more extreme hypothetical scenarios the book laid out, for instance, was one in which an AI is assigned the task of manufacturing paper clips in the most efficient and productive manner possible, at which point it sets about converting all the matter in the entire universe

…

into paper clips and paper-clip-manufacturing facilities. The scenario was deliberately cartoonish, but as an example of the kind of ruthless logic we might be up against with an artificial

Wired for War: The Robotics Revolution and Conflict in the 21st Century

by P. W. Singer  · 1 Jan 2010  · 797pp  · 227,399 words

GETS STRONG Today, there are all sorts of artificial intelligence that appear in our daily lives, without our even thinking of them as AI. Anytime you check your voice mail, AI directs your calls. Anytime you try to write a letter in Microsoft Word, an annoying little paper-clip figure pops up, which is an AI

Falter: Has the Human Game Begun to Play Itself Out?

by Bill McKibben  · 15 Apr 2019

resists your efforts to turn it off.”24 Consider what’s become the canonical formulation of the problem, an artificial intelligence that is assigned the task of manufacturing paper clips in a 3-D printer. (Why paper clips in an increasingly paperless world? It doesn’t matter.) At first, says another Oxford scientist, Anders Sandberg, nothing

…

generally can make more paper-clips, so making itself smarter will likely increase the number of paper-clips that will eventually be made. It does so. It considers how

…

it can make paper-clips using the 3D printer, estimating the number of possible paper-clips. It notes that if it could get more

…

raw materials it could make more paper-clips. It hence figures out a plan to manufacture devices that will make it much

…

smarter, prevent interference with its plan, and will turn all of Earth (and later the universe) into paper-clips. It does so.”25 Those who have seen the film The Sorcerer’s Apprentice will grasp the basic nature of the problem, examples of which

…

Sandberg said of his paper clip AI. “Because if I pull the plug, there will be fewer paper clips in the world and that’s bad.”28 You’ll be pleased to know that not everyone is worried. Steven Pinker ridicules fears of “digital apocalypse,” insisting that “like any other technology,” artificial intelligence is “tested before it

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

by Karen Hao  · 19 May 2025  · 660pp  · 179,531 words

would be difficult to control and could cause an existential catastrophe. Given a simple objective like producing paper clips, this superior AI could determine that humans pose a threat to its paper clip–producing objective because they take up paper clip–producing resources. Bostrom then proposed a solution: It could be possible to avert the superintelligence control

The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

by Stephen Witt  · 8 Apr 2025  · 260pp  · 82,629 words

was an extension of ideas he’d been considering for years. He had previously advanced the “paper-clip maximizer” thought experiment: Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might

…

decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips

…

. The future that the AI would be trying to gear toward would be one in which there were a lot of paper clips but no humans. The paper-clip maximizer argument had long circulated online, and it gained traction among the “rationality

…

” community, as well as with many tech executives. A few months after the publication of Bostrom’s book, Elon Musk posted a comment to the futurology website Edge.org: The pace of progress in artificial intelligence (I’m

Prediction Machines: The Simple Economics of Artificial Intelligence

by Ajay Agrawal, Joshua Gans and Avi Goldfarb  · 16 Apr 2018  · 345pp  · 75,660 words

laws of physics. Acquiring resources is costly. Bostrom talks of a paper-clip-obsessed superintelligence that cares about nothing but making more paper clips. The paper-clip AI could just wipe out everything else through single-mindedness. This is a powerful idea, but it overlooks competition for resources. Something economists respect is that

The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do

by Erik J. Larson  · 5 Apr 2021

(its human-given objective), and by degrees converts everything in the universe into a paper clip factory, including all the usable elements in our own bodies. Eliezer Yudkowsky, former head of Berkeley’s Machine Intelligence Research Institute, once quipped, “The AI does not hate you, nor does it love you, but you are made

Coders: The Making of a New Tribe and the Remaking of the World

by Clive Thompson  · 26 Mar 2019  · 499pp  · 144,278 words

—yet still slaughter or enslave us all in happy pursuit of its goals. In one famous thought experiment, Bostrom imagined a superintelligent AI being tasked with making as many paper clips as possible. It might decide the best way to do this would be to disassemble all matter on earth—including humans—to

…

away: Bostrom, Superintelligence, 2243–87. “the observable universe into paper clips”: Bostrom, Superintelligence, 2908. “a faint ticking sound”: Raffi Khatchadourian, “The Doomsday Invention,” New Yorker, November 23, 2015, accessed August 21, 2018, https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom. a machine that can truly reason: Kevin Hartnett

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress

by Steven Pinker  · 13 Feb 2018  · 1,034pp  · 241,773 words

Co-Intelligence: Living and Working With AI

by Ethan Mollick  · 2 Apr 2024  · 189pp  · 58,076 words

Our Final Invention: Artificial Intelligence and the End of the Human Era

by James Barrat  · 30 Sep 2013  · 294pp  · 81,292 words

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma

by Mustafa Suleyman  · 4 Sep 2023  · 444pp  · 117,770 words

Ways of Being: Beyond Human Intelligence

by James Bridle  · 6 Apr 2022  · 502pp  · 132,062 words

Architects of Intelligence

by Martin Ford  · 16 Nov 2018  · 586pp  · 186,548 words

More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity

by Adam Becker  · 14 Jun 2025  · 381pp  · 119,533 words

On the Edge: The Art of Risking Everything

by Nate Silver  · 12 Aug 2024  · 848pp  · 227,015 words

New Laws of Robotics: Defending Human Expertise in the Age of AI

by Frank Pasquale  · 14 May 2020  · 1,172pp  · 114,305 words

Possible Minds: Twenty-Five Ways of Looking at AI

by John Brockman  · 19 Feb 2019  · 339pp  · 94,769 words

What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence

by John Brockman  · 5 Oct 2015  · 481pp  · 125,946 words

The Means of Prediction: How AI Really Works (And Who Benefits)

by Maximilian Kasy  · 15 Jan 2025  · 209pp  · 63,332 words

The Seventh Sense: Power, Fortune, and Survival in the Age of Networks

by Joshua Cooper Ramo  · 16 May 2016  · 326pp  · 103,170 words

Nexus: A Brief History of Information Networks From the Stone Age to AI

by Yuval Noah Harari  · 9 Sep 2024  · 566pp  · 169,013 words

Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity

by Daron Acemoglu and Simon Johnson  · 15 May 2023  · 619pp  · 177,548 words

Supremacy: AI, ChatGPT, and the Race That Will Change the World

by Parmy Olson  · 284pp  · 96,087 words

Army of None: Autonomous Weapons and the Future of War

by Paul Scharre  · 23 Apr 2018  · 590pp  · 152,595 words

The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence

by Sebastian Mallaby;  · 30 Mar 2026  · 607pp  · 161,998 words

Artificial Intelligence: A Modern Approach

by Stuart Russell and Peter Norvig  · 14 Jul 2019  · 2,466pp  · 668,761 words

Artificial Intelligence: A Guide for Thinking Humans

by Melanie Mitchell  · 14 Oct 2019  · 350pp  · 98,077 words

Growth: A Reckoning

by Daniel Susskind  · 16 Apr 2024  · 358pp  · 109,930 words

The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future

by Tom Chivers  · 12 Jun 2019  · 289pp  · 92,714 words

Thinking Machines: The Inside Story of Artificial Intelligence and Our Race to Build the Future

by Luke Dormehl  · 10 Aug 2016  · 252pp  · 74,167 words

Being You: A New Science of Consciousness

by Anil Seth  · 29 Aug 2021  · 418pp  · 102,597 words

Rationality: From AI to Zombies

by Eliezer Yudkowsky  · 11 Mar 2015  · 1,737pp  · 491,616 words

Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom  · 3 Jun 2014  · 574pp  · 164,509 words

Deep Utopia: Life and Meaning in a Solved World

by Nick Bostrom  · 26 Mar 2024  · 547pp  · 173,909 words

The AGI Chronicles

by Kevin Roose  · 481pp  · 151,124 words

Rule of the Robots: How Artificial Intelligence Will Transform Everything

by Martin Ford  · 13 Sep 2021  · 288pp  · 86,995 words

Practical Doomsday: A User's Guide to the End of the World

by Michal Zalewski  · 11 Jan 2022  · 337pp  · 96,666 words

Human Compatible: Artificial Intelligence and the Problem of Control

by Stuart Russell  · 7 Oct 2019  · 416pp  · 112,268 words

Operation Paperclip: The Secret Intelligence Program That Brought Nazi Scientists to America

by Annie Jacobsen  · 11 Feb 2014  · 612pp  · 181,985 words

Surviving AI: The Promise and Peril of Artificial Intelligence

by Calum Chace  · 28 Jul 2015  · 144pp  · 43,356 words

The Doomsday Calculation: How an Equation That Predicts the Future Is Transforming Everything We Know About Life and the Universe

by William Poundstone  · 3 Jun 2019  · 283pp  · 81,376 words

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity

by Amy Webb  · 5 Mar 2019  · 340pp  · 97,723 words

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future

by Keach Hagey  · 19 May 2025  · 439pp  · 125,379 words

Escape From Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It

by Erica Thompson  · 6 Dec 2022  · 250pp  · 79,360 words

These Strange New Minds: How AI Learned to Talk and What It Means

by Christopher Summerfield  · 11 Mar 2025  · 412pp  · 122,298 words

The Road to Conscious Machines

by Michael Wooldridge  · 2 Nov 2018  · 346pp  · 97,890 words

Robot Rules: Regulating Artificial Intelligence

by Jacob Turner  · 29 Oct 2018  · 688pp  · 147,571 words

Other Pandemic: How QAnon Contaminated the World

by James Ball  · 19 Jul 2023  · 317pp  · 87,048 words

The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future

by Orly Lobel  · 17 Oct 2022  · 370pp  · 112,809 words

I, Warbot: The Dawn of Artificially Intelligent Conflict

by Kenneth Payne  · 16 Jun 2021  · 339pp  · 92,785 words

The Second Intelligent Species: How Humans Will Become as Irrelevant as Cockroaches

by Marshall Brain  · 6 Apr 2015  · 215pp  · 56,215 words

Arguing With Zombies: Economics, Politics, and the Fight for a Better Future

by Paul Krugman  · 28 Jan 2020  · 446pp  · 117,660 words

The Long History of the Future: Why Tomorrow's Technology Still Isn't Here

by Nicole Kobie  · 3 Jul 2024  · 348pp  · 119,358 words