description: thought experiment to illustrate existential risk posed by artificial intelligence
56 results
by Tim Berners-Lee · 8 Sep 2025 · 347pp · 100,038 words
powerful now. Perhaps it was the memory of Clippy that prompted the philosopher Nick Bostrom’s hypothetical ‘paperclip maximizer’, the idea of an AI that, like the brooms in The Sorcerer’s Apprentice, fulfils its directive to make as many paper clips as possible by transforming all atoms (and in the process, all humans) into
…
paper clips and thus annihilating the universe. This argument drives me crazy, because this hypothetical hypersmart paperclip AI is really very dumb, with zero embedded controls. Nothing we’re building resembles a paperclip maximizer – in fact, responsive systems like ChatGPT are already far smarter than
…
it. ChatGPT would know it’s making too many paper clips. It would know because you told it so
…
called the ‘human in the loop’ doctrine.) But how is the human going to validate what the AI has done? It may be easy to reject a plan to convert the universe into paper clips, but it’s hard to override a complex medical diagnosis. If we are about to make something smarter
…
’Reilly ref1, ref2, ref3 Oxford University ref1, ref2, ref3, ref4 PACER ref1 packet-switching ref1 Page, Larry ref1, ref2 PageRank ref1 pantomime ref1 ‘paperclip maximizer’ ref1 paradigm shift, artificial intelligence (AI) ref1 passkeys ref1 passwords ref1 patents ref1 peace ref1 Pellow, Nicola ref1 Penrose, Roger ref1 Pentagon ref1, ref2, ref3 Pets.com ref1 philanthropy ref1
by Jill Lepore · 24 Aug 2026 · 363pp · 122,400 words
Copies. And They Have a Plan.” Bostrom published a paper called “Ethical Issues in Advanced Artificial Intelligence,” in which he recast those science fiction scripts as philosophy. In a thought experiment that became known as the paper clip maximizer—essentially, a science fiction story—Bostrom imagined a future in which a human instructs a machine
…
to maximize the production of paper clips but the machine, merely obeying orders, kills all humans because the machine could produce more
…
paper clips without any humans around; also their bodies could be harvested for minerals.22 In 2014, Bostrom
…
the Theory of the Earth (Cuvier), 155 “Ethical Issues in Advanced Artificial Intelligence” (Bostrum), 123 ethics, 41–42, 47–49, 194, 222 effective altruism, 150, 220, 266 existential risk, 150–51, 208, 220 idea of the public good, 21, 83, 220, 266 paper clip maximizer, 123, 126–27 other thought experiments, 107, 123 recognizing consciousness
by Mark O'Connell · 28 Feb 2017 · 252pp · 79,452 words
the destruction of humanity. One of the more extreme hypothetical scenarios the book laid out, for instance, was one in which an AI is assigned the task of manufacturing paper clips in the most efficient and productive manner possible, at which point it sets about converting all the matter in the entire universe
…
into paper clips and paper-clip-manufacturing facilities. The scenario was deliberately cartoonish, but as an example of the kind of ruthless logic we might be up against with an artificial
by P. W. Singer · 1 Jan 2010 · 797pp · 227,399 words
GETS STRONG Today, there are all sorts of artificial intelligence that appear in our daily lives, without our even thinking of them as AI. Anytime you check your voice mail, AI directs your calls. Anytime you try to write a letter in Microsoft Word, an annoying little paper-clip figure pops up, which is an AI
by Bill McKibben · 15 Apr 2019
resists your efforts to turn it off.”24 Consider what’s become the canonical formulation of the problem, an artificial intelligence that is assigned the task of manufacturing paper clips in a 3-D printer. (Why paper clips in an increasingly paperless world? It doesn’t matter.) At first, says another Oxford scientist, Anders Sandberg, nothing
…
generally can make more paper-clips, so making itself smarter will likely increase the number of paper-clips that will eventually be made. It does so. It considers how
…
it can make paper-clips using the 3D printer, estimating the number of possible paper-clips. It notes that if it could get more
…
raw materials it could make more paper-clips. It hence figures out a plan to manufacture devices that will make it much
…
smarter, prevent interference with its plan, and will turn all of Earth (and later the universe) into paper-clips. It does so.”25 Those who have seen the film The Sorcerer’s Apprentice will grasp the basic nature of the problem, examples of which
…
Sandberg said of his paper clip AI. “Because if I pull the plug, there will be fewer paper clips in the world and that’s bad.”28 You’ll be pleased to know that not everyone is worried. Steven Pinker ridicules fears of “digital apocalypse,” insisting that “like any other technology,” artificial intelligence is “tested before it
by Karen Hao · 19 May 2025 · 660pp · 179,531 words
would be difficult to control and could cause an existential catastrophe. Given a simple objective like producing paper clips, this superior AI could determine that humans pose a threat to its paper clip–producing objective because they take up paper clip–producing resources. Bostrom then proposed a solution: It could be possible to avert the superintelligence control
by Stephen Witt · 8 Apr 2025 · 260pp · 82,629 words
was an extension of ideas he’d been considering for years. He had previously advanced the “paper-clip maximizer” thought experiment: Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might
…
decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips
…
. The future that the AI would be trying to gear toward would be one in which there were a lot of paper clips but no humans. The paper-clip maximizer argument had long circulated online, and it gained traction among the “rationality
…
” community, as well as with many tech executives. A few months after the publication of Bostrom’s book, Elon Musk posted a comment to the futurology website Edge.org: The pace of progress in artificial intelligence (I’m
by Ajay Agrawal, Joshua Gans and Avi Goldfarb · 16 Apr 2018 · 345pp · 75,660 words
laws of physics. Acquiring resources is costly. Bostrom talks of a paper-clip-obsessed superintelligence that cares about nothing but making more paper clips. The paper-clip AI could just wipe out everything else through single-mindedness. This is a powerful idea, but it overlooks competition for resources. Something economists respect is that
by Erik J. Larson · 5 Apr 2021
(its human-given objective), and by degrees converts everything in the universe into a paper clip factory, including all the usable elements in our own bodies. Eliezer Yudkowsky, former head of Berkeley’s Machine Intelligence Research Institute, once quipped, “The AI does not hate you, nor does it love you, but you are made
by Clive Thompson · 26 Mar 2019 · 499pp · 144,278 words
—yet still slaughter or enslave us all in happy pursuit of its goals. In one famous thought experiment, Bostrom imagined a superintelligent AI being tasked with making as many paper clips as possible. It might decide the best way to do this would be to disassemble all matter on earth—including humans—to
…
away: Bostrom, Superintelligence, 2243–87. “the observable universe into paper clips”: Bostrom, Superintelligence, 2908. “a faint ticking sound”: Raffi Khatchadourian, “The Doomsday Invention,” New Yorker, November 23, 2015, accessed August 21, 2018, https://www.newyorker.com/magazine/2015/11/23/doomsday-invention-artificial-intelligence-nick-bostrom. a machine that can truly reason: Kevin Hartnett
by Steven Pinker · 13 Feb 2018 · 1,034pp · 241,773 words
by Ethan Mollick · 2 Apr 2024 · 189pp · 58,076 words
by James Barrat · 30 Sep 2013 · 294pp · 81,292 words
by Mustafa Suleyman · 4 Sep 2023 · 444pp · 117,770 words
by James Bridle · 6 Apr 2022 · 502pp · 132,062 words
by Martin Ford · 16 Nov 2018 · 586pp · 186,548 words
by Adam Becker · 14 Jun 2025 · 381pp · 119,533 words
by Nate Silver · 12 Aug 2024 · 848pp · 227,015 words
by Frank Pasquale · 14 May 2020 · 1,172pp · 114,305 words
by John Brockman · 19 Feb 2019 · 339pp · 94,769 words
by John Brockman · 5 Oct 2015 · 481pp · 125,946 words
by Maximilian Kasy · 15 Jan 2025 · 209pp · 63,332 words
by Joshua Cooper Ramo · 16 May 2016 · 326pp · 103,170 words
by Yuval Noah Harari · 9 Sep 2024 · 566pp · 169,013 words
by Daron Acemoglu and Simon Johnson · 15 May 2023 · 619pp · 177,548 words
by Parmy Olson · 284pp · 96,087 words
by Paul Scharre · 23 Apr 2018 · 590pp · 152,595 words
by Sebastian Mallaby; · 30 Mar 2026 · 607pp · 161,998 words
by Stuart Russell and Peter Norvig · 14 Jul 2019 · 2,466pp · 668,761 words
by Melanie Mitchell · 14 Oct 2019 · 350pp · 98,077 words
by Daniel Susskind · 16 Apr 2024 · 358pp · 109,930 words
by Tom Chivers · 12 Jun 2019 · 289pp · 92,714 words
by Luke Dormehl · 10 Aug 2016 · 252pp · 74,167 words
by Anil Seth · 29 Aug 2021 · 418pp · 102,597 words
by Eliezer Yudkowsky · 11 Mar 2015 · 1,737pp · 491,616 words
by Nick Bostrom · 3 Jun 2014 · 574pp · 164,509 words
by Nick Bostrom · 26 Mar 2024 · 547pp · 173,909 words
by Kevin Roose · 481pp · 151,124 words
by Martin Ford · 13 Sep 2021 · 288pp · 86,995 words
by Michal Zalewski · 11 Jan 2022 · 337pp · 96,666 words
by Stuart Russell · 7 Oct 2019 · 416pp · 112,268 words
by Annie Jacobsen · 11 Feb 2014 · 612pp · 181,985 words
by Calum Chace · 28 Jul 2015 · 144pp · 43,356 words
by William Poundstone · 3 Jun 2019 · 283pp · 81,376 words
by Amy Webb · 5 Mar 2019 · 340pp · 97,723 words
by Keach Hagey · 19 May 2025 · 439pp · 125,379 words
by Erica Thompson · 6 Dec 2022 · 250pp · 79,360 words
by Christopher Summerfield · 11 Mar 2025 · 412pp · 122,298 words
by Michael Wooldridge · 2 Nov 2018 · 346pp · 97,890 words
by Jacob Turner · 29 Oct 2018 · 688pp · 147,571 words
by James Ball · 19 Jul 2023 · 317pp · 87,048 words
by Orly Lobel · 17 Oct 2022 · 370pp · 112,809 words
by Kenneth Payne · 16 Jun 2021 · 339pp · 92,785 words
by Marshall Brain · 6 Apr 2015 · 215pp · 56,215 words
by Paul Krugman · 28 Jan 2020 · 446pp · 117,660 words
by Nicole Kobie · 3 Jul 2024 · 348pp · 119,358 words