AI safety

The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence

by Sebastian Mallaby;  · 30 Mar 2026  · 607pp  · 161,998 words

money was not their main objective. Meanwhile, Zoufonoun had brushed aside Suleyman’s talk about AI governance. Over the coming years, Facebook’s indifference to AI safety would come to be well known. The DeepMind duo already sensed it. Zoufonoun reported back to Zuckerberg. DeepMind had a strong roster of AI scientists

…

structure that reduced our control over an asset that we were spending a great deal of money on.” Google’s leaders were open to pondering AI safety. But they preferred to do this pondering themselves, without outsiders second-guessing them. “As a lawyer, I’m actually not sure a company can sign

…

had nine hundred million users. The ethics and safety meeting promised to be fraught, however. Breakthroughs such as Vlad Mnih’s Atari agent had imbued AI safety discussions with a fresh urgency. The higher the existential stakes, the bigger the egos that clamored to take part, and the harder it became to

…

organization, which would in turn prevent nations from acquiring their own arsenals. Nothing had come of Oppenheimer’s proposal.[19] The more realistic path to AI safety, Hoffman argued, was to learn from multiparty democracy. The leaders of the field should back a handful of frontier research labs. Each would be animated

…

democracies, the shared values included fealty to the constitution and the rule of law; for AI development, they would involve a good-faith commitment to AI safety. From Hoffman’s perspective, Musk’s egocentric fury was extreme. But it illustrated a larger truth about the inevitability of AI competition. * * * • • • With all these

…

to ensure that powerful AI, when it emerged, would not fall under the sole sway of the parent company’s shareholders. For anyone concerned with AI safety, this saga remains relevant today. It shows what happens when, under unusually favorable conditions, a handful of leaders set out to create a control structure

…

21, Hassabis and Suleyman experienced a rude awakening. Google’s chief legal officer, David Drummond, showed up in London to meet them. Regarding DeepMind’s AI safety and governance objectives, “everyone is in agreement,” Drummond affirmed. But regarding the idea of a spin-out, there were “concerns,” he added. Drummond then elaborated

…

’s apparently contrary position, Pichai had the authority to do so. Even David Drummond, the lawyer and bad cop, had assured them that Google favored AI safety. With a bit more pushing, Hassabis and Suleyman reckoned, they could get what they wanted. * * * • • • Back in 2013, Hassabis and Suleyman had administered a particular

…

public had an interest in DeepMind breaking free from Google.[6] The claim would be that a spin-out served the public interest by bolstering AI safety. Surely Google cared too much about its reputation to challenge this proposition in court? Besides, even if deserting Google involved a legal risk, the threat

…

Geoffrey Irving. He was first and foremost a safety pioneer—later, he would quit DeepMind to become the chief scientist at the UK government’s AI Safety Institute. But, in a paradox that was still common at the time, he was simultaneously a leader in building the technology. A few years later

…

Amodei, the safety-minded AI scientist who went on to found the rival lab Anthropic; and Paul Christiano, the future scientific chief at the US AI Safety Institute. The way Irving and his colleagues saw things, you had to ask the hard questions: “What if we get to human-level systems? How

…

just a political or legal one.[2] The central problem, as Irving’s group saw it, was how to engineer an AlphaGo-type leap for AI safety. As in the case of Go, the rules of safety might be simple—do not harm people, do not deceive people. As in the case

…

safely under myriad conditions. AlphaGo had shown how an intelligent machine could master such complexity; Irving’s driving passion was to repeat this trick for AI safety. The idea was that, even if the machines of tomorrow operated far beyond the human capacity to understand, humans could design a set of rules

…

hard questions. Once the invention has happened and the scientists have lost control, they call on others to regulate it. It was almost as though AI safety was caught in a catch-22. Those with caution lacked power. Those with power lacked caution. “The problem is if we build machines that are

…

entitled “Planning for AGI and Beyond”: Remarkably, it spun the release of ChatGPT as a mark of OpenAI’s responsibility. Frequent product releases would promote AI safety, the argument went. They would allow the public to adjust to the technology, step by step; they would cause incipient threats to be identified before

…

rush. It’s kind of vulgar. “And so, going back to the letter, I think it did what we wanted. We made it clear that AI safety should be in scope to debate. After that letter, if someone said, ‘Oh, Yann thinks we don’t need a safety debate,’ the retort would

…

’s release, in October 2022, they rolled out a wide-ranging ban on the supply of advanced semiconductors to the country.[20] Tasked with tackling AI safety, Buchanan now set out to address the Hinton–Bengio challenge, leveraging the power of government. He began from a position of sympathy with the labs

…

the loop as the systems approached potential danger points.[24] Alongside the executive order, the White House also announced a plan to create a national AI safety institute, which would work on a voluntary basis with companies to do pre-deployment testing. The institute would also define what effective red-teaming looked

…

discussion with the senators, Buchanan flew to Britain; he was part of a delegation led by Vice President Kamala Harris. Prime Minister Sunak’s international AI safety conference, proposed by Hassabis six months before, was about to get started. On the eve of the gathering, DeepMind threw a party for the conference

…

models become stronger, they are entering a phase where simple reinforcement learning fails. And it fails in a way that was predicted by the original AI safety people. The danger of sub-goals. That turns out to matter.”[28] The question was what to do about this failure. Yoshua Bengio’s answer

…

an artificial intelligence counterpart to the International Atomic Energy Agency, with a responsibility to watch over national AI programs. He repeated his support for the AI safety institutes in the United States and Britain, as well as for the periodic international summits that extended the discussion begun at Bletchley. “There are many

…

, “given enough time, and the scientific method, and enough of our smartest people working on it.” Then he added a rider. The world could have AI safety if it embraced some version of his plan. But the plan required everybody to sign on. Responsible players doing the right thing couldn’t protect

…

Brought AI to Google, Facebook, and the World (Dutton, 2021), 152–56. On Musk’s contrasting fear of AI, Luke Nosek recalls Musk attending an AI safety conference organized by the Future of Life Institute in Puerto Rico in January 2015 and saying, “I am utterly saturated with fear.” Luke Nosek, author

…

the DealBook Summit, Musk recalled the speciesist interchange. “I’m like, OK, listen, this guy’s calling me a speciesist. He doesn’t care about AI safety. We’ve got to have some counterpoint here.” “DealBook Summit 2023 Elon Musk Interview,” Rev, transcript available at rev.com/transcripts/dealbook-summit-2023-elon

…

, David Silver suggests that Page was both a transhumanist, caring about the survival of intelligence rather than the survival of humans, but also focused on AI safety. “If you take account of the suffering on the way to a future of intelligent machines, then safety matters. That’s how the two thoughts

…

. Fuse Was Lit.” BACK TO NOTE REFERENCE 20 Mustafa Suleyman, author interview, May 5, 2024. BACK TO NOTE REFERENCE 21 Elaborating on his view of AI safety, Suleyman recalls, “I thought there would be two or three big AGI developers and it would be safer if there was the singleton developer that

…

in Chinchilla,” arXiv, July 18, 2023, arxiv.org/abs/2307.09458. BACK TO NOTE REFERENCE 28 Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras, “Scalable AI Safety via Doubly-Efficient Debate,” arXiv, November 27, 2023, arxiv.org/abs/2311.14125. BACK TO NOTE REFERENCE 29 Tripp Mickle et al., “Inside OpenAI’s

…

, 57–59 Intelligenesis, 56 International Mathematical Olympiad, 359, 378 introspection, 144 intuition, 142–43, 146–47, 305, 417n16 Iron Man, 314 Irving, Geoffrey, 431n25 on AI safety problems, 280–82 DeepMind hiring, 280, 283–84 Flamingo and, 295–96 Gato and, 296 Gopher and, 286, 288 human feedback problem of, 336–37

Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI

by Carissa Véliz  · 21 Apr 2026  · 503pp  · 129,255 words

the other a businessman, I wouldn’t have guessed Sunak was the head of state. That Sunak created this stunt on the occasion of the AI Safety Summit, an international conference on the governance of AI, made it much less likely that any of it could be taken seriously. Zoe Kleinman, the

…

about poverty and “got little interest.”[13] Lucky for the effective altruists, they stumbled upon a topic that they would turn into a gold mine: AI safety. Nick Bostrom, the former director of the now-defunct Future of Humanity Institute at Oxford, published Superintelligence in 2014.[14] In it, he warned of

…

, 243, 244, 247–49, 278, 286, 290 Aguilar, Miguel, 125 AI. See artificial intelligence (AI) Air Canada, 97, 101 AirDrop, 148 airplanes, 90, 254, 255 AI Safety Summit, 82 AI Snake Oil (Narayanan and Kapoor), 93 Alameda Research, 211, 212, 214–15 Alberti, Leon Battista, 28 Alexander the Great, 10, 17, 224

We Are as Gods: A Survival Guide for the Age of Abundance

by Peter H. Diamandis and Steven Kotler  · 13 Apr 2026  · 225pp  · 76,418 words

fear is a runaway intelligence explosion that leaves humans irrelevant. And he is not alone in this view. Leopold Aschenbrenner is another prophetic voice in AI safety—and he’s been even more vocal than Musk. Aschenbrenner started his career at Columbia University, graduating valedictorian at the age of nineteen, with majors

…

bad things,” he told The New York Times. A few weeks later, Hinton joined a who’s who of AI pioneers, signing the Center for AI Safety’s official statement about these dangers: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as

…

. Ask it to end poverty? Maybe it decides to standardize living conditions at a subsistence level. These aren’t just thought experiments; they’re what AI safety researchers call alignment problems. This is why companies like Anthropic are pioneering “constitutional AI” systems with built-in ethical constraints; DeepMind’s “recursive reward modeling

…

://www.nytimes.com/2023/05/01/technology/ai-google-chatbot-engineer-quits-hinton.html. A few weeks later, Hinton: “Statement on AI Risk,” Center for AI Safety, https://aistatement.com. A system that might replace: Orianna Rosa Royle, “Silicon Valley Billionaire Vinod Khosla Says AI Will Handle 80% of Work in 80

The Alignment Problem: Machine Learning and Human Values

by Brian Christian  · 5 Oct 2020  · 625pp  · 167,349 words

, where the episode is instantly deemed “hilarious” by all concerned. In its cartoonish, destructive slapstick, it certainly is. But for Amodei—who now leads the AI safety team at San Francisco research lab OpenAI—there is another, more sobering message. At some level, this is exactly what he’s worried about. The

…

understanding evolution, human motivation, and the delicacy of incentives, with implications for business and parenting alike. Part three takes us to the forefront of technical AI safety research, as we tour some of the best ideas currently going for how to align complex autonomous systems with norms and values too subtle or

…

he was perhaps one of the last alignment researchers who first had to live a kind of double life before being able to work in AI safety directly: working on more conventional problems to get his academic credentials, while figuring out a way to do the work he felt was truly important

…

I also, like, really wasn’t clear what I was going to do with my life.”32 Then he started reading about the idea of AI safety, through Nick Bostrom and Milan Ćirković’s book Global Catastrophic Risks, some discussions on the internet forum LessWrong, and a couple papers by Eliezer Yudkowsky

…

ask for some career advice. “I just randomly emailed him out of the blue, telling him, you know, I want to do a PhD in AI safety, can you give me some advice on where to go? And then I attached some work I’d done or something so he would hopefully

…

in which such an agent is liable to, as Leike put it, “misbehave drastically.”33 With his doctorate in hand, he prepared to enter the AI safety job market: that is, to join one of the three or four places in the world where one could make a career working on

…

AI safety. After a six-month stint at the Future of Humanity Institute in Oxford, he settled into a permanent role in London at DeepMind. “At the

…

look deeply into this question of how machines might learn complicated reward functions from humans. The project would ultimately become one of the most significant AI safety papers of 2017, remarkable not only for what it found but for what it represented: a marquee collaboration between the world’s two most active

…

AI safety research labs, and a tantalizing path forward for alignment research.34 Together they came up with a plan to implement the largest-scale test of

…

the same issue.” I remark that there’s a certain irony that his twenty-year-old idea ended up being the foundation for his current AI safety agenda. His idle thoughts on the walk to Safeway became, twenty years later, a plan to avert possible civilization-level catastrophe. “That was a complete

…

, not because it leads in bad directions, but because it leads in no directions at all.”38 Similar paradoxes and problems of definition haunt the AI safety research community. It would be good, for instance, for there to be a similar kind of precautionary principle: for systems to be designed to err

…

” or “impactful” behavior precise is a considerable challenge in itself.39 One of the first people to think about these issues in the context of AI safety was Stuart Armstrong, who works at Oxford University’s Future of Humanity Institute.40 Rather than trying to enumerate all of the things we don

…

theoretical conversation but also to create simple, game-like virtual worlds to illustrate these various problems and make the thought experiments concrete. They call these “AI safety gridworlds”—simple, Atari-like, two-dimensional (hence “grid”) environments in which new ideas and algorithms can be put to a practical test.46 The gridworld

…

instance, once a box is pushed into a corner, any states of the world that have that box anyplace else now become “unreachable.” In the AI safety gridworlds, agents looking out for stepwise relative reachability, alongside their normal goals and rewards, appear to behave rather conscientiously: agents don’t put boxes into

…

after it’s done whatever point-scoring actions the game incentivizes. Fascinatingly, the mandate to preserve attainable utility seems to foster good behavior in the AI safety gridworlds even when the auxiliary goals are generated at random.49 When Turner first elaborated the idea, on a library whiteboard at Oregon State, he

…

things out.”50 Over the course of 2018, he turned the math into working code and tossed his attainable-utility-preserving agent into DeepMind’s AI safety gridworlds. It did work. Acting to maximize each individual game’s rewards while at the same time preserving its future ability to satisfy four or

…

, we often take actions not only whose unintended effects are difficult to envision, but whose intended effects are difficult to envision. Publishing a paper on AI safety, for instance (or, for that matter, a book): it seems like a helpful thing to do, but who can say or foresee exactly how? I

…

ask Jan Leike, who coauthored the “AI Safety Gridworlds” paper with Krakovna, what he makes of the response so far to his and Krakovna’s gridworlds research. “I’ve been contacted by lots

…

of people, especially students, who get into the area and they’re like, ‘Oh, AI safety sounds cool. This is some open-source code I can just throw an agent at and play around with.’ And a lot of people have

…

. . . . I don’t know. It’s hard to know.” CORRIGIBILITY, DEFERENCE, AND COMPLIANCE One of the most chilling and prescient quotations in the field of AI safety comes in a famous 1960 article on the “Moral and Technical Consequences of Automation” by MIT’s Norbert Wiener: “If we use, to achieve our

…

and perfectly specified what we did and didn’t want the machine to do, then we had better be sure we can intervene. In the AI safety literature, this concept goes by the name of “corrigibility,” and—soberingly—it’s a whole lot more complicated than it seems.52 Almost any discussion

…

is so deep that, as we have seen in the last few chapters, much of the work being done in advanced AI applications and in AI safety in particular is about moving beyond systems that take in an explicit objective, and toward systems that attempt to imitate humans (in the case of

…

the enormity of the stakes, is not haste at all but the opposite. Bostrom’s essay came up more than once as I asked various AI safety researchers how they decided to commit their lives to that cause. “I found that argument pretty weird initially,” says Paul Christiano, “or it seemed off

…

trained on one set of examples finds itself operating in a different kind of environment, without necessarily realizing it. Amodei et al., “Concrete Problems in AI Safety.” gives an overview of this issue, which comes up in various subsequent chapters of this book. 37. Hardt, “How Big Data Is Unfair.” 38. Jacky

…

. Training systems which themselves are (or may become) optimizers of some “inner” reward function is a source of concern and of active research among contemporary AI-safety researchers. See Hubinger et al., “Risks from Learned Optimization in Advanced Machine Learning Systems.” 57. Andrew Barto, personal interview, May 9, 2018. 58. See Singh

…

”; see Ramakrishnan, Zhang, and Shah, “Perturbation Training for Human-Robot Teams.” 54. Murdoch, The Bell. 55. Some researchers at the intersection of cognitive science and AI safety, including the Future of Humanity Institute’s Owain Evans, are working on ways to do inverse reinforcement learning to take into account a person who

…

as Irreversibility.” See also Sunstein, “Irreversibility.” 38. Sunstein, “Beyond the Precautionary Principle.” See also Sunstein, Laws of Fear. 39. Amodei et al., “Concrete Problems in AI Safety,” has an excellent and broad discussion of “avoiding negative side effects” and “impact regularizers,” and Taylor et al., “Alignment for Advanced Machine Learning Systems,” also

…

Using Relative Reachability,” June 5, 2018, https://vkrakovna.wordpress.com/2018/06/05/measuring-and-avoiding-side-effects-using-relative-reachability/. 46. Leike et al., “AI Safety Gridworlds.” 47. Victoria Krakovna, personal interview, December 8, 2017. 48. The idea of stepwise baselines was suggested by Alexander Turner in https://www.alignmentforum.org

…

.” See Turner, “Optimal Farsighted Agents Tend to Seek Power.” For more on the notion of power in an AI safety context, including an information-theoretic account of “empowerment,” see Amodei et al., “Concrete Problems in AI Safety,” which, in turn, references Salge, Glackin, and Polani, “Empowerment: An Introduction,” and Mohamed and Rezende, “Variational Information

…

Turner, personal interview, July 11, 2019. 51. Wiener, “Some Moral and Technical Consequences of Automation.” 52. According to Paul Christiano, “corrigibility” as a tenet of AI safety began with the Machine Intelligence Research Institute’s Eliezer Yudkowsky, and the name itself came from Robert Miles. See Christiano’s “Corrigibility,” https://ai-alignment

…

command, see, e.g., Coman et al., “Social Attitudes of AI Rebellion,” and Aha and Coman, “The AI Rebellion.” 62. Smitha Milli, “Approaches to Achieving AI Safety” (interview), Melbourne, Australia, August 2017, https://www.youtube.com/watch?v=l82SQfrbdj4. 63. For more on corrigibility and model misspecification using this paradigm, see also

…

to the regularization method known as “early stopping”; see Yao, Rosasco, and Caponnetto, “On Early Stopping in Gradient Descent Learning.” For further discussion, in an AI safety context, about when “a metric which can be used to improve a system is used to such an extent that further optimization is ineffective or

…

?” An intriguing research direction in AI alignment involves developing machine-learning systems able to engage in debate with one another; see Irving, Christiano, and Amodei, “AI Safety via Debate.” 22. Jan Leike, “General Reinforcement Learning” (lecture), Colloquium Series on Robust and Beneficial AI 2016, Machine Intelligence Research Institute, Berkeley, California, June 9

…

.” Master’s thesis, University of Missouri–Kansas City, 1971. Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” arXiv Preprint arXiv:1606.06565, 2016. Ampère, André-Marie. Essai sur la philosophie des sciences; ou, Exposition analytique d’une classification naturelle de toutes les

…

. “Adversarial Examples Are Not Bugs, They Are Features.” In Advances in Neural Information Processing Systems, 125–36. 2019. Irving, Geoffrey, Paul Christiano, and Dario Amodei. “AI Safety via Debate.” arXiv Preprint arXiv:1805.00899, 2018. Jackson, Frank. “Procrastinate Revisited.” Pacific Philosophical Quarterly 95, no. 4 (2014): 634–47. Jackson, Frank, and Robert

…

Direction.” arXiv Preprint arXiv:1811.07871, 2018. Leike, Jan, Miljan Martic, Victoria Krakovna, Pedro A. Ortega, Tom Everitt, Andrew Lefrancq, Laurent Orseau, and Shane Legg. “AI Safety Gridworlds.” arXiv Preprint arXiv:1711.09883, 2017. Lenson, David. On Drugs. University of Minnesota Press, 1995. Letham, Benjamin, Cynthia Rudin, Tyler H. McCormick, David Madigan

…

racial bias Against Prediction (Harcourt), 78–79 age bias, 32, 396n7 AGI. See artificial general intelligence Agüera y Arcas, Blaise, 247 AI Now Institute, 396n9 AI safety artificial general intelligence delay risks and, 310 corrigibility, 295–302, 392–93n51 field growth, 12, 249–50, 263 gridworlds for, 292–93, 294, 295, 390n29

…

, 305 equiprobabiliorism, 303 equivant (company), 337n5 ergodicity assumption, 320 Ermon, Stefano, 324 ethics actualism vs. possibilism and, 239, 379n71 in reinforcement learning, 149 See also AI safety; fairness; moral uncertainty evaluation function. See value function Evans, Owain, 386–87n55 evolution, 170, 171–74, 368n56 expectations, 138–39, 197 See also TD (temporal

…

vs. statistical prediction, 91–94, 97–98 transparency and, 319, 397n19 word embedding and, 342n61 See also imitation; value alignment human-machine cooperation, 267–76 AI safety and, 268–69 aspiration and, 275–76, 386–87n55 CIRL, 267–68, 385nn40, 43–44 dangers of, 274–76 demonstration learning for, 271 feedback learning

…

shaping, 162 training data bias, 29 uncertainty, 288 value alignment, 266–67 UC Davis, 111 UC San Diego, 161 Ullman, Tomer, 345n94 uncertainty, 277–310 AI safety and, 291–92 Bayesian neural networks and, 283–86 confidence and, 281–82 corrigibility and, 295–302 deep learning brittleness, 279–81, 387n8 dissent and

…

of human values and, 247 inference and, 252–53, 269, 323–24, 385n39 inverse reward design for, 301–02 models and, 325–27 See also AI safety; human-machine cooperation; inverse reinforcement learning value functions, 138–39, 143, 238–39, 241–42, 245, 379n71 variable ratio reinforcement schedules, 153 Vaughan, Jenn Wortman

Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom  · 3 Jun 2014  · 574pp  · 164,509 words

be called for before we close this chapter. We distinguished two broad classes of methods for dealing with the agency problem at the heart of AI safety: capability control and motivation selection. Table 10 gives a summary. Table 10 Control methods Capability control Boxing methods The system is confined in such a

Artificial Intelligence: A Modern Approach

by Stuart Russell and Peter Norvig  · 14 Jul 2019  · 2,466pp  · 668,761 words

wide variety of games (Mnih et al., 2015). DeepMind in turn open-sourced several agent platforms, including the DeepMind Lab (Beattie et al., 2016), the AI Safety Gridworlds (Leike et al., 2017), the Unity game platform (Juliani et al., 2018), and the DM Control Suite (Tassa et al., 2018). Blizzard released the

…

these kinds of specification failures and take steps to avoid them. To help them do that, Krakovna was part of the team that released the AI Safety Gridworlds environments (Leike et al., 2017), which allows designers to test how well their agents perform. The moral is that we need to be very

…

requires explainable decisions for its battlefield systems, and has issued a call for research in the area (Gunning, 2016). AI safety: The book Artificial Intelligence Safety and Security (Yampolskiy, 2018) collects essays on AI safety, both recent and classic, going back to Bill Joy’s Why the Future Doesn’t Need Us (Joy, 2000

…

. OpenAI blog, blog.openai.com/ai-and-compute/. Amodei, D., Olah, C., Steinhardt, J., Christiano, P, Schulman, J., and Mané, D. (2016). Concrete problems in AI safety. arXiv:1606.06565. Andersen, S. K., Olesen, K. G., Jensen, F. V., and Jensen, F. (1989). HUGIN—A shell for building Bayesian belief universes for

…

AI Now Institute, 1046, 1059 Airborne Collision Avoidance System X (ACAS X), 588 aircraft carrier scheduling, 401 airport, driving to, 403 airport siting, 530, 535 AI safety, 1061 AI Safety Gridworlds, 873 AISB (Society for Artificial Intelligence and Simulation of Behaviour), 53 Aitken, S., 799, 1092 AI winter, 42, 45 Aizerman, M., 735, 1085

The Singularity Is Nearer: When We Merge with AI

by Ray Kurzweil  · 25 Jun 2024

training AI to imitate how humans draw inferences, so as to make it safer and more reliable when applying its knowledge in unfamiliar situations.[59] “AI safety via debate” uses competing AIs to point out flaws in each other’s ideas, allowing humans to judge issues too complex to properly evaluate unassisted

…

. There are encouraging signs as this book goes to press that major governments are taking the challenge seriously—like the Bletchley Declaration following the 2023 AI Safety Summit in the UK—but much will depend on how such initiatives are actually implemented.[78] One optimistic argument, which is based on the principle

…

9, 2021, https://www.alignmentforum.org/posts/JKj5Krff5oKMb8TjT/imitative-generalisation-aka-learning-the-prior-1. BACK TO NOTE REFERENCE 59 Geoffrey Irving and Dario Amodei, “AI Safety via Debate,” OpenAI, May 3, 2018, https://openai.com/blog/debate. BACK TO NOTE REFERENCE 60 For an insightful sequence of posts explaining iterated amplification

…

29, 2018, https://www.alignmentforum.org/s/EmDuGeRw749sD3GKd. BACK TO NOTE REFERENCE 61 For more details on the technical challenges of AI safety, see Dario Amodei et al., “Concrete Problems in AI Safety,” arXiv:1606.06565v2 [cs.AI], July 25, 2016, https://arxiv.org/pdf/1606.06565.pdf. BACK TO NOTE REFERENCE 62 To

…

TO NOTE REFERENCE 77 “The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023,” UK Government, November 1, 2023, https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit-1-2-november-2023. BACK TO NOTE REFERENCE 78 For

…

, 199, 201–3, 203, 219 vertical, 169, 171, 178, 179–83 Agüera Arcas, Blaise, 56 AI. See artificial intelligence AI effect, 63 AI Impacts, 61 AI safety via debate, 279 Alexander II of Russia, 162 algebra, 15 algorithms, 2, 56, 86–87 evolutionary, 20, 23, 25, 33 internet search, 45–46, 53

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

by Karen Hao  · 19 May 2025  · 660pp  · 179,531 words

real-world AI systems.” This was distinct from other AI-related challenges, he and his coauthors wrote, including privacy, security, fairness, and economic impact. AI “safety” in this framework, in other words, was about preventing rogue, misaligned AI—the root from which, as described by Nick Bostrom, superintelligence could become an

…

AI. Deborah Raji, an AI accountability researcher at the University of California, Berkeley, would come to champion the reexamination of the overwhelming focus of AI safety research on theoretical rogue AI and its possible existential risks to the detriment and de-prioritization of other real, evidence-based problems, coauthoring a 2020

…

cost.” Within OpenAI, various researchers, some of them among the small handful of women of color at the company, would press executives to expand their “AI safety” definition and include research on areas such as the discriminatory impacts of deep learning models. Executives were dismissive. “That’s not our role,” one

…

scale. * * * — Radford was given more of the company’s most precious resource: compute. His work dovetailed with a new project Amodei was overseeing in AI safety, in line with what Nick Bostrom’s Superintelligence had suggested. In 2017, one of Amodei’s teams began to explore a new technique for aligning

…

a meaningful conversation about human values and preferences.” Radford and Amodei joined forces. As Radford collected a bigger and more diverse dataset, Amodei and other AI safety researchers trained up progressively larger models. They set their sights on a final model with 1.5 billion parameters, or variables, at the time

…

the importance of building an ecosystem through those partnerships. Working with high-profile institutions would help foster more cooperation between industry and academia for addressing AI safety risks. It would also get broader buy-in into OpenAI’s efforts to shift research release norms and simultaneously help burnish the lab’s

…

as through a physical or virtual agent taking actions in the real world. In company documents, researchers weighed the merits of the different approaches, with AI safety staff at one point debating the virtues of the “pure language” hypothesis by drawing repugnant analogies to people with disabilities. The discussions revealed how

…

and human in society. Example, Helen Keller,” read the document under the heading “Some initial arguments for the centrality of language.” In the margins, AI safety researchers continued their arguments for and against “pure language” through threaded comments. “Also blind people are about as capable as sighted people,” wrote one researcher

…

-moment retreat in Hawai’i, growing especially close to Jakub Pachocki and Szymon Sidor, two Polish scientists who were roommates and best friends. Amodei’s AI safety teams, and the core members of the Nest team in particular, formed another contingent, bound together by their shared concern, in varying degrees, of

…

Brockman as they felt similarly sidelined by the dwindling of their compute resources, along with their visibility into the company’s core research. Amodei’s AI safety contingent, meanwhile, was also growing disquieted with some of Altman’s behaviors. Shortly after OpenAI’s Microsoft deal was inked, several of them were

…

like it commits us to an uncomfortable place.” This was against the backdrop of a growing paranoia over different issues across the company. Within the AI safety contingent, it centered on what they saw as strengthening evidence that powerful misaligned AI systems could lead to disastrous outcomes. One bizarre experience in

…

that maximally benefits humanity.” Though Altman never name-checked anyone, employees read between the lines. Sutskever was the face of Exploratory Research; Amodei and his AI safety contingent focused on extreme risks constituted Safety; Brockman was the champion of Startup. Soon after, the pandemic hit, and everyone began working remotely, making

…

middleman platform and then Scale AI. Where self-driving cars need data annotators to learn how to recognize street scenes and navigate roads, the AI safety researchers asked its RLHF workers to show GPT-3 how to respond helpfully to prompts and avoid harmful answers. The researchers first asked the workers

…

and sent them back to the earth so we could all see them. At the time, InstructGPT received limited external attention. But within OpenAI, the AI safety researchers had proved their point: RLHF did make large language models significantly more appealing as products. The company began using the technique—asking workers to

…

in 2011, which they later named Open Philanthropy. They began ramping up funding to the key issue areas that MacAskill had recommended—its grants toward AI safety research in particular were guided by the EA framework. Open Philanthropy became an independent organization in June 2017. More recently, a new tech billionaire

…

psychological toll of a global catastrophe had also left many people anxious and unmoored, searching for purpose. The growing membership in the AI safety community, which knit together EA-backed AI safety with other strains of catastrophic, existential, and risk-focused thinking, swelled Anthropic’s ranks just as it restocked OpenAI’s Safety clan

…

many people rapidly disaffiliated. But even without the label, the movement’s social networks, its values and lingo, and the prominence it secured for existential AI safety issues would persist. It would also give rise to a countervailing force: e/acc (pronounced “ee-ack”), or effective accelerationism. What began largely as

…

a joke to lampoon the EA movement would quickly enshrine its polar opposite spirit: Where EA and the broader AI safety community cultivated the most extreme perspectives about slowing down and even slamming the brakes on AI development, or, as in Amodei’s view, accelerating

…

the worst light possible. So many of the things that put OpenAI on the map and would bring it increasing commercial success had begun as AI safety projects: scaling laws, code generation, reinforcement learning from human feedback, the combination of these three into incredibly compelling large language and then multimodal models.

…

and break things” operation. In private conversations with Safety, Altman expressed sympathy for their perspective, agreeing that the company was not on track with its AI safety research and needed to invest in it more. In private conversations with Applied, he pressed them to keep going. During board meetings, he nodded

…

OpenAI’s representatives were Altman, Miles Brundage, the head of policy research, and Jan Leike, the head of alignment, which oversaw the continued development of AI safety techniques like RLHF. Both policy research and alignment had become the new emerging strongholds of the Safety clan. DSB created a formal governance structure for

…

slate of internet abuses, like fraud, cybercrime, and election interference. But wrapped up in the confusion was how their work related to that of the AI safety people within Leike’s alignment and Brundage’s policy research teams who often discussed unknowable, catastrophic doom. OpenAI labeled all of them as “safety”

…

teams, but they seemed to be speaking fundamentally different languages. Although the vocabulary of existential AI safety had, with its popularization through EA, become common parlance in the field, employees coming from traditional tech company backgrounds had never heard of AI timelines

…

told his mentor that AGI was imminent. Where before, Sutskever focused more on pushing OpenAI researchers to advance new capabilities, he shifted his attention to AI safety research with new urgency. He began his mantra “Feel the AGI” and urged people to prepare themselves for dramatic changes. “You’re suddenly going

…

First, create an agency that would develop and administer a licensing regime for models above a certain threshold of capabilities; second, create a set of AI safety standards for measuring “dangerous” capabilities; third, require independent audits on those standards to check for compliance. He later elaborated that capability thresholds could be approximated

…

many tied to the Doomer community, it pushed once again for a new licensing regime for AI models using compute thresholds, and the development of AI safety evaluations for dangerous capabilities including the ability to manipulate and persuade and the creation of novel biological weapon recipes. The fifty-one-page document also

…

a strategic alliance with Google and Anthropic to launch the Frontier Model Forum, a group for advancing relevant research and influencing the policy agenda on AI safety risks. It was a rare issue in which the interests of Doomers, Boomers, and profit-motivated corporates aligned: Keeping frontier models front and center

…

gateway into effective altruism and, in turn, to broader Doomer ideology. Yudkowksy had also cofounded the blog LessWrong, a central hub for AI safety researchers to foster community and propagate AI safety ideas, where he’d advocated with increasing alarmism to put a full pause on AI development as his p(doom) shot up

…

connecting with employees, information about the company’s happenings was instead filtering up to the independent directors through their own personal relationships from the broader AI safety and tech communities. They also relied on Altman himself as a conduit for keeping tabs on important information. What worried them with growing intensity

…

received reports from their own sources about various problems, including the company’s lack of preparation before and significant tumult after ChatGPT, the continued AI safety concerns surrounding GPT-4’s release, and the unprecedented pace with which OpenAI was sprinting to launch new products before it had resolved many of

…

disregard with which Microsoft had broken protocol and Altman had passed over it. OpenAI’s models were on a rapid advancement trajectory and, from an AI safety perspective, they believed, could soon pass a point where such a violation could result in potentially catastrophic, if not existential, consequences. In their view,

…

Altman’s laxness with Microsoft’s breach set a dangerous precedent for how he might treat AI safety processes around model releases once the stakes went up. Meanwhile, there were other concerning examples of Altman’s behavior. In March 2023, he had

…

benignly. Sutskever rolled his eyes. The independent board members, Toner explained, were looking to strengthen the board by adding a new director with a strong AI safety background. If he had different ideas, she would be glad to hear them. At this, Sutskever latched on. The board needed to be better

…

had popularized in Washington: cybersecurity threats, CBRN weapons, persuasion, and the evasion of human control. In the week running up to the launch, an AI safety researcher still left at the company wrote an impassioned memo: Upstream processes and the rushed release of Scallion had left the Preparedness team, headed by

…

held another all-hands meeting internally to address, all at once, the Johansson and equity crises, and any continued concerns over OpenAI’s commitment to AI safety. The mood was tense. The leadership team gave a series of quick explanations. The equity issue was “unacceptable” and being corrected as quickly as

…

new level of preparedness that Altman had previously mentioned, including leveling up the security of its research clusters, reorganizing to better focus on long-term AI safety research, and forming a new AGI readiness group, led by Aleksander Mądry, to improve coordination among leadership on advancing this objective. Altman then opened

…

with questions demanding greater clarity on the various allegations from Leike, Kokotajlo, and others, as well as Piper’s reporting over OpenAI’s disregard of AI safety and suppression of employee criticism. Meanwhile, the chaos among leadership and frustrations at Altman continued unabated. Combined with the ongoing Boomer-Doomer tussle, it

…

NOTE REFERENCE IN TEXT “the problem of accidents”: Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané, “Concrete Problems in AI Safety,” preprint, arXiv, July 25, 2016, 1–29, doi.org/10.48550/arXiv.1606.06565. GO TO NOTE REFERENCE IN TEXT By November 2024, it had

…

IN TEXT OpenAI executives internally: Screenshot of Slack announcement, May 14, 2024. GO TO NOTE REFERENCE IN TEXT “Being AGI ready”: All quotes about AI safety and the Superalignment team are pulled are from an audio recording of the all-hands meeting, May 15, 2024. GO TO NOTE REFERENCE IN TEXT

…

systems that are generally smarter than humans,’ ” Twitter (now X), June 4, 2024, x.com/DKokotajlo67142/status/1797994238468407380; and his posts and comments on the AI Safety forum LessWrong: “Daniel Kokotajlo,” LessWrong, accessed November 25, 2024, lesswrong.com/users/daniel-kokotajlo. The estimated value of his equity comes from Kevin Roose,

…

17, 353, 387–88 AI Index, 105 AI Insight Forums, 311 AI Now Institute, 308 Airbnb, 36, 41, 136, 150, 202, 367 air pollution, 286 AI safety, 55–58, 122–32, 301–12, 316–24, 419. See also data privacy; existential risks alignment and, 122–23, 124, 145–46, 316–18 effective

…

274–75, 277, 287 Mechanical Turk, 194 American Sign Language, 254 Amodei, Daniela, 55–56, 58, 144–45, 156, 157, 230 Amodei, Dario, 55–58 AI safety and risks, 55–56, 57–58, 87, 122–27, 131, 133, 134, 145–46, 147, 149–52, 156–57, 362 Altman’s firing, 366 at

…

60, 171–73, 275–78, 295, 309 Carnegie Mellon University, 97, 106, 172 Carr, Andrew, 385 Carter, Ashton, 43 CBRN weapons, 301, 380 Center for AI Safety, 322 Center for Security and Emerging Technology (CSET), 7, 307, 321, 357, 358 Center on Long-Term Risk, 388 Centre for the Governance of AI

…

dinner, 28, 46, 47, 48, 55 Rubik’s Cube, 71 Russia, 146 Ukraine war, 52, 191 Rwanda, 102, 260 S Safe Superintelligence, 405 safety. See AI safety Salinas, Alejandra, 290–91 Sama AI, 190–92, 206–13, 218–19, 242, 416 Santiago, Chile, 271–74, 285, 287–88, 295–96, 299–300

More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity

by Adam Becker  · 14 Jun 2025  · 381pp  · 119,533 words

they kill everyone. Yudkowsky has written prolifically on these subjects for over twenty years; an entire field has grown out of his work, known as “AI safety.” Despite his relatively low public profile, Yudkowsky’s ideas are deeply influential. Behind the multitude of tech CEOs and AI experts giving magazine interviews and

…

certainty of killing literally everyone,’” he writes. “If there are any survivors, you solved alignment.… It does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.”19 Both Yudkowsky and Bostrom are convinced that this is a problem humanity may

…

have the solutions. I wish I did.”75 Many of today’s prominent AI companies are deeply influenced by the rationalists as well, with dedicated “AI safety” teams working on solving the alignment problem. OpenAI used to broadcast their work on alignment quite publicly, though they shuttered their “superalignment” team in 2024

…

energy via perpetual motion machines were pointing at carbon emissions as a justification for their activities.”126 Gebru agrees. “People talk about building bridges between AI safety and AI ethics,” she says. “Just tell me why I need to be building bridges with these people. It just makes no sense to me

…

the feet of an all-powerful AI god whose arrival is just around the corner—unless the wrong one shows up first. Rationalist memes about AI safety depict unaligned superintelligences as horrors beyond our comprehension, like some kind of Lovecraftian beast from outside of time. Yudkowsky and others have explicitly invoked the

…

, Downing Street acknowledged for the first time the ‘existential risks’ now being faced.”52 Sunak’s government was in the midst of setting up an AI safety task force, with major input from effective altruists at all levels of UK AI policy.53 Meanwhile, the day after I visited Trajan House, a

…

text from a friend pointed me to more AI safety news. “Today many of the key people in AI came together to make a one-sentence statement on AI risk,” Ord tweeted that day.54

…

other societal-scale risks such as pandemics and nuclear war,” the statement read.55 The statement and its release was organized by the Center for AI Safety, yet another nonprofit funded by Open Philanthropy—and by a $6.5 million grant from the FTX Foundation, according to court documents in the FTX

…

, Ord explains, it wasn’t so much the Bing chatbot itself that concerned him. He sees it as a symptom of a cavalier attitude toward AI safety at Microsoft. “I was mainly alarmed that it was released, to be honest.” Ord claims his fears about insufficient guardrails on superintelligent AGI are shared

…

specific in a paper he wrote in 2021 with Hilary Greaves titled “The Case for Strong Longtermism.” Discussing the cost-effectiveness of funding work on “AI safety” (that is, preventing a superintelligent AI from destroying humanity), they cite the same surveys that Ord pointed to as justification for their estimate of the

…

-driven catastrophe (as bad or worse than human extinction) over the coming century.” Combining that estimate with a guess about the efficacy of investment in AI safety research and with their “reasonable estimate” of the number of future beings—which they claim is “at least” 1024—MacAskill and Greaves arrive at a

…

stunning conclusion. “Every $100 spent [on AI safety] has, on average, an impact as valuable as saving one trillion [lives]… far more than the near-future benefits of [malaria] bednet distribution.”66 For

…

a strong longtermist, investing in a Silicon Valley AI safety company is a more worthwhile humanitarian endeavor than saving lives in the tropics. This is not an isolated problem; it’s been part of longtermism

…

from the start. Nick Beckstead, an AI safety consultant, has a long history with longtermism and EA. He was CEO of the FTX Future Fund before FTX imploded in 2022. Before that, he

…

’s often, I think, very hard to pull an argument out of people here.”74 Longtermists, then, are making arguments with incredibly strong conclusions—funding AI safety research is trillions of times more cost-effective than preventing the spread of malaria! Saving a billion people today isn’t as good as a

…

to be running simulations of pivotal historical moments they wanted to learn more about, right? And what could be more pivotal than the birth of AI safety?”54 But Cegłowski thinks that the simulation hypothesis reflects a deep fear, rather than a deep yearning. “Fantasies of control come with a dark side

…

Little-Known AI Group That Got $660 Million,” Politico, March 26, 2024, www.politico.com/news/2024/03/25/a-665m-crypto-war-chest-roils-ai-safety-fight-00148621; Future of Life Institute, private communication. See also Future of Life Institute’s 2021 Form 990, available on ProPublica’s Nonprofit Explorer: https

…

Hinton et al., “Statement on AI Risk,” Center for AI Safety, May 30, 2023, www.safe.ai/statement-on-ai-risk. 56 “Center for AI Safety—General Support (2023),” Open Philanthropy, April 2023, www.openphilanthropy.org/grants/center-for-ai-safety-general-support-2023/; “Center for AI Safety—Philosophy Fellowship and NeurIPS Prizes,” Open Philanthropy, February 2023, www

…

.openphilanthropy.org/grants/center-for-ai-safety-philosophy-fellowship/; “Center for AI Safety—General Support (2022),” Open Philanthropy, November

…

2022, www.openphilanthropy.org/grants/center-for-ai-safety-general-support/; Jonathan Randles and Steven

…

Church, “FTX Is Probing $6.5 Million Paid to Leading Nonprofit Group on AI Safety,” Bloomberg, October 25, 2023, www.bloomberg.com

…

/news/articles/2023-10-25/ftx-probing-6-5-million-paid-to-leading-ai-safety-nonprofit. See also docket no. 3369 from “Ftx Trading Ltd., Case no. 22-11068,” November

…

practices is a big part of this too, especially on the academic side. See Shazeda Ahmed et al., “Field-Building and the Epistemic Culture of AI Safety,” First Monday 29, no. 4 (April 1, 2024), https://dx.doi.org/10.5210/fm.v29i4.13626. 59 Paul Krugman, “The Rich Are Crazier Than

Architects of Intelligence

by Martin Ford  · 16 Nov 2018  · 586pp  · 186,548 words

are ways to build an AI system that has much better properties, in terms of safety and control. MARTIN FORD: Related to these issues of AI safety and control, a lot of people worry about an arms race with other countries, especially China. Is that something we should take seriously, something we

…

that governments should jump into this at a larger scale? NICK BOSTROM: I think there could be more resources on AI safety. It’s not actually just us: DeepMind also has an AI safety group that we work with, but I do think more resources would be beneficial. There is already a lot more

…

research groups doing just that, including here at the FHI, where we have joint technical research seminars with DeepMind, also OpenAI has a number of AI safety researchers, and there are other groups like the Machine Intelligence Research Institute at Berkeley. I’m not sure whether there would have been as much

…

address, and one of the challenges in addressing it right now is that we don’t know what values are. Personally, I think that when AI safety researchers talk about value alignment, they have a very simplistic and maybe naive idea of what a value even is. In some of the work

…

work that we’re doing at AI2—and that other people are also doing—on natural language understanding, seems like a very valuable contribution to AI safety, at least as valuable as worrying about the alignment problem, which ultimately is just a technical problem having to do with reinforcement learning and objective

…

functions. So, I wouldn’t say that we’re underinvesting in being prepared for AI safety, and certainly some of the work that we’re doing at AI2 is actually implicitly a key investment in AI safety. MARTIN FORD: Any concluding thoughts? OREN ETZIONI: Well, there’s one other point I wanted

Warnings

by Richard A. Clarke  · 10 Apr 2017  · 428pp  · 121,717 words

Army of None: Autonomous Weapons and the Future of War

by Paul Scharre  · 23 Apr 2018  · 590pp  · 152,595 words

Robot Rules: Regulating Artificial Intelligence

by Jacob Turner  · 29 Oct 2018  · 688pp  · 147,571 words

Four Battlegrounds

by Paul Scharre  · 18 Jan 2023

Supremacy: AI, ChatGPT, and the Race That Will Change the World

by Parmy Olson  · 284pp  · 96,087 words

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future

by Keach Hagey  · 19 May 2025  · 439pp  · 125,379 words

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma

by Mustafa Suleyman  · 4 Sep 2023  · 444pp  · 117,770 words

The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future

by Tom Chivers  · 12 Jun 2019  · 289pp  · 92,714 words

Extremely Hardcore: Inside Elon Musk's Twitter

by Zoë Schiffer  · 13 Feb 2024  · 343pp  · 92,693 words

Possible Minds: Twenty-Five Ways of Looking at AI

by John Brockman  · 19 Feb 2019  · 339pp  · 94,769 words

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity

by Amy Webb  · 5 Mar 2019  · 340pp  · 97,723 words

Human Compatible: Artificial Intelligence and the Problem of Control

by Stuart Russell  · 7 Oct 2019  · 416pp  · 112,268 words

Elon Musk

by Walter Isaacson  · 11 Sep 2023  · 562pp  · 201,502 words

The Precipice: Existential Risk and the Future of Humanity

by Toby Ord  · 24 Mar 2020  · 513pp  · 152,381 words

If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

by Eliezer Yudkowsky and Nate Soares  · 15 Sep 2025  · 215pp  · 64,699 words

These Strange New Minds: How AI Learned to Talk and What It Means

by Christopher Summerfield  · 11 Mar 2025  · 412pp  · 122,298 words

The Means of Prediction: How AI Really Works (And Who Benefits)

by Maximilian Kasy  · 15 Jan 2025  · 209pp  · 63,332 words

Human + Machine: Reimagining Work in the Age of AI

by Paul R. Daugherty and H. James Wilson  · 15 Jan 2018  · 523pp  · 61,179 words

Code Dependent: Living in the Shadow of AI

by Madhumita Murgia  · 20 Mar 2024  · 336pp  · 91,806 words

What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence

by John Brockman  · 5 Oct 2015  · 481pp  · 125,946 words

To Be a Machine: Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death

by Mark O'Connell  · 28 Feb 2017  · 252pp  · 79,452 words

A Hacker's Mind: How the Powerful Bend Society's Rules, and How to Bend Them Back

by Bruce Schneier  · 7 Feb 2023  · 306pp  · 82,909 words

Artificial You: AI and the Future of Your Mind

by Susan Schneider  · 1 Oct 2019  · 331pp  · 47,993 words

On the Edge: The Art of Risking Everything

by Nate Silver  · 12 Aug 2024  · 848pp  · 227,015 words

Nexus: A Brief History of Information Networks From the Stone Age to AI

by Yuval Noah Harari  · 9 Sep 2024  · 566pp  · 169,013 words

Genius Makers: The Mavericks Who Brought A. I. To Google, Facebook, and the World

by Cade Metz  · 15 Mar 2021  · 414pp  · 109,622 words

AI Superpowers: China, Silicon Valley, and the New World Order

by Kai-Fu Lee  · 14 Sep 2018  · 307pp  · 88,180 words

What We Owe the Future: A Million-Year View

by William MacAskill  · 31 Aug 2022  · 451pp  · 125,201 words

Searches: Selfhood in the Digital Age

by Vauhini Vara  · 8 Apr 2025  · 301pp  · 105,209 words

Deep Utopia: Life and Meaning in a Solved World

by Nick Bostrom  · 26 Mar 2024  · 547pp  · 173,909 words

What If We Get It Right?: Visions of Climate Futures

by Ayana Elizabeth Johnson  · 17 Sep 2024  · 588pp  · 160,825 words

The Age of Extraction: How Tech Platforms Conquered the Economy and Threaten Our Future Prosperity

by Tim Wu  · 4 Nov 2025  · 246pp  · 65,143 words

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms: Proceedings of the Agi Workshop 2006

by Ben Goertzel and Pei Wang  · 1 Jan 2007  · 303pp  · 67,891 words

The Doomsday Calculation: How an Equation That Predicts the Future Is Transforming Everything We Know About Life and the Universe

by William Poundstone  · 3 Jun 2019  · 283pp  · 81,376 words

Against the Machine: On the Unmaking of Humanity

by Paul Kingsnorth  · 23 Sep 2025  · 388pp  · 110,920 words

The People vs Tech: How the Internet Is Killing Democracy (And How We Save It)

by Jamie Bartlett  · 4 Apr 2018  · 170pp  · 49,193 words

The Quiet Damage: QAnon and the Destruction of the American Family

by Jesselyn Cook  · 22 Jul 2024  · 321pp  · 95,778 words

The Dark Cloud: How the Digital World Is Costing the Earth

by Guillaume Pitron  · 14 Jun 2023  · 271pp  · 79,355 words

Superbloom: How Technologies of Connection Tear Us Apart

by Nicholas Carr  · 28 Jan 2025  · 231pp  · 85,135 words

The Wealth Ladder: Proven Strategies for Every Step of Your Financial Life

by Nick Maggiulli  · 22 Jul 2025

The Mysterious Mr. Nakamoto: A Fifteen-Year Quest to Unmask the Secret Genius Behind Crypto

by Benjamin Wallace  · 18 Mar 2025  · 431pp  · 116,274 words

Ways of Being: Beyond Human Intelligence

by James Bridle  · 6 Apr 2022  · 502pp  · 132,062 words

Co-Intelligence: Living and Working With AI

by Ethan Mollick  · 2 Apr 2024  · 189pp  · 58,076 words

This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web

by Tim Berners-Lee  · 8 Sep 2025  · 347pp  · 100,038 words

Applied Artificial Intelligence: A Handbook for Business Leaders

by Mariya Yao, Adelyn Zhou and Marlene Jia  · 1 Jun 2018  · 161pp  · 39,526 words