AI safety

back to index

description: research area on making artificial intelligence safe and beneficial

generative artificial intelligence

60 results

The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence

by Sebastian Mallaby;  · 30 Mar 2026  · 607pp  · 161,998 words

money was not their main objective. Meanwhile, Zoufonoun had brushed aside Suleyman’s talk about AI governance. Over the coming years, Facebook’s indifference to AI safety would come to be well known. The DeepMind duo already sensed it. Zoufonoun reported back to Zuckerberg. DeepMind had a strong roster of AI scientists

…

structure that reduced our control over an asset that we were spending a great deal of money on.” Google’s leaders were open to pondering AI safety. But they preferred to do this pondering themselves, without outsiders second-guessing them. “As a lawyer, I’m actually not sure a company can sign

…

had nine hundred million users. The ethics and safety meeting promised to be fraught, however. Breakthroughs such as Vlad Mnih’s Atari agent had imbued AI safety discussions with a fresh urgency. The higher the existential stakes, the bigger the egos that clamored to take part, and the harder it became to

…

organization, which would in turn prevent nations from acquiring their own arsenals. Nothing had come of Oppenheimer’s proposal.[19] The more realistic path to AI safety, Hoffman argued, was to learn from multiparty democracy. The leaders of the field should back a handful of frontier research labs. Each would be animated

…

democracies, the shared values included fealty to the constitution and the rule of law; for AI development, they would involve a good-faith commitment to AI safety. From Hoffman’s perspective, Musk’s egocentric fury was extreme. But it illustrated a larger truth about the inevitability of AI competition. * * * • • • With all these

…

to ensure that powerful AI, when it emerged, would not fall under the sole sway of the parent company’s shareholders. For anyone concerned with AI safety, this saga remains relevant today. It shows what happens when, under unusually favorable conditions, a handful of leaders set out to create a control structure

…

21, Hassabis and Suleyman experienced a rude awakening. Google’s chief legal officer, David Drummond, showed up in London to meet them. Regarding DeepMind’s AI safety and governance objectives, “everyone is in agreement,” Drummond affirmed. But regarding the idea of a spin-out, there were “concerns,” he added. Drummond then elaborated

…

’s apparently contrary position, Pichai had the authority to do so. Even David Drummond, the lawyer and bad cop, had assured them that Google favored AI safety. With a bit more pushing, Hassabis and Suleyman reckoned, they could get what they wanted. * * * • • • Back in 2013, Hassabis and Suleyman had administered a particular

…

public had an interest in DeepMind breaking free from Google.[6] The claim would be that a spin-out served the public interest by bolstering AI safety. Surely Google cared too much about its reputation to challenge this proposition in court? Besides, even if deserting Google involved a legal risk, the threat

…

Geoffrey Irving. He was first and foremost a safety pioneer—later, he would quit DeepMind to become the chief scientist at the UK government’s AI Safety Institute. But, in a paradox that was still common at the time, he was simultaneously a leader in building the technology. A few years later

…

Amodei, the safety-minded AI scientist who went on to found the rival lab Anthropic; and Paul Christiano, the future scientific chief at the US AI Safety Institute. The way Irving and his colleagues saw things, you had to ask the hard questions: “What if we get to human-level systems? How

…

just a political or legal one.[2] The central problem, as Irving’s group saw it, was how to engineer an AlphaGo-type leap for AI safety. As in the case of Go, the rules of safety might be simple—do not harm people, do not deceive people. As in the case

…

safely under myriad conditions. AlphaGo had shown how an intelligent machine could master such complexity; Irving’s driving passion was to repeat this trick for AI safety. The idea was that, even if the machines of tomorrow operated far beyond the human capacity to understand, humans could design a set of rules

…

hard questions. Once the invention has happened and the scientists have lost control, they call on others to regulate it. It was almost as though AI safety was caught in a catch-22. Those with caution lacked power. Those with power lacked caution. “The problem is if we build machines that are

…

entitled “Planning for AGI and Beyond”: Remarkably, it spun the release of ChatGPT as a mark of OpenAI’s responsibility. Frequent product releases would promote AI safety, the argument went. They would allow the public to adjust to the technology, step by step; they would cause incipient threats to be identified before

…

rush. It’s kind of vulgar. “And so, going back to the letter, I think it did what we wanted. We made it clear that AI safety should be in scope to debate. After that letter, if someone said, ‘Oh, Yann thinks we don’t need a safety debate,’ the retort would

…

’s release, in October 2022, they rolled out a wide-ranging ban on the supply of advanced semiconductors to the country.[20] Tasked with tackling AI safety, Buchanan now set out to address the Hinton–Bengio challenge, leveraging the power of government. He began from a position of sympathy with the labs

…

the loop as the systems approached potential danger points.[24] Alongside the executive order, the White House also announced a plan to create a national AI safety institute, which would work on a voluntary basis with companies to do pre-deployment testing. The institute would also define what effective red-teaming looked

…

discussion with the senators, Buchanan flew to Britain; he was part of a delegation led by Vice President Kamala Harris. Prime Minister Sunak’s international AI safety conference, proposed by Hassabis six months before, was about to get started. On the eve of the gathering, DeepMind threw a party for the conference

…

models become stronger, they are entering a phase where simple reinforcement learning fails. And it fails in a way that was predicted by the original AI safety people. The danger of sub-goals. That turns out to matter.”[28] The question was what to do about this failure. Yoshua Bengio’s answer

…

an artificial intelligence counterpart to the International Atomic Energy Agency, with a responsibility to watch over national AI programs. He repeated his support for the AI safety institutes in the United States and Britain, as well as for the periodic international summits that extended the discussion begun at Bletchley. “There are many

…

, “given enough time, and the scientific method, and enough of our smartest people working on it.” Then he added a rider. The world could have AI safety if it embraced some version of his plan. But the plan required everybody to sign on. Responsible players doing the right thing couldn’t protect

…

Brought AI to Google, Facebook, and the World (Dutton, 2021), 152–56. On Musk’s contrasting fear of AI, Luke Nosek recalls Musk attending an AI safety conference organized by the Future of Life Institute in Puerto Rico in January 2015 and saying, “I am utterly saturated with fear.” Luke Nosek, author

…

the DealBook Summit, Musk recalled the speciesist interchange. “I’m like, OK, listen, this guy’s calling me a speciesist. He doesn’t care about AI safety. We’ve got to have some counterpoint here.” “DealBook Summit 2023 Elon Musk Interview,” Rev, transcript available at rev.com/transcripts/dealbook-summit-2023-elon

…

, David Silver suggests that Page was both a transhumanist, caring about the survival of intelligence rather than the survival of humans, but also focused on AI safety. “If you take account of the suffering on the way to a future of intelligent machines, then safety matters. That’s how the two thoughts

…

. Fuse Was Lit.” BACK TO NOTE REFERENCE 20 Mustafa Suleyman, author interview, May 5, 2024. BACK TO NOTE REFERENCE 21 Elaborating on his view of AI safety, Suleyman recalls, “I thought there would be two or three big AGI developers and it would be safer if there was the singleton developer that

…

in Chinchilla,” arXiv, July 18, 2023, arxiv.org/abs/2307.09458. BACK TO NOTE REFERENCE 28 Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras, “Scalable AI Safety via Doubly-Efficient Debate,” arXiv, November 27, 2023, arxiv.org/abs/2311.14125. BACK TO NOTE REFERENCE 29 Tripp Mickle et al., “Inside OpenAI’s

…

, 57–59 Intelligenesis, 56 International Mathematical Olympiad, 359, 378 introspection, 144 intuition, 142–43, 146–47, 305, 417n16 Iron Man, 314 Irving, Geoffrey, 431n25 on AI safety problems, 280–82 DeepMind hiring, 280, 283–84 Flamingo and, 295–96 Gato and, 296 Gopher and, 286, 288 human feedback problem of, 336–37

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma

by Mustafa Suleyman  · 4 Sep 2023  · 444pp  · 117,770 words

maybe even the contents of the entire cosmos into paper clips. Start following chains of logic like this and myriad sequences of unnerving events unspool. AI safety researchers worry (correctly) that should something like an AGI be created, humanity would no longer control its own destiny. For the first time, we would

…

most software businesses for decades. It’s worth remembering just how safe years of effort have made many existing technologies—and building on it. Frontier AI safety research is still an undeveloped, nascent field focusing on keeping ever more autonomous systems from superseding our ability to understand or control them. I see

…

Weapons Convention, has a budget of just $1.4 million and only four full-time employees—fewer than the average McDonald’s. The number of AI safety researchers is still minuscule: up from around a hundred at top labs worldwide in 2021 to three or four hundred in 2022. Given there are

…

’s a clear must-do here: encourage, incentivize, and directly fund much more work in this area. It’s time for an Apollo program on AI safety and biosafety. Hundreds of thousands should be working on it. Concretely, a good proposal for legislation would be to require that a fixed portion—say

…

stakeholders need to be included—senior leaders at Alphabet and elsewhere started having these conversations on a regular basis. Across tech companies the kinds of AI safety discussions that felt fringe a decade ago are now becoming routine. The need to balance profits with a positive contribution and cutting-edge safety is

…

conference in Puerto Rico in 2015 that aimed to do something similar for AI. With a mixed group, it wanted to raise the profile of AI safety, start building a culture of caution, and sketch real answers. We met again in 2017, at the symbolic venue of Asilomar, to draft a set

…

bioweapons Toby Ord, The Precipice: Existential Risk and the Future of Humanity (London: Bloomsbury, 2020), 57. GO TO NOTE REFERENCE IN TEXT The number of AI safety researchers Benaich and Hogarth, State of AI Report 2022. GO TO NOTE REFERENCE IN TEXT Given there are around For an estimate the number of

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

by Karen Hao  · 19 May 2025  · 660pp  · 179,531 words

real-world AI systems.” This was distinct from other AI-related challenges, he and his coauthors wrote, including privacy, security, fairness, and economic impact. AI “safety” in this framework, in other words, was about preventing rogue, misaligned AI—the root from which, as described by Nick Bostrom, superintelligence could become an

…

AI. Deborah Raji, an AI accountability researcher at the University of California, Berkeley, would come to champion the reexamination of the overwhelming focus of AI safety research on theoretical rogue AI and its possible existential risks to the detriment and de-prioritization of other real, evidence-based problems, coauthoring a 2020

…

cost.” Within OpenAI, various researchers, some of them among the small handful of women of color at the company, would press executives to expand their “AI safety” definition and include research on areas such as the discriminatory impacts of deep learning models. Executives were dismissive. “That’s not our role,” one

…

scale. * * * — Radford was given more of the company’s most precious resource: compute. His work dovetailed with a new project Amodei was overseeing in AI safety, in line with what Nick Bostrom’s Superintelligence had suggested. In 2017, one of Amodei’s teams began to explore a new technique for aligning

…

a meaningful conversation about human values and preferences.” Radford and Amodei joined forces. As Radford collected a bigger and more diverse dataset, Amodei and other AI safety researchers trained up progressively larger models. They set their sights on a final model with 1.5 billion parameters, or variables, at the time

…

the importance of building an ecosystem through those partnerships. Working with high-profile institutions would help foster more cooperation between industry and academia for addressing AI safety risks. It would also get broader buy-in into OpenAI’s efforts to shift research release norms and simultaneously help burnish the lab’s

…

as through a physical or virtual agent taking actions in the real world. In company documents, researchers weighed the merits of the different approaches, with AI safety staff at one point debating the virtues of the “pure language” hypothesis by drawing repugnant analogies to people with disabilities. The discussions revealed how

…

and human in society. Example, Helen Keller,” read the document under the heading “Some initial arguments for the centrality of language.” In the margins, AI safety researchers continued their arguments for and against “pure language” through threaded comments. “Also blind people are about as capable as sighted people,” wrote one researcher

…

-moment retreat in Hawai’i, growing especially close to Jakub Pachocki and Szymon Sidor, two Polish scientists who were roommates and best friends. Amodei’s AI safety teams, and the core members of the Nest team in particular, formed another contingent, bound together by their shared concern, in varying degrees, of

…

Brockman as they felt similarly sidelined by the dwindling of their compute resources, along with their visibility into the company’s core research. Amodei’s AI safety contingent, meanwhile, was also growing disquieted with some of Altman’s behaviors. Shortly after OpenAI’s Microsoft deal was inked, several of them were

…

like it commits us to an uncomfortable place.” This was against the backdrop of a growing paranoia over different issues across the company. Within the AI safety contingent, it centered on what they saw as strengthening evidence that powerful misaligned AI systems could lead to disastrous outcomes. One bizarre experience in

…

that maximally benefits humanity.” Though Altman never name-checked anyone, employees read between the lines. Sutskever was the face of Exploratory Research; Amodei and his AI safety contingent focused on extreme risks constituted Safety; Brockman was the champion of Startup. Soon after, the pandemic hit, and everyone began working remotely, making

…

middleman platform and then Scale AI. Where self-driving cars need data annotators to learn how to recognize street scenes and navigate roads, the AI safety researchers asked its RLHF workers to show GPT-3 how to respond helpfully to prompts and avoid harmful answers. The researchers first asked the workers

…

and sent them back to the earth so we could all see them. At the time, InstructGPT received limited external attention. But within OpenAI, the AI safety researchers had proved their point: RLHF did make large language models significantly more appealing as products. The company began using the technique—asking workers to

…

in 2011, which they later named Open Philanthropy. They began ramping up funding to the key issue areas that MacAskill had recommended—its grants toward AI safety research in particular were guided by the EA framework. Open Philanthropy became an independent organization in June 2017. More recently, a new tech billionaire

…

psychological toll of a global catastrophe had also left many people anxious and unmoored, searching for purpose. The growing membership in the AI safety community, which knit together EA-backed AI safety with other strains of catastrophic, existential, and risk-focused thinking, swelled Anthropic’s ranks just as it restocked OpenAI’s Safety clan

…

many people rapidly disaffiliated. But even without the label, the movement’s social networks, its values and lingo, and the prominence it secured for existential AI safety issues would persist. It would also give rise to a countervailing force: e/acc (pronounced “ee-ack”), or effective accelerationism. What began largely as

…

a joke to lampoon the EA movement would quickly enshrine its polar opposite spirit: Where EA and the broader AI safety community cultivated the most extreme perspectives about slowing down and even slamming the brakes on AI development, or, as in Amodei’s view, accelerating

…

the worst light possible. So many of the things that put OpenAI on the map and would bring it increasing commercial success had begun as AI safety projects: scaling laws, code generation, reinforcement learning from human feedback, the combination of these three into incredibly compelling large language and then multimodal models.

…

and break things” operation. In private conversations with Safety, Altman expressed sympathy for their perspective, agreeing that the company was not on track with its AI safety research and needed to invest in it more. In private conversations with Applied, he pressed them to keep going. During board meetings, he nodded

…

OpenAI’s representatives were Altman, Miles Brundage, the head of policy research, and Jan Leike, the head of alignment, which oversaw the continued development of AI safety techniques like RLHF. Both policy research and alignment had become the new emerging strongholds of the Safety clan. DSB created a formal governance structure for

…

slate of internet abuses, like fraud, cybercrime, and election interference. But wrapped up in the confusion was how their work related to that of the AI safety people within Leike’s alignment and Brundage’s policy research teams who often discussed unknowable, catastrophic doom. OpenAI labeled all of them as “safety”

…

teams, but they seemed to be speaking fundamentally different languages. Although the vocabulary of existential AI safety had, with its popularization through EA, become common parlance in the field, employees coming from traditional tech company backgrounds had never heard of AI timelines

…

told his mentor that AGI was imminent. Where before, Sutskever focused more on pushing OpenAI researchers to advance new capabilities, he shifted his attention to AI safety research with new urgency. He began his mantra “Feel the AGI” and urged people to prepare themselves for dramatic changes. “You’re suddenly going

…

First, create an agency that would develop and administer a licensing regime for models above a certain threshold of capabilities; second, create a set of AI safety standards for measuring “dangerous” capabilities; third, require independent audits on those standards to check for compliance. He later elaborated that capability thresholds could be approximated

…

many tied to the Doomer community, it pushed once again for a new licensing regime for AI models using compute thresholds, and the development of AI safety evaluations for dangerous capabilities including the ability to manipulate and persuade and the creation of novel biological weapon recipes. The fifty-one-page document also

…

a strategic alliance with Google and Anthropic to launch the Frontier Model Forum, a group for advancing relevant research and influencing the policy agenda on AI safety risks. It was a rare issue in which the interests of Doomers, Boomers, and profit-motivated corporates aligned: Keeping frontier models front and center

…

gateway into effective altruism and, in turn, to broader Doomer ideology. Yudkowksy had also cofounded the blog LessWrong, a central hub for AI safety researchers to foster community and propagate AI safety ideas, where he’d advocated with increasing alarmism to put a full pause on AI development as his p(doom) shot up

…

connecting with employees, information about the company’s happenings was instead filtering up to the independent directors through their own personal relationships from the broader AI safety and tech communities. They also relied on Altman himself as a conduit for keeping tabs on important information. What worried them with growing intensity

…

received reports from their own sources about various problems, including the company’s lack of preparation before and significant tumult after ChatGPT, the continued AI safety concerns surrounding GPT-4’s release, and the unprecedented pace with which OpenAI was sprinting to launch new products before it had resolved many of

…

disregard with which Microsoft had broken protocol and Altman had passed over it. OpenAI’s models were on a rapid advancement trajectory and, from an AI safety perspective, they believed, could soon pass a point where such a violation could result in potentially catastrophic, if not existential, consequences. In their view,

…

Altman’s laxness with Microsoft’s breach set a dangerous precedent for how he might treat AI safety processes around model releases once the stakes went up. Meanwhile, there were other concerning examples of Altman’s behavior. In March 2023, he had

…

benignly. Sutskever rolled his eyes. The independent board members, Toner explained, were looking to strengthen the board by adding a new director with a strong AI safety background. If he had different ideas, she would be glad to hear them. At this, Sutskever latched on. The board needed to be better

…

had popularized in Washington: cybersecurity threats, CBRN weapons, persuasion, and the evasion of human control. In the week running up to the launch, an AI safety researcher still left at the company wrote an impassioned memo: Upstream processes and the rushed release of Scallion had left the Preparedness team, headed by

…

held another all-hands meeting internally to address, all at once, the Johansson and equity crises, and any continued concerns over OpenAI’s commitment to AI safety. The mood was tense. The leadership team gave a series of quick explanations. The equity issue was “unacceptable” and being corrected as quickly as

…

new level of preparedness that Altman had previously mentioned, including leveling up the security of its research clusters, reorganizing to better focus on long-term AI safety research, and forming a new AGI readiness group, led by Aleksander Mądry, to improve coordination among leadership on advancing this objective. Altman then opened

…

with questions demanding greater clarity on the various allegations from Leike, Kokotajlo, and others, as well as Piper’s reporting over OpenAI’s disregard of AI safety and suppression of employee criticism. Meanwhile, the chaos among leadership and frustrations at Altman continued unabated. Combined with the ongoing Boomer-Doomer tussle, it

…

NOTE REFERENCE IN TEXT “the problem of accidents”: Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané, “Concrete Problems in AI Safety,” preprint, arXiv, July 25, 2016, 1–29, doi.org/10.48550/arXiv.1606.06565. GO TO NOTE REFERENCE IN TEXT By November 2024, it had

…

IN TEXT OpenAI executives internally: Screenshot of Slack announcement, May 14, 2024. GO TO NOTE REFERENCE IN TEXT “Being AGI ready”: All quotes about AI safety and the Superalignment team are pulled are from an audio recording of the all-hands meeting, May 15, 2024. GO TO NOTE REFERENCE IN TEXT

…

systems that are generally smarter than humans,’ ” Twitter (now X), June 4, 2024, x.com/DKokotajlo67142/status/1797994238468407380; and his posts and comments on the AI Safety forum LessWrong: “Daniel Kokotajlo,” LessWrong, accessed November 25, 2024, lesswrong.com/users/daniel-kokotajlo. The estimated value of his equity comes from Kevin Roose,

…

17, 353, 387–88 AI Index, 105 AI Insight Forums, 311 AI Now Institute, 308 Airbnb, 36, 41, 136, 150, 202, 367 air pollution, 286 AI safety, 55–58, 122–32, 301–12, 316–24, 419. See also data privacy; existential risks alignment and, 122–23, 124, 145–46, 316–18 effective

…

274–75, 277, 287 Mechanical Turk, 194 American Sign Language, 254 Amodei, Daniela, 55–56, 58, 144–45, 156, 157, 230 Amodei, Dario, 55–58 AI safety and risks, 55–56, 57–58, 87, 122–27, 131, 133, 134, 145–46, 147, 149–52, 156–57, 362 Altman’s firing, 366 at

…

60, 171–73, 275–78, 295, 309 Carnegie Mellon University, 97, 106, 172 Carr, Andrew, 385 Carter, Ashton, 43 CBRN weapons, 301, 380 Center for AI Safety, 322 Center for Security and Emerging Technology (CSET), 7, 307, 321, 357, 358 Center on Long-Term Risk, 388 Centre for the Governance of AI

…

dinner, 28, 46, 47, 48, 55 Rubik’s Cube, 71 Russia, 146 Ukraine war, 52, 191 Rwanda, 102, 260 S Safe Superintelligence, 405 safety. See AI safety Salinas, Alejandra, 290–91 Sama AI, 190–92, 206–13, 218–19, 242, 416 Santiago, Chile, 271–74, 285, 287–88, 295–96, 299–300

Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom  · 3 Jun 2014  · 574pp  · 164,509 words

be called for before we close this chapter. We distinguished two broad classes of methods for dealing with the agency problem at the heart of AI safety: capability control and motivation selection. Table 10 gives a summary. Table 10 Control methods Capability control Boxing methods The system is confined in such a

Army of None: Autonomous Weapons and the Future of War

by Paul Scharre  · 23 Apr 2018  · 590pp  · 152,595 words

that it “runs counter to our current understandings of the limitations that computational complexity places on algorithms for learning and reasoning.” Dietterich did acknowledge that AI safety was an important issue, but said that risks from AI have more to do with what humans allow AI-enabled autonomous systems to do. “The

Warnings

by Richard A. Clarke  · 10 Apr 2017  · 428pp  · 121,717 words

from superintelligence gone rogue. Yudkowsky’s concern is so novel and perhaps so widely dismissed that there is not even an accepted scientific convention on AI safety protocols. A legitimate attempt at regulation would have to be done multilaterally, at the highest levels of every industrialized government (likely via UN treaty) and

Human Compatible: Artificial Intelligence and the Problem of Control

by Stuart Russell  · 7 Oct 2019  · 416pp  · 112,268 words

but never allowing them to affect the real world directly? This is the idea behind Oracle AI, which has been discussed at length in the AI safety community.22 An Oracle AI system can be arbitrarily intelligent, but can answer only yes or no (or give corresponding probabilities) to each question. It

…

dislikes of the strange bipedal creatures who inhabit this planet. Reasons for Caution You may find the Partnership on AI’s promises of cooperation on AI safety less than reassuring if you have been following progress in self-driving cars. That field is ruthlessly competitive, for some very good reasons: the first

…

that it always gets +1. The inevitable conclusion is that a sufficiently capable AlphaGo++ that is designed as a reward-signal maximizer will wirehead. The AI safety community has discussed wireheading as a possibility for several years.25 The concern is not just that a reinforcement learning system such as AlphaGo might

…

still more intelligent, and the process will repeat itself until, in Good’s words, “the intelligence of man would be left far behind.” Researchers in AI safety, particularly at the Machine Intelligence Research Institute in Berkeley, have studied the question of whether intelligence explosions can occur safely.26 Initially, this might seem

…

All but Tencent and Alibaba are members of the Partnership on AI, an industry consortium that includes among its tenets a promise of cooperation on AI safety. Finally, although the vast majority of humans possess little in the way of AI expertise, there is at least a superficial willingness among other players

…

new law forbids AI systems to impersonate humans in certain circumstances. These last two items—explainability and impersonation—certainly have some bearing on issues of AI safety and control. At present, there are no implementable recommendations that can be made to governments or other organizations considering the issue of maintaining control over

…

deadly sins of AI predictions,” MIT Technology Review, October 6, 2017. 32. Pinker, “Thinking does not imply subjugating.” 33. For an optimistic view arguing that AI safety problems will necessarily be resolved in our favor: Steven Pinker, “Tech prophecy.” 34. On the unsuspected alignment between “skeptics” and “believers” in AI risk: Alexander

…

have argued that biology and neuroscience are also directly relevant. See, for example, Gopal Sarma, Adam Safron, and Nick Hay, “Integrative biological simulation, neuropsychology, and AI safety,” arxiv.org/abs/1811.03493 (2018). 2. On the possibility of making computers liable for damages: Paulius Čerka, Jurgita Grigienė, and Gintarė Sirbikytė, “Liability for

Robot Rules: Regulating Artificial Intelligence

by Jacob Turner  · 29 Oct 2018  · 688pp  · 147,571 words

in AI including privacy,111 the Trolley Problem,112 algorithmic bias,113 transparency 114 and liability for harm caused by AI.115 In terms of AI safety, the White Paper explains that:Because the achieved goals of artificial intelligence technology are influenced by its initial settings, the goal of artificial intelligence design

…

Artificial Intelligence Containment”, arXiv preprint arXiv:1707.08476 (2017); Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané, “Concrete Problems in AI Safety”, arXiv preprint arXiv:1606.06565 (2016); Jessica Taylor, Eliezer Yudkowsky, Patrick LaVictoire, and Andrew Critch, “Alignment for Advanced Machine Learning Systems”, Machine Intelligence Research Institute

The Alignment Problem: Machine Learning and Human Values

by Brian Christian  · 5 Oct 2020  · 625pp  · 167,349 words

, where the episode is instantly deemed “hilarious” by all concerned. In its cartoonish, destructive slapstick, it certainly is. But for Amodei—who now leads the AI safety team at San Francisco research lab OpenAI—there is another, more sobering message. At some level, this is exactly what he’s worried about. The

…

understanding evolution, human motivation, and the delicacy of incentives, with implications for business and parenting alike. Part three takes us to the forefront of technical AI safety research, as we tour some of the best ideas currently going for how to align complex autonomous systems with norms and values too subtle or

…

he was perhaps one of the last alignment researchers who first had to live a kind of double life before being able to work in AI safety directly: working on more conventional problems to get his academic credentials, while figuring out a way to do the work he felt was truly important

…

I also, like, really wasn’t clear what I was going to do with my life.”32 Then he started reading about the idea of AI safety, through Nick Bostrom and Milan Ćirković’s book Global Catastrophic Risks, some discussions on the internet forum LessWrong, and a couple papers by Eliezer Yudkowsky

…

ask for some career advice. “I just randomly emailed him out of the blue, telling him, you know, I want to do a PhD in AI safety, can you give me some advice on where to go? And then I attached some work I’d done or something so he would hopefully

…

in which such an agent is liable to, as Leike put it, “misbehave drastically.”33 With his doctorate in hand, he prepared to enter the AI safety job market: that is, to join one of the three or four places in the world where one could make a career working on

…

AI safety. After a six-month stint at the Future of Humanity Institute in Oxford, he settled into a permanent role in London at DeepMind. “At the

…

look deeply into this question of how machines might learn complicated reward functions from humans. The project would ultimately become one of the most significant AI safety papers of 2017, remarkable not only for what it found but for what it represented: a marquee collaboration between the world’s two most active

…

AI safety research labs, and a tantalizing path forward for alignment research.34 Together they came up with a plan to implement the largest-scale test of

…

the same issue.” I remark that there’s a certain irony that his twenty-year-old idea ended up being the foundation for his current AI safety agenda. His idle thoughts on the walk to Safeway became, twenty years later, a plan to avert possible civilization-level catastrophe. “That was a complete

…

, not because it leads in bad directions, but because it leads in no directions at all.”38 Similar paradoxes and problems of definition haunt the AI safety research community. It would be good, for instance, for there to be a similar kind of precautionary principle: for systems to be designed to err

…

” or “impactful” behavior precise is a considerable challenge in itself.39 One of the first people to think about these issues in the context of AI safety was Stuart Armstrong, who works at Oxford University’s Future of Humanity Institute.40 Rather than trying to enumerate all of the things we don

…

theoretical conversation but also to create simple, game-like virtual worlds to illustrate these various problems and make the thought experiments concrete. They call these “AI safety gridworlds”—simple, Atari-like, two-dimensional (hence “grid”) environments in which new ideas and algorithms can be put to a practical test.46 The gridworld

…

instance, once a box is pushed into a corner, any states of the world that have that box anyplace else now become “unreachable.” In the AI safety gridworlds, agents looking out for stepwise relative reachability, alongside their normal goals and rewards, appear to behave rather conscientiously: agents don’t put boxes into

…

after it’s done whatever point-scoring actions the game incentivizes. Fascinatingly, the mandate to preserve attainable utility seems to foster good behavior in the AI safety gridworlds even when the auxiliary goals are generated at random.49 When Turner first elaborated the idea, on a library whiteboard at Oregon State, he

…

things out.”50 Over the course of 2018, he turned the math into working code and tossed his attainable-utility-preserving agent into DeepMind’s AI safety gridworlds. It did work. Acting to maximize each individual game’s rewards while at the same time preserving its future ability to satisfy four or

…

, we often take actions not only whose unintended effects are difficult to envision, but whose intended effects are difficult to envision. Publishing a paper on AI safety, for instance (or, for that matter, a book): it seems like a helpful thing to do, but who can say or foresee exactly how? I

…

ask Jan Leike, who coauthored the “AI Safety Gridworlds” paper with Krakovna, what he makes of the response so far to his and Krakovna’s gridworlds research. “I’ve been contacted by lots

…

of people, especially students, who get into the area and they’re like, ‘Oh, AI safety sounds cool. This is some open-source code I can just throw an agent at and play around with.’ And a lot of people have

…

. . . . I don’t know. It’s hard to know.” CORRIGIBILITY, DEFERENCE, AND COMPLIANCE One of the most chilling and prescient quotations in the field of AI safety comes in a famous 1960 article on the “Moral and Technical Consequences of Automation” by MIT’s Norbert Wiener: “If we use, to achieve our

…

and perfectly specified what we did and didn’t want the machine to do, then we had better be sure we can intervene. In the AI safety literature, this concept goes by the name of “corrigibility,” and—soberingly—it’s a whole lot more complicated than it seems.52 Almost any discussion

…

is so deep that, as we have seen in the last few chapters, much of the work being done in advanced AI applications and in AI safety in particular is about moving beyond systems that take in an explicit objective, and toward systems that attempt to imitate humans (in the case of

…

the enormity of the stakes, is not haste at all but the opposite. Bostrom’s essay came up more than once as I asked various AI safety researchers how they decided to commit their lives to that cause. “I found that argument pretty weird initially,” says Paul Christiano, “or it seemed off

…

trained on one set of examples finds itself operating in a different kind of environment, without necessarily realizing it. Amodei et al., “Concrete Problems in AI Safety.” gives an overview of this issue, which comes up in various subsequent chapters of this book. 37. Hardt, “How Big Data Is Unfair.” 38. Jacky

…

. Training systems which themselves are (or may become) optimizers of some “inner” reward function is a source of concern and of active research among contemporary AI-safety researchers. See Hubinger et al., “Risks from Learned Optimization in Advanced Machine Learning Systems.” 57. Andrew Barto, personal interview, May 9, 2018. 58. See Singh

…

”; see Ramakrishnan, Zhang, and Shah, “Perturbation Training for Human-Robot Teams.” 54. Murdoch, The Bell. 55. Some researchers at the intersection of cognitive science and AI safety, including the Future of Humanity Institute’s Owain Evans, are working on ways to do inverse reinforcement learning to take into account a person who

…

as Irreversibility.” See also Sunstein, “Irreversibility.” 38. Sunstein, “Beyond the Precautionary Principle.” See also Sunstein, Laws of Fear. 39. Amodei et al., “Concrete Problems in AI Safety,” has an excellent and broad discussion of “avoiding negative side effects” and “impact regularizers,” and Taylor et al., “Alignment for Advanced Machine Learning Systems,” also

…

Using Relative Reachability,” June 5, 2018, https://vkrakovna.wordpress.com/2018/06/05/measuring-and-avoiding-side-effects-using-relative-reachability/. 46. Leike et al., “AI Safety Gridworlds.” 47. Victoria Krakovna, personal interview, December 8, 2017. 48. The idea of stepwise baselines was suggested by Alexander Turner in https://www.alignmentforum.org

…

.” See Turner, “Optimal Farsighted Agents Tend to Seek Power.” For more on the notion of power in an AI safety context, including an information-theoretic account of “empowerment,” see Amodei et al., “Concrete Problems in AI Safety,” which, in turn, references Salge, Glackin, and Polani, “Empowerment: An Introduction,” and Mohamed and Rezende, “Variational Information

…

Turner, personal interview, July 11, 2019. 51. Wiener, “Some Moral and Technical Consequences of Automation.” 52. According to Paul Christiano, “corrigibility” as a tenet of AI safety began with the Machine Intelligence Research Institute’s Eliezer Yudkowsky, and the name itself came from Robert Miles. See Christiano’s “Corrigibility,” https://ai-alignment

…

command, see, e.g., Coman et al., “Social Attitudes of AI Rebellion,” and Aha and Coman, “The AI Rebellion.” 62. Smitha Milli, “Approaches to Achieving AI Safety” (interview), Melbourne, Australia, August 2017, https://www.youtube.com/watch?v=l82SQfrbdj4. 63. For more on corrigibility and model misspecification using this paradigm, see also

…

to the regularization method known as “early stopping”; see Yao, Rosasco, and Caponnetto, “On Early Stopping in Gradient Descent Learning.” For further discussion, in an AI safety context, about when “a metric which can be used to improve a system is used to such an extent that further optimization is ineffective or

…

?” An intriguing research direction in AI alignment involves developing machine-learning systems able to engage in debate with one another; see Irving, Christiano, and Amodei, “AI Safety via Debate.” 22. Jan Leike, “General Reinforcement Learning” (lecture), Colloquium Series on Robust and Beneficial AI 2016, Machine Intelligence Research Institute, Berkeley, California, June 9

…

.” Master’s thesis, University of Missouri–Kansas City, 1971. Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” arXiv Preprint arXiv:1606.06565, 2016. Ampère, André-Marie. Essai sur la philosophie des sciences; ou, Exposition analytique d’une classification naturelle de toutes les

…

. “Adversarial Examples Are Not Bugs, They Are Features.” In Advances in Neural Information Processing Systems, 125–36. 2019. Irving, Geoffrey, Paul Christiano, and Dario Amodei. “AI Safety via Debate.” arXiv Preprint arXiv:1805.00899, 2018. Jackson, Frank. “Procrastinate Revisited.” Pacific Philosophical Quarterly 95, no. 4 (2014): 634–47. Jackson, Frank, and Robert

…

Direction.” arXiv Preprint arXiv:1811.07871, 2018. Leike, Jan, Miljan Martic, Victoria Krakovna, Pedro A. Ortega, Tom Everitt, Andrew Lefrancq, Laurent Orseau, and Shane Legg. “AI Safety Gridworlds.” arXiv Preprint arXiv:1711.09883, 2017. Lenson, David. On Drugs. University of Minnesota Press, 1995. Letham, Benjamin, Cynthia Rudin, Tyler H. McCormick, David Madigan

…

racial bias Against Prediction (Harcourt), 78–79 age bias, 32, 396n7 AGI. See artificial general intelligence Agüera y Arcas, Blaise, 247 AI Now Institute, 396n9 AI safety artificial general intelligence delay risks and, 310 corrigibility, 295–302, 392–93n51 field growth, 12, 249–50, 263 gridworlds for, 292–93, 294, 295, 390n29

…

, 305 equiprobabiliorism, 303 equivant (company), 337n5 ergodicity assumption, 320 Ermon, Stefano, 324 ethics actualism vs. possibilism and, 239, 379n71 in reinforcement learning, 149 See also AI safety; fairness; moral uncertainty evaluation function. See value function Evans, Owain, 386–87n55 evolution, 170, 171–74, 368n56 expectations, 138–39, 197 See also TD (temporal

…

vs. statistical prediction, 91–94, 97–98 transparency and, 319, 397n19 word embedding and, 342n61 See also imitation; value alignment human-machine cooperation, 267–76 AI safety and, 268–69 aspiration and, 275–76, 386–87n55 CIRL, 267–68, 385nn40, 43–44 dangers of, 274–76 demonstration learning for, 271 feedback learning

…

shaping, 162 training data bias, 29 uncertainty, 288 value alignment, 266–67 UC Davis, 111 UC San Diego, 161 Ullman, Tomer, 345n94 uncertainty, 277–310 AI safety and, 291–92 Bayesian neural networks and, 283–86 confidence and, 281–82 corrigibility and, 295–302 deep learning brittleness, 279–81, 387n8 dissent and

…

of human values and, 247 inference and, 252–53, 269, 323–24, 385n39 inverse reward design for, 301–02 models and, 325–27 See also AI safety; human-machine cooperation; inverse reinforcement learning value functions, 138–39, 143, 238–39, 241–42, 245, 379n71 variable ratio reinforcement schedules, 153 Vaughan, Jenn Wortman

Supremacy: AI, ChatGPT, and the Race That Will Change the World

by Parmy Olson  · 284pp  · 96,087 words

technologists were leaning too far into anxiety about the future dangers of artificial intelligence, as part of a nascent field of study referred to as “AI safety.” While that research was important, some of that panic had turned into fearmongering, and it seemed like these advocates for humanity were letting emotions get

…

the better of them. “Unfortunately, some of the communities involved in AI safety are the people who are least calm,” Altman said. “That’s a dangerous situation.… It’s an extremely high-strung community.” But he was also

…

and philosophy and whose essays enthralled the site’s members. Yudkowsky was the kind of person Altman had been referring to when he called the AI safety community “high strung.” He believed that AI was more likely than anyone realized to annihilate humanity. Once it got to a certain level of intelligence

…

belongs to the world.” “Probably worth a conversation,” Musk replied. A month later, Altman emailed again, proposing a lab that would build “the first general AI.… Safety should be a first-class requirement.” The AI would be owned by a nonprofit and used “for the good of the world.” Musk replied, “Agree

…

” can refer to different research goals, and in recent years, their proponents have been at odds with one another. Researchers who say they work in AI safety tend to swim in the same waters as Yudkowsky and Jaan Tallinn and want to ensure that a superintelligent AGI system won’t cause catastrophic

…

of them to “smoke the peace pipe,” according to someone who heard the comment directly. In 2017, both Altman and Hassabis took part in an AI safety conference in California, set up by the Future of Life Institute. Hoffman was there, and afterward, he tried to set up a dinner between the

…

’t as worried about bias as he was about AI’s existential threat to humanity. He had written a research paper called “Concrete Problems in AI Safety” in which he highlighted the potential accidents that could happen with poorly designed AI systems. If AI builders specified the wrong objective in their designs

…

’s secret sauce. Just as Amodei had predicted, Anthropic was able to raise huge amounts of money almost immediately from the usual passel of rich AI-safety patrons, including Jaan Tallinn and Dustin Moskovitz, the billionaire cofounder of Facebook who was Mark Zuckerberg’s roommate at Harvard University. Money in Silicon Valley

…

extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” They came under the umbrella of “AI safety,” painting the future threat in vague terms and rarely spelling out what rogue AI systems would do or when it would happen. They also tended

…

stereotyping and who feared that AI systems would continue to perpetuate inequality. Over time, they became increasingly outraged by the actions of those in the “AI safety” camp, not least because that group was making so much money. The funding disparity was stark. The ethics side was often scrambling for cash. Groups

…

scrutinized how AI was used in healthcare and the criminal justice system, had a budget of less than $1 million. Groups that were focused on AI “safety” and the extinction threat got far more funding, often via billionaire benefactors. The Future of Life Institute, a Cambridge, Massachusetts–based nonprofit that studied how

…

. Open Philanthropy, the charitable vehicle of Facebook billionaire Dustin Moskovitz, has sprinkled a number of multimillion-dollar grants to AI safety work over the years, including a $5 million donation to the Center for AI Safety in 2022 and an $11 million donation to Berkeley’s Center for Human-Compatible AI. All told, Moskovitz

…

’s charity has been the biggest donor to AI safety, by virtue of the near $14 billion fortune that he and his wife, Cari Tuna, plan to mostly give away. That includes a $30 million

…

in the United States, advising young university graduates on careers that would have the greatest moral impact. It often steered the technically minded ones toward AI safety work. But the group also encouraged graduates to pick careers that paid the highest salaries, allowing them to donate as much money as possible to

…

with an infinite cost, you still get a problem that is infinitely large. This rationale is all the more potent if you believe, as some AI safety advocates do, that computers of the future will host the conscious minds of billions of people and also create new forms of sentient, digital lives

…

AI extinction causes. Moskovitz’s Open Philanthropy increased the number of grants it was giving to issues relating to so-called long-termist causes including AI safety research, from $2 million in 2015 to more than $100 million in 2021. Bankman-Fried had jumped in too. His FTX Future Fund, run by

…

grant-making, told one podcast that hers was between 20 and 30 percent. Nobody knew Bankman-Fried’s p(doom), but he cared enough about AI safety to invest $500 million in Anthropic. His FTX cofounders and fellow effective altruists, Nishad Singh and Caroline Ellison, also invested in the start-up that

…

network of AI doom was stirring up anxiety beyond just Washington. Two months later in the UK, British prime minister Rishi Sunak hosted an international AI Safety Summit, the first of its kind set up by a government, and gave it a strong focus on saving citizens from annihilation. “People will be

…

OpenAI, a cultural rift had grown between Altman and Sutskever over how quickly OpenAI was commercializing its technology. Sutskever had become more involved in overseeing AI safety at the company and his concerns weren’t all that different to Dario Amodei’s before him. He especially didn’t like OpenAI’s launch

…

Fairness, Accountability, and Transparency (June 2022): 173–84. Brockman, Greg. “My Path to OpenAI.” blog.gregbrockman.com, May 3, 2016. Conn, Ariel. “Concrete Problems in AI Safety with Dario Amodei and Seth Baum.” Future of Life Institute (podcast), August 31, 2016. Dowd, Maureen. “Elon Musk’s Billion-Dollar Crusade to Stop the

…

of Industry in AI Research.” Science, March 2, 2023. Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. “Concrete Problems in AI Safety.” www.arxiv.org, July 25, 2016. Copeland, Rob. “Google Management Shuffle Points to Retreat from Alphabet Experiment.” Wall Street Journal, December 5, 2019. Coulter, Martin

…

Today’s YouTube channel, June 8, 2023. Singer, Peter. The Life You Can Save. New York: Random House, 2010. “Statement on AI Risk.” Center for AI Safety, www.safe.ai, May 2023. Vallance, Chris. “Artificial Intelligence Could Lead to Extinction, Experts Warn.” BBC News, May 30, 2023. Vincent, James. “OpenAI Sued for

…

, Daron Acton advertising, Loopt and Age of Spiritual Machines, The (Kurzweil) AGI. See artificial general intelligence. AI accelerationists AI Act (Europe) AI Now Institute Airbnb AI Safety Summit Alphabet autonomous units model and China and DeepMind and valuation of See also Google AlphaFold AlphaFold Protein Structure Database AlphaGo Altman, Connie Altman, Jerry

…

removal from OpenAI board and restructuring of OpenAI and Brookings Institute Bullfrog Productions Bumble Buolamwini, Joy Buterin, Vitalik Calico Cambridge Analytica Cambridge University Center for AI Safety Center for Effective Altruism Center for Human-Compatible AI Character.ai ChatGPT ChatGPT Plus China Chrome Claude Claude Pro cloud computing Common Crawl COMPAS (Correctional

…

Offender Management Profiling for Alternative Sanctions) “Concrete Problems in AI Safety” (Amodei) Copilot Coppin, Ben coreference resolution Cotra, Ajeya Cruise DALL-E 2 D’Angelo, Adam Dartmouth College Datasheets for Datasets Dayan, Peter Dean, Jeff Deep

…

Asia Millar, George Minecraft Mitchell, Margaret at Google concerns about bias paper with Bender and Molyneux, Peter Moskovitz, Dustin Mozilla Foundation Murati, Mira Musk, Elon AI Safety Summit and bad-mouthing of Hassabis and call for AI pause and DeepMind and effective altruism and fears about AI and Hassabis and ideological focus

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future

by Keach Hagey  · 19 May 2025  · 439pp  · 125,379 words

Elon Musk

by Walter Isaacson  · 11 Sep 2023  · 562pp  · 201,502 words

Four Battlegrounds

by Paul Scharre  · 18 Jan 2023

Extremely Hardcore: Inside Elon Musk's Twitter

by Zoë Schiffer  · 13 Feb 2024  · 343pp  · 92,693 words

End Times Fascism: And the Fight for the Living World

by Naomi Klein and Astra Taylor  · 14 Sep 2026  · 393pp  · 124,539 words

Architects of Intelligence

by Martin Ford  · 16 Nov 2018  · 586pp  · 186,548 words

Artificial Intelligence: A Modern Approach

by Stuart Russell and Peter Norvig  · 14 Jul 2019  · 2,466pp  · 668,761 words

Possible Minds: Twenty-Five Ways of Looking at AI

by John Brockman  · 19 Feb 2019  · 339pp  · 94,769 words

The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future

by Tom Chivers  · 12 Jun 2019  · 289pp  · 92,714 words

More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity

by Adam Becker  · 14 Jun 2025  · 381pp  · 119,533 words

The Singularity Is Nearer: When We Merge with AI

by Ray Kurzweil  · 25 Jun 2024

The AGI Chronicles

by Kevin Roose  · 481pp  · 151,124 words

The Precipice: Existential Risk and the Future of Humanity

by Toby Ord  · 24 Mar 2020  · 513pp  · 152,381 words

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity

by Amy Webb  · 5 Mar 2019  · 340pp  · 97,723 words

These Strange New Minds: How AI Learned to Talk and What It Means

by Christopher Summerfield  · 11 Mar 2025  · 412pp  · 122,298 words

If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All

by Eliezer Yudkowsky and Nate Soares  · 15 Sep 2025  · 215pp  · 64,699 words

A Hacker's Mind: How the Powerful Bend Society's Rules, and How to Bend Them Back

by Bruce Schneier  · 7 Feb 2023  · 306pp  · 82,909 words

The Means of Prediction: How AI Really Works (And Who Benefits)

by Maximilian Kasy  · 15 Jan 2025  · 209pp  · 63,332 words

Artificial You: AI and the Future of Your Mind

by Susan Schneider  · 1 Oct 2019  · 331pp  · 47,993 words

Code Dependent: Living in the Shadow of AI

by Madhumita Murgia  · 20 Mar 2024  · 336pp  · 91,806 words

Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI

by Carissa Véliz  · 21 Apr 2026  · 503pp  · 129,255 words

Human + Machine: Reimagining Work in the Age of AI

by Paul R. Daugherty and H. James Wilson  · 15 Jan 2018  · 523pp  · 61,179 words

On the Edge: The Art of Risking Everything

by Nate Silver  · 12 Aug 2024  · 848pp  · 227,015 words

Nexus: A Brief History of Information Networks From the Stone Age to AI

by Yuval Noah Harari  · 9 Sep 2024  · 566pp  · 169,013 words

The Rise and Fall of the Artificial State

by Jill Lepore  · 24 Aug 2026  · 363pp  · 122,400 words

We Are as Gods: A Survival Guide for the Age of Abundance

by Peter H. Diamandis and Steven Kotler  · 13 Apr 2026  · 225pp  · 76,418 words

What We Owe the Future: A Million-Year View

by William MacAskill  · 31 Aug 2022  · 451pp  · 125,201 words

What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence

by John Brockman  · 5 Oct 2015  · 481pp  · 125,946 words

To Be a Machine: Adventures Among Cyborgs, Utopians, Hackers, and the Futurists Solving the Modest Problem of Death

by Mark O'Connell  · 28 Feb 2017  · 252pp  · 79,452 words

Searches: Selfhood in the Digital Age

by Vauhini Vara  · 8 Apr 2025  · 301pp  · 105,209 words

Deep Utopia: Life and Meaning in a Solved World

by Nick Bostrom  · 26 Mar 2024  · 547pp  · 173,909 words

Genius Makers: The Mavericks Who Brought A. I. To Google, Facebook, and the World

by Cade Metz  · 15 Mar 2021  · 414pp  · 109,622 words

AI Superpowers: China, Silicon Valley, and the New World Order

by Kai-Fu Lee  · 14 Sep 2018  · 307pp  · 88,180 words

The Age of Extraction: How Tech Platforms Conquered the Economy and Threaten Our Future Prosperity

by Tim Wu  · 4 Nov 2025  · 246pp  · 65,143 words

What If We Get It Right?: Visions of Climate Futures

by Ayana Elizabeth Johnson  · 17 Sep 2024  · 588pp  · 160,825 words

Love Machines: How Artificial Intelligence Is Transforming Our Relationships

by James Muldoon  · 7 Sep 2026  · 217pp  · 71,182 words

The Doomsday Calculation: How an Equation That Predicts the Future Is Transforming Everything We Know About Life and the Universe

by William Poundstone  · 3 Jun 2019  · 283pp  · 81,376 words

Against the Machine: On the Unmaking of Humanity

by Paul Kingsnorth  · 23 Sep 2025  · 388pp  · 110,920 words

How AI Thinks

by Nigel Toon  · 13 Dec 2023

Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms: Proceedings of the Agi Workshop 2006

by Ben Goertzel and Pei Wang  · 1 Jan 2007  · 303pp  · 67,891 words

The Quiet Damage: QAnon and the Destruction of the American Family

by Jesselyn Cook  · 22 Jul 2024  · 321pp  · 95,778 words

The People vs Tech: How the Internet Is Killing Democracy (And How We Save It)

by Jamie Bartlett  · 4 Apr 2018  · 170pp  · 49,193 words

The Dark Cloud: How the Digital World Is Costing the Earth

by Guillaume Pitron  · 14 Jun 2023  · 271pp  · 79,355 words

Superbloom: How Technologies of Connection Tear Us Apart

by Nicholas Carr  · 28 Jan 2025  · 231pp  · 85,135 words

The Wealth Ladder: Proven Strategies for Every Step of Your Financial Life

by Nick Maggiulli  · 22 Jul 2025

The Mysterious Mr. Nakamoto: A Fifteen-Year Quest to Unmask the Secret Genius Behind Crypto

by Benjamin Wallace  · 18 Mar 2025  · 431pp  · 116,274 words

Co-Intelligence: Living and Working With AI

by Ethan Mollick  · 2 Apr 2024  · 189pp  · 58,076 words

Ways of Being: Beyond Human Intelligence

by James Bridle  · 6 Apr 2022  · 502pp  · 132,062 words

This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web

by Tim Berners-Lee  · 8 Sep 2025  · 347pp  · 100,038 words

Applied Artificial Intelligence: A Handbook for Business Leaders

by Mariya Yao, Adelyn Zhou and Marlene Jia  · 1 Jun 2018  · 161pp  · 39,526 words