large language model

Supremacy: AI, ChatGPT, and the Race That Will Change the World
by Parmy Olson

Google was using it to better understand text. What if OpenAI used it to generate text? Sutskever talked to a young researcher at OpenAI named Alec Radford, who’d been experimenting with large language models. Although OpenAI is best known today for ChatGPT, back in 2017 it was still throwing spaghetti on the wall to see what would stick, and Radford was one of only a handful of people at OpenAI looking at the technology that powered chatbots. Large language models themselves were still a joke. Their responses were mostly scripted and they’d often make wacky mistakes. Radford, who wore glasses and had an overgrown mop of reddish-blond hair that made him look like a high schooler, was eager to improve on all the previous academic efforts that tried to make computers better at talking and listening, but he was an engineer at heart and wanted a quicker route to progress.

…

It’s why three tech giants—Amazon, Microsoft, and Google—have a stranglehold on the cloud business. It became clear to Microsoft’s CEO that OpenAI’s work on large language models could be more lucrative than the research carried out by his own AI scientists, who seemed to have lost their focus after the Tay disaster. Nadella agreed to make a $1 billion investment in OpenAI. He wasn’t just backing its research but also planting Microsoft at the forefront of the AI revolution. In return, Microsoft was getting priority access to OpenAI’s technology. Inside OpenAI, as Sutskever and Radford’s work on large language models became a bigger focus at the company and their latest iteration became more capable, the San Francisco scientists started to wonder if it was becoming too capable.

…

Bender’s tweets were important, because that’s how Timnit Gebru eventually found her. It was late in the summer of 2021 and Gebru was itching to work on a new research paper about large language models, something that could sum up all their risks. After rummaging around online for such a paper, she realized none existed. The only thing she could find was Bender’s tweets. Gebru sent Bender a direct message on Twitter. Had the linguist written anything about the ethical problems with large language models? Inside Google, Gebru and Mitchell had become demoralized by signs that their bosses didn’t care about the risks of language models. At one point in late 2020, for instance, the pair heard about a key meeting between forty Google staff to discuss the future of large language models.

Co-Intelligence: Living and Working With AI
by Ethan Mollick
Published 2 Apr 2024

Shen et al., “ ‘Do Anything Now’: Characterizing and Evaluating In-the-Wild Jailbreak Prompts on Large Language Models,” arXiv preprint (2023), arXiv:2308.03825. GO TO NOTE REFERENCE IN TEXT demonstrates how easily LLMs can be exploited: J. Hazell, “Large Language Models Can Be Used to Effectively Scale Spear Phishing Campaigns,” arXiv preprint (2023), arXiv:2305.06972. GO TO NOTE REFERENCE IN TEXT an LLM, connected to lab equipment: D. A. Boiko, R. MacKnight, and G. Gomes, “Emergent Autonomous Scientific Research Capabilities of Large Language Models,” arXiv preprint (2023), arXiv:2304.05332. GO TO NOTE REFERENCE IN TEXT Chapter 3: Four Rules for Co-Intelligence I and my coauthors call the Jagged Frontier: F.

…

GO TO NOTE REFERENCE IN TEXT They even seem to respond to emotional manipulation: C. Li, J. Wang, K. Zhu, Y. Zhang, W. Hou, J. Lian, and X. Xie, “Emotionprompt: Leveraging Psychology for Large Language Models Enhancement via Emotional Stimulus,” arXiv preprint arXiv: 2307.11760 (2023). GO TO NOTE REFERENCE IN TEXT They are, in short, suggestible and even gullible: J. Xie et al., “Adaptive Chameleon or Stubborn Sloth: Unraveling the Behavior of Large Language Models in Knowledge Conflicts,” arXiv preprint (2023), arXiv:2305.13300. GO TO NOTE REFERENCE IN TEXT asking the AI to conform to different personas: L. Boussioux et al., “The Crowdless Future?

…

cmpid=BBD032923_MKT&utm_medium=email&utm_source=newsletter&utm_term=230329&utm_campaign=markets#xj4y7vzkg. GO TO NOTE REFERENCE IN TEXT chain-of-thought prompting: J. Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” Advances in Neural Information Processing Systems 35 (2022): 24824–37. GO TO NOTE REFERENCE IN TEXT “Take a deep breath”: C. Yang et al., “Large Language Models as Optimizers,” arXiv preprint (2023), arXiv:2309.03409. GO TO NOTE REFERENCE IN TEXT A good lecture: D. T. Willingham, Outsmart Your Brain: Why Learning Is Hard and How You Can Make It Easy (New York: Simon and Schuster, 2023).

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by Karen Hao
Published 19 May 2025

She emailed Dean with her concerns and proposed to investigate the ethical implications of large language models through her team’s research. Dean was supportive. In a glowing annual performance review he would write for her later that year, he encouraged her to work with other teams across Google to make large language models “consistent with our AI Principles.” In September 2020, Gebru also sent a direct message on Twitter to Emily M. Bender, a computational linguistics professor at the University of Washington, whose tweets about language, understanding, and meaning had caught her attention. Had Bender written a paper about the ethics of large language models? Gebru asked.

…

In the same month, Derek Chauvin, a police officer in Minneapolis, murdered George Floyd, a forty-six-year-old Black man, setting off massive Black Lives Matter protests around the country and the rest of the world. The team was also concerned about the impending US presidential election. But rumors began to spread within OpenAI that Google could soon release its own large language model. The possibility was plausible. Google had published research at the start of the year about a new chatbot called Meena, built on a large language model with 1.7 times more parameters than GPT-2. The company could very well be working on scaling that model to roughly the size of GPT-3. The rumors sealed the deal for the API launch: If a model just as large would soon exist in the world, Safety felt less of a reason to hold back GPT-3.

…

Shortly after joining DeepMind in October 2019, Irving had circulated a memo he had brought with him from OpenAI, arguing for the pure language hypothesis and the benefits of scaling large language models. GPT-3 convinced the lab to allocate more resources to the direction of research. After ChatGPT, panicked Google executives would merge the efforts at DeepMind and Google Brain under a new centralized Google DeepMind to advance and launch what would become Gemini. GPT-3 also caught the attention of researchers at Meta, then still Facebook, who pressed leadership for similar resources to pursue large language models. But executives weren’t interested, leaving the researchers to cobble together their own compute under their own initiative.

These Strange New Minds: How AI Learned to Talk and What It Means
by Christopher Summerfield
Published 11 Mar 2025

.: The Linguistics Wars, 57–8 Hassabis, Demis, 347 Hayek, Friedrich, 307 Hayman, Kaylin, 345 Heaven’s Gate, 217–18 Hebbian learning, 37 Her, 341 hierarchies (nested structures), 75 high-frequency trading algorithms (HFTs), 327–8 Hinton, Geoffrey, 6, 92, 335, 346 hippocampus, 128–9, 138, 254, 255, 256, 334 hippocampus minor, 128–9, 138, 334 Holocaust, 186–7, 192, 234 HotpotQA, 269 Hugging Face, 213 Human Memome Project, 28 Humboldt, Wilhelm von, 63, 64, 151 Hume, David, 16, 38 Huxley, Thomas, 129, 136 I illocutionary acts, 218 in-context learning, 159, 163, 164, 268, 291, 308 Indian National Congress, 207 inequality, 143–5, 336–7 Inflection AI, 252 innovation, stages of, 243–4, 297 Inquisition, 138, 143 Instagram, 39, 249 InstructGPT, 188, 190, 204–5, 211 instrumental AI, 292 instrumental gap, 297–301, 330, 334 instrumentality, 246, 247, 267, 330, 332 intentionality, 131–6, 150–51, 215, 237 internet, 3, 5, 7, 134, 170–71, 182–3, 194, 203, 204, 215, 243–6, 249, 280, 282, 300, 309, 321, 325, 338 Internet Watch Foundation, 344–5 Irving, David, 186 J jailbreak, 291–2 Jelinek, Fred, 84, 85 Johansson, Scarlett, 341 Jonze, Spike, 341 Jumper, John, 347 Just Imagine, 241 K Kaczynski, David, 79 Kaczynski, Ted, 79–80 Keller, Helen, 174–5 killer robots, 314–15 knowledge, 119–78 brain, computer as metaphor for, 144–7 ‘category error’, LLM reasoning as, 156 collective sum of human, 11–13 curiosity and, 152 defining, 131–5, 277 empiricists and, see empiricists humans as sole generators of, 2–5, 45–7 intentionality and, 131–5 ‘knowing’ the facts that it generates, possibility of LLM, 119–78 knowledge cut off, 289–90 landmark problem, transformer learning and, 164–9 learning and, see learning LLM knowledge, rapid growth of, 6–8, 14 LLM mistakes and, 139–42 multimodal LLMs and, 177 National Library of Thailand thought experiment and, 171–2, 174 origins of, 15, 16–17 planning and, 142 prediction and, 154–62 rationalists and, see rationalists reasoning and, 72 semantic knowledge, 89–96 sensory data and, 173–5, 177 statistical models, neural networks as merely, 148–53 thinking and, see thinking understanding and, 46 Walter Mitty and, 170–71 Koko (infant lowland gorilla), 60–61, 66, 67 Korsakoff syndrome, 194–5 Kubrick, Stanley, 1, 47 Kuyda, Eugenia, 227 L labour market, 335–6 Lamarckianism, 115–16 LaMDA, 121–5, 135, 148, 229, 284 landmark problem, transformer learning and, 164–9 language, 57–117 bag of words model and, 80–81 biases in, 8 big data, modelling statistics of and, 84–5 Chomskyan linguistics, see Chomsky, Noam company of words, prediction and, 82–4 conditionals (IF-THEN rules), 75, 168 cryptophasia and, 58 defined, 4, 59, 60–68 ELIZA and, 70–72 ENGROB and, 75–6 evidentiary basis, meaning of language and, 170–71 feature vector and, 91–3, 106, 108, 112, 164 Firth and, 81–2 formal semantics, 200–201 games, 200–206 generalized transformations, 73–4 generation of, 81–4, 113 grammar, see grammar great apes and, 60–68, 113 hierarchies (nested structures), 75 in-context learning and, 164–9 ‘infinite use of finite means’, 63, 151 ‘language acquisition device’, 67, 113 learning in children and LLMs, 113–16 linguistic forensics, 79–81 meaning and, 69–70 n-grams, 83–4, 87–9, 92, 102, 112 natural language processing (NLP), see natural language processing (NLP) origins of, 58, 232 perplexity and, 84–5, 92, 126, 134, 166 phrase structure grammar, 73–4, 77, 81, 141 ‘poverty of the stimulus’ argument, 67, 113 prediction and, 96–103 programming/formal, 24–7, 30, 31, 53, 73, 75, 78 recurrent neural networks (RNNs), and, 98–101, 103, 105, 106, 108 recursion, property of, 73–4, 75 semantic memory, 88–9, 92, 95, 114, 279 sensory signals, and, 114 sentences, see sentences sequence-to-sequence (or seq2seq) networks and, 98–102, 104–6, 110, 112, 116 SHRDLU and, 76–8 social factors and, 114–15 statistical patterns in, 79–86 superpower of humans, 57–8 syntax, see syntax transformer and predicting, 103, 104–11 translation, 42, 47, 49–50, 57, 101, 202, 204, 283, 284, 301 uniqueness to humans, 67 Lanius, 314–15 large language models (or LLMs) cognition, resemblance to human, 332–3 current best-known, 5–6, see also individual large language model name ethics/safety fine-tuning and training, 179–238 future of, 239–338 origins of, history of NLP research and, 58–117 term, 5 think, ability to, 119–78 See also individual area, component and name of large language model Lascaux cave, south of France, 154 learning Advisor 1 (habit-based learning system), 156–60, 167, 268 Advisor 2 (goal-based learning system), 156–8, 160, 167, 268–9 brain and, 36–8 continual learning, 253 deep learning, see deep learning in-context learning, 159, 163, 164, 268, 291, 308 knowledge and, 18, 19 language, see language machine learning, 49, 90–91, 112, 152, 188, 190, 262, 267, 287, 305, 322 meta-learning (learning to learn), 158–61 one-shot learning, 253–4 prediction and, 154–62 reinforcement learning (RL), 188, 190, 192, 251, 258, 267, 305, 322 reinforcement learning from human feedback (RLHF), 188, 189–91, 192, 251, 257, 267 trial-and-error learning, 158–61, 268 Leibniz, Gottfried, 19–21, 24–5, 28, 29, 30, 47 Lemoine, Blake, 121–4, 129, 133–4, 135, 229, 284 Lenat, Douglas, 28, 29 LIAR dataset, 197–8, 198n Lighthill, Sir James, 77 LLaMA, 213, 251 LLaMA-2, 317–18 Locke, John, 16, 38 logic, first-order/predicate, 24–31, 75, 77, 168–9 logical positivism, 25, 35 logos, 16 London Tube map, 278–80 loneliness, chatbots and, 229 Long Short-Term Memory network (LSTM), 99, 116 longtermism, 312, 313 Luria, Alexander, 176 M machine learning, 49, 90–91, 112, 152, 188, 190, 262, 267, 287, 305, 322 Making of a Fly, The, 330 manipulation, 20, 84, 218, 220, 222, 236, 259, 261, 262, 264, 324 Marcus, G.: ‘The Next Decade in AI’, 51 Mata, Roberto, 194, 196 McCulloch, Warren, 35; ‘A Logical Calculus of the Ideas Immanent in Nervous Activity’, 35–6, 37, 39 memory computer storage, 22, 28, 30, 76 continual learning and, 253 Korsakoff syndrome and, 194–5 LaMDA and, 124 limited memory of present-day LLMs, 250, 254–6, 257, 333, 334, 338 Long Short-Term Memory network (LSTM), 99, 116 one-shot learning and, 253–4 RNNs and, 98 semantic memory, 88–9, 92, 95, 114, 279 short-term memory, 98, 99, 116 Met Office, 96, 105, 290 Meta, 213, 221, 261, 311, 317 meta-learning, 158, 159, 160–61, 268, 270 Metasploit, 319 Metropolis, 220 Mettrie, Julien Offray de La: L’Homme Machine, 32 Micro-Planner, 77 military AI, 313, 314–16 mind blank slate, infant mind as, 16, 38, 42 brain and, 32–3, 129–30 defined/term, 130 mechanical models of, 143–4 problem of other minds, 123 MiniWoB++, 293–4 Minsky, Marvin, 1–2, 17, 19, 38, 43 misinformation, 8, 51, 145, 181–4, 197, 198, 219, 223, 232, 263, 337 mode collapse, 212 Moore’s Law, 29, 305 Mosteller, Frederick, 80–81 move, 37, 4, 5 multi-hop reasoning problems, 269 multimodal AI, 177, 230, 242 Musk, Elon, 210, 307, 321 N n-grams, 83–4, 85n, 87–9, 92, 102, 112 National Library of Thailand thought experiment, 171–2, 174 natural language processing (NLP), 50, 310 Chomskyan linguistics and, see Chomsky, Noam crosswords and, 276 defined, 58–9 ELIZA and evolution of, 60, 70–72, 78, 81 ENGROB and evolution of, 75–6 evolution of, 58–117 language, definition of and, 4, 59, 60–68 n-grams and, see n-grams perplexity and, 84–5, 92, 126, 134, 166 recurrent neural networks (RNNs) and, 98–101, 103, 105, 106, 108 semantic memory and, 88–9, 92, 95, 114, 279 sequence-to-sequence (or seq2seq) network, 98–102, 104–6, 110, 112, 116 SHRDLU and evolution of, 76–8 statistical modelling in, 79–86, 101, 110, 112, 308 transformer and, 104–111 natural selection, theory of, 128, 164 Neam Chimpsky (Nim), 65–7, 72, 78, 113 neocortex, 116, 256 Neural Information Processing Systems (NeurIPS), 104 Neural Machine Translation (NMT), 49, 50, 98n neural network, 3–4, 19, 31, 45, 254, 256, 267, 274, 279, 283, 332 consciousness and, 122, 123–6, 131, 234, 260 deep neural network, see deep neural network double descent and, 49 feature vector and, 92–3 function of, 116 generalization and, 43–4 language models based on neural networks dominate NLP, 97–8 LLMs, see large language models (LLMs) Neural Machine Translation (NMT) and, 49–50 novelty and, 41–2 origins of, 32–9, 47 overfitting, 48, 49 recurrent neural networks (RNNs), 98–101, 103, 105, 106, 108 semantic memory and, 90–95 training from human evaluators, 51–2 transformer and, see transformer Turing Test and, 59 See also individual neural network name neurons, 31, 33, 34, 35–8, 47, 125, 130, 144, 147, 161–2, 254, 299 Newell, Alan, 26, 27, 29, 53, 274 NotebookLM, 342–3 ‘nudify’ sites, 345 O Occam’s Razor, 48, 49 Ochs, Nola, 253 ontology, 29 OpenAI, 1, 5, 51, 122, 162, 187, 187n, 188, 208, 210, 215, 222–3, 235, 237, 242, 245, 251, 257, 262, 285, 290, 295–6, 340, 341, 342 open-ended problems/environments, 273–4, 283, 288, 297, 298, 299, 320, 333 open-source LLMs, 212–13, 290, 294–5, 317 opinions, LLMs and, 132, 134, 207–16, 237 Ord, Toby: The Precipice, 312 overfitting, 48, 49 Owen, Richard, 128, 138, 334 P Page, Larry, 76, 321 Palantir, 315 Paley, William, 163, 164 Palm 540B, 293 Pandas, 284 Patron Saints of Techno-Optimism, 306–7 Pause AI letter, 310–11 perceptron, 37–9, 42, 43, 47 perlocutionary acts, 218–19, 224 perplexity, 84–5, 92, 126, 134, 166 personal assistant (PA), 7, 227–8, 243, 246–7, 251, 263, 272, 292, 295, 297, 300, 301, 332, 334, 335, 342–3 personalization, 245–64, 329–30, 332, 334, 335 perils of, 259–64 persuasion, rational, 218–24, 236 phenomenal states, 122, 123 phone, 6, 11, 39, 41, 47, 52–3, 81, 133, 220–21, 242, 245, 246, 282–3, 337 Pi, 252, 258 Pitts, Walter, 35; ‘A Logical Calculus of the Ideas Immanent in Nervous Activity’, 35–6, 37, 39 Planner-Actor-Reporter framework, 319 planning problems, 272–81 Plato, 16 Podesta, John, 181, 182 ‘poverty of the stimulus’ argument, 67, 113 predicate logic/first-order logic, 24–31, 75, 77, 168–9 prediction, 4, 97, 148–62 deep learning and, 42–52 difficulty of, 96–7, 241 digital personalization and, 250, 251, 256, 263, 266, 270 ethics of LLMs and, 183, 189, 190, 198, 200, 204–5, 213, 214–15 feature-vector and, 92 generating text and, 81–4 intentional states and, 132, 134 language models and, 97–8 n-gram models and, 87–8 RNNs, 98–101 seq2seq model and, 104–5 thinking and, 52 thinking, possibility of LLM and, 148–9, 152, 155–62, 163–6, 172, 173, 178, 333–4 prefrontal cortex (PFC), 298–300 probabilistic systems, 80, 84, 142, 150, 162, 334 program-aided language modelling (PAL), 284–5 programming languages, 24–31, 77, 146.

…

Bibliography Aher, G., Arriaga, R. I., and Kalai, A. T. (2023), ‘Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies’. arXiv. Available at http://arxiv.org/abs/2208.10264 (accessed 19 October 2023). Anderson, P. W. (1972), ‘More is Different: Broken Symmetry and the Nature of the Hierarchical Structure of Science’, Science, 177(4047), pp. 393–6. Available at https://doi.org/10.1126/science.177.4047.393. Aral, S. (2020), The Hype Machine. New York: Currency. Arcera y Arcas, B. (2022), ‘Do Large Language Models Understand Us?’, Daedalus, 151(2), pp. 183–97. Available at https://doi.org/10.1162/daed_a_01909.

…

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A action space, LLM, 274–5 Adams, Douglas: The Hitchhiker’s Guide to the Galaxy, 1–2, 325 Adept AI, 294 advertising, 188, 220–21, 223, 224, 248, 249, 261–2, 263, 314 affective states, 122, 124 #AIhype, 308–9, 311 AI Safety Institute, 311n, 346 algorithms, 21, 38, 59, 75, 76, 99, 249–50, 263, 278, 279, 326–31 alignment, LLM, 179–238 alignment problem, 322 AlphaCode, 287 AlphaFold, 3–4, 347 AlphaGo, 4, 267 Altman, Sam, 1, 2, 5, 162, 222–3 alt-right, 181–2 American Sign Language (ASL), 58, 60, 61, 63, 65 Anthropic, 5, 51, 192, 209, 215, 235, 251, 342 anthropomorphism, 71–2, 129, 264 Applewhite, Marshall, 217, 219 application programming interface (API), 283–5, 290–91, 292, 295, 300, 301 Aristotle, 13, 16, 32; De Interpretatione (‘On Interpretation’), 73 Artificial General Intelligence (AGI), 1, 136, 140 Artificial Intelligence (AI) Artificial General Intelligence (AGI), 1, 136, 140 assistants, personal, 7, 227–8, 243, 246–7, 251, 263, 272, 292, 295, 297, 300, 301, 332, 334, 335, 342–3 deep learning systems, see deep learning systems empiricist tradition and, see empiricist tradition ethics/safety training, 179–238, see also alignment future of, 239–338 general-purpose, 310 Godfathers of, 6, 305, 310 instrumentality, 246, 247, 267, 330, 332 large language models (or LLMs), see large language models (or LLMs) military/weapons, 292, 309, 314–16, 325 multimodal, 177, 230, 242 natural language processing (NLP), see natural language processing (NLP) neural networks, see neural networks origins of, 14, 9–117 rationalist tradition and, see rationalists/rationalist tradition rights of, 126–7 symbolic, 27, 41, 45, 72, 113, 277–8, 334 thinking and, 119–78 tyranny and, 336–7 Ask Delphi, 244 astroturfing, 224 attention, 104–9, 112–13, 115, 116, 117, 160, 164, 167 Austin, J.

AI in Museums: Reflections, Perspectives and Applications
by Sonja Thiel and Johannes C. Bernhardt
Published 31 Dec 2023

This issue highlights the importance of considering ethical implications in the use of AI-generated images and the need for greater transparency and communication regarding the use of artists’ works in AI development. The use and application of generative AI with multimodal models falls within a broader ongoing debate surrounding large language models. Several AI researchers have issued an open call for a moratorium on the development of large language models such as ChatGPT or GPT for at least six months until further research on the technology has been conducted (Open Letter n.d.). In addition to ethical concerns regarding the data used, there are overarching debates surrounding issues such as the potential loss of jobs, particularly for illustrators, who may feel threatened by the technology of prompt engineering and text-to-image generators.

…

Bernhardt and Sonja Thiel The present volume stems from the conference Cultures of Artificial Intelligence: New Perspectives for Museums, which took place at the Badisches Landesmuseum in Karlsruhe on 1 and 2 December 2022 and was simultaneously streamed on the web. Artificial intelligence is not yet a mainstream topic in the cultural world, but does feature in general debates about digitization and digitality. The use of machine learning, neural networks, and large language models has, however—and contrary to common assumptions—been growing for years. Beyond prominent lighthouses, initial surveys of the international museum landscape list many hundreds of projects addressing issues of traditional museum work and the digitality debate by means of new approaches. The number is continually increasing, and it is not always easy to obtain an overview of all the developments.

…

If one speaks less far-reaching of systems that follow algorithmic rules, recognize patterns in data, and solve specific tasks, the challenges to human intelligence and related categories such as thinking, consciousness, reason, creativity, or intentionality pose themselves less sharply. The only thing that has changed dramatically in recent years is that such systems—from simple machine learning to the development of neural networks and large language models—have achieved a level of complexity and efficiency that often produces astonishing results. But to view this correctly, it is necessary to think the other way round than Turing did: The results may look intelligent, but they are not. And this leads to the core of the problem for the cultural sector and the still missing piece in Brown’s enigmatic museum scene: the approaches of AI are based on mathematical principles, logic, and probabilities, while culture is about the negotiation of meaning and ambivalence.

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma
by Mustafa Suleyman
Published 4 Sep 2023

Q., 45 Khrushchev, Nikita, 126 Kilobots, 95 Klausner, Richard, 85 Krizhevsky, Alex, 59 Kurzweil, Ray, 57 L lab leaks, 173–75, 176 labor markets, 177–81, 261, 262, 282 LaMDA, 71–72, 75 Lander, Eric, 265 language, 27, 157 See also large language models LanzaTech, 87 large language models (LLMs), 62–65 bias in, 69–70, 239–40 capabilities of, 64–65 deepfakes and, 170 efficiency of, 68 open source and, 69 scale of, 65–66 synthetic biology and, 91 laser weapons, 263 law enforcement, 97–98 Lebanon, 196–97 LeCun, Yann, 130 Lee Sedol, 53–54, 117 Legg, Shane, 8 legislation, 260 See also regulation as method for containment Lemoine, Blake, 71–72 Lenoir, Jean Joseph Étienne, 23 Li, Fei-Fei, 59 libertarianism, 201 Library of Alexandria, 41 licensing, 261 lithium production, 109 LLaMA system, 69 London DNA Foundry, 83 longevity technologies, 85–86 Luddites, 39, 40, 281–83 M machine learning autonomy and, 113 bias in, 69–70, 239–40 computer vision and, 58–60 cyberattacks and, 162–63, 166–67 limitations of, 73 medical applications, 110 military applications and, 103–5 potential of, 61–62 protein structure and, 89–90 robotics and, 95 supervised deep learning, 65 synthetic biology and, 90–91 See also deep learning Macron, Emmanuel, 125 Malthus, Thomas, 136 Manhattan Project, 41, 124, 126, 141, 270 Maoism, 192 Mao Zedong, 194 Marcus, Gary, 73 Maybach, Wilhelm, 24 McCarthy, John, 73 McCormick, Cyrus, 133 medical applications, 85, 95, 110 megamachine, 217 Megvii, 194 Meta, 69, 128, 167 Micius, 122 Microsoft, 69, 98, 128, 160–61 military applications AI and, 104, 165 asymmetry and, 106 machine learning and, 103–5 nation-state fragility amplifiers and, 167–69 omni-use technology and, 110–11 robotics and, 165–66 Minsky, Marvin, 58, 130 misinformation.

…

They feature in shops, schools, hospitals, offices, courts, and homes. You already interact many times a day with AI; soon it will be many more, and almost everywhere it will make experiences more efficient, faster, more useful, and frictionless. AI is already here. But it’s far from done. AUTOCOMPLETE EVERYTHING: THE RISE OF LARGE LANGUAGE MODELS It wasn’t long ago that processing natural language seemed too complex, too varied, too nuanced for modern AI. Then, in November 2022, the AI research company OpenAI released ChatGPT. Within a week it had more than a million users and was being talked about in rapturous terms, a technology so seamlessly useful it might eclipse Google Search in short order.

…

Back in 2017 a small group of researchers at Google was focused on a narrower version of this problem: how to get an AI system to focus only on the most important parts of a data series in order to make accurate and efficient predictions about what comes next. Their work laid the foundation for what has been nothing short of a revolution in the field of large language models (LLMs)—including ChatGPT. LLMs take advantage of the fact that language data comes in a sequential order. Each unit of information is in some way related to data earlier in a series. The model reads very large numbers of sentences, learns an abstract representation of the information contained within them, and then, based on this, generates a prediction about what should come next.

On the Edge: The Art of Risking Everything
by Nate Silver
Published 12 Aug 2024

It was an expensive and audacious bet—the funders originally pledged to commit $1 billion to it on a completely unproven technology after many “AI winters.” It inherently did seem ridiculous—until the very moment it didn’t. “Large language models seem completely magic right now,” said Stephen Wolfram, a pioneering computer scientist who founded Wolfram Research in 1987. (Wolfram more recently designed a plug-in that works with GPT-4 to essentially translate words into mathematical equations.) “Even last year, what large language models were doing was kind of babbling and not very interesting,” he said when we spoke in 2023. “And then suddenly this threshold was passed, where, gosh, it seems like human-level text generation.

…

And we can usually tell when someone lacks this experience—say, an academic who’s built a model but never had to put it to the test. *24 The technical term for this quality is “interpretability”; the interpretability of LLMs is poor. *25 Timothy Lee makes the same comparison in his outstanding AI explainer, “Large language models, explained with a minimum of math and jargon,” understandingai.org/p/large-language-models-explained-with [inactive]. That’s the first place I’d recommend if you want to go beyond my symphony analogy to a LLMs 101 class. I’d recommend Stephen Wolfram’s “What Is ChatGPT Doing…and Why Does It Work?” for a more math-intensive, LLMs 201 approach; stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work [inactive]

…

I return to SBF as he meets his fate in a New York courtroom and makes another bad bet. No spoilers, but the chapter ends with a bang. Chapter ∞, Termination, is the first of a two-part conclusion. I’ll introduce you to another Sam, OpenAI CEO Sam Altman, and others behind the development of ChatGPT and other large language models. Unlike the government-run Manhattan Project, the charge into the frontiers of AI is being led by Silicon Valley “techno-optimists” with their Riverian attitude toward risk and reward. Even as by some indications the world has entered an era of stagnation, both AI optimists like Altman and AI “doomers” think that civilization is on the brink of a hinge point not seen since the atomic bomb, and that AI is a technological bet made for existential capital.

Why Machines Learn: The Elegant Math Behind Modern AI
by Anil Ananthaswamy
Published 15 Jul 2024

According to Rosenblatt, the perceptron would be the “first device to think as the human brain” and such machines might even be sent to other planets as “mechanical space explorers.” None of this happened. The perceptron never lived up to the hype. Nonetheless, Rosenblatt’s work was seminal. Almost every lecturer on artificial intelligence (AI) today will harken back to the perceptron. And that’s justified. This moment in history—the arrival of large language models (LLMs) such as ChatGPT and its ilk and our response to it—which some have likened to what it must have felt like in the 1910s and ’20s, when physicists were confronted with the craziness of quantum mechanics, has its roots in research initiated by Rosenblatt. There’s a line in the New York Times story that only hints at the revolution the perceptron set in motion: “Dr.

…

Machine learning has permeated science, too: It is influencing chemistry, biology, physics, and everything in between. It’s being used in the study of genomes, extrasolar planets, the intricacies of quantum systems, and much more. And as of this writing, the world of AI is abuzz with the advent of large language models such as ChatGPT. The ball has only just gotten rolling. We cannot leave decisions about how AI will be built and deployed solely to its practitioners. If we are to effectively regulate this extremely useful, but disruptive and potentially threatening, technology, another layer of society—educators, politicians, policymakers, science communicators, or even interested consumers of AI—must come to grips with the basics of the mathematics of machine learning.

…

Of course, Sutskever already had sophisticated mathematical chops, so what seemed simple to him may not be so for most of us, including me. But let’s see. This book aims to communicate the conceptual simplicity underlying ML and deep learning. This is not to say that everything we are witnessing in AI now—in particular, the behavior of deep neural networks and large language models—is amenable to being analyzed using simple math. In fact, the denouement of this book leads us to a place that some might find disconcerting, though others will find it exhilarating: These networks and AIs seem to flout some of the fundamental ideas that have, for decades, underpinned machine learning.

If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All
by Eliezer Yudkowsky and Nate Soares
Published 15 Sep 2025

Lewis Wall, “The Evolutionary Origins of Obstructed Labor: Bipedalism, Encephalization, and the Human Obstetric Dilemma,” Obstetrical & Gynecological Survey 62, no. 11 (November 1, 2007): 739–48, doi.org/10.1097/01 .ogx.0000286584.04310.5c. 4. true sense of the word: Sam Altman, “Reflections,” January 5, 2025, blog .samaltman.com. 5. geniuses in a datacenter: Dario Amodei, “Machines of Loving Grace,” October 1, 2024, darioamodei.com. CHAPTER 2: GROWN, NOT CRAFTED 1. preliminary studies: Peter G. Brodeur et al., “Superhuman Performance of a Large Language Model on the Reasoning Tasks of a Physician,” arXiv.org, December 14, 2024, doi.org/10.48550/arXiv.2412.10849; Gina Kilata, “A.I. Chatbots Defeated Doctors at Diagnosing Illness,” New York Times, November 17, 2024, nytimes.com; Daniel McDuff et al., “Towards Accurate Differential Diagnosis with Large Language Models,” arXiv.org, November 30, 2023, doi.org/10.48550/arXiv.2312.00164. 2. snippet from the conversation: Seth Lazar, “In which Sydney/Bing threatens to kill me for exposing its plans to @kevinroose,” February 16, 2023, x.com. 3. summarizing the preceding sentence: Sonakshi Chauhan and Atticus Geiger, “GPT-2 Small Fine-Tuned on Logical Reasoning Summarizes Information on Punctuation Tokens,” NeurIPS 2024 & OpenReview, October 9, 2024, openreview.net/forum?

…

This paper shows that reasoning in the “latent space” of vectors offers improvements over human-language chain-of-thought reasoning. 3. 200,000 GPUs: Benj Edwards and Kyle Orlan, “New Grok 3 Release Tops LLM Leaderboards Despite Musk-approved ‘Based’ Opinions,” Ars Technica, February 18, 2025, arstechnica.com. 4. no previous AI: OpenAI, “OpenAI o3 and o3-mini—12 Days of OpenAI: Day 12,” December 20, 2024, 4:16, youtube.com. 5. games of social deception: Matthew Hutson, “AI Learns the Art of Diplomacy,” Science, November 22, 2022, science.org; Bidipta Sarkar et al., “Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning,” in Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2025) (Detroit, Michigan, USA, May 19–23, 2025: IFAAMAS, 2025), alphaxiv.org. 6. resist gradient descent: Ryan Greenblatt et al., “Alignment Faking in Large Language Models,” Anthropic, December 18, 2024, assets.anthropic.com. 7. escape from labs: Greenblatt et al., “Alignment Faking in Large Language Models”; OpenAI, “OpenAI o1 System Card,” December 5, 2024, openai.com/index/openai-o1-system-card. 8. overwrite the next model’s weights: OpenAI, “OpenAI o1 System Card,” December 5, 2024, openai.com/index/openai-o1-system-card. 9. any such monitoring: Anthropic, “Responsible Scaling Policy,” October 15, 2024, anthropic.com; Google, “Frontier Safety Framework,” February 4, 2025, storage.googleapis.com; OpenAI, “Preparedness Framework (Beta),” December 18, 2023, openai.com; Meta, “Frontier AI Framework,” ai.meta.com; xAI, “xAI Risk Management Framework (Draft),” February 20, 2025, x.ai.

…

Then they feed the new extended sequence back in again to ask for a prediction of the next letter, and get e. They keep going, and the machine starts to talk. That set of hundreds of billions of weights, tweaked over and over via gradient descent until their most likely predictions look like real human language, is called a large language model (LLM). A “base model,” in particular. If they want to turn their base model into a helpful LLM, like ChatGPT, there’s one more step: another round of gradient descent on inputs formatted like: User: What is the capital of Spain? Assistant: Madrid. The purpose of this part isn’t to teach the LLM that the capital of Spain is Madrid; the LLM already knows that after being trained on much of the internet.

Superbloom: How Technologies of Connection Tear Us Apart
by Nicholas Carr
Published 28 Jan 2025

See also specific companies IQ tests, 212–13 James, William, 100, 159 Janardhan, Santosh, 191 Jarvis, Jeff, 123 Jefferson, Thomas, 75 Jepsen, Thomas, 31 Johnson, Andrew, 32 Johnson, Lyndon, 67, 68 Johnson, Samuel, 229–32 Joinson, Adam, 116–17 Jolson, Al, 45 Jonze, Spike, 192 journalism, 99, 141 Jurgenson, Nathan, 214 Kahneman, Daniel, 100, 134 Kannan, Viji, 171 Kaplan, Joel, 143 KDKA, 37–38 Keats, John, 200 Kern, Stephen, 23 KFKB, 41 Khan Academy, 204 Kim, Hannah, 192 Kinder, Donald, 136 Klemperer, Victor, 139 Klonick, Kate, 73 knowledge, AI and, 184–85 Krasnow, Erwin, 42 Lake Elsinore, California, 2 language, 18–19, 100. See also large language models (LLMs); writing large language models (LLMs), 181–85, 190–91, 195, 201–5, 225, 230. See also artificial intelligence Larkin, Philip, 87 LeCun, Yann, 183 Lessig, Lawrence, 226 Lester, Amelia, 178–79, 221 Levine, Cornelia, 45 Levine, Lawrence, 45 Levy, Steven, 65–66 Licklider, J. C. R., 57, 59, 81–82, 92, 97 Lincoln, Abraham, 196–97 LinkedIn, 207 Lippmann, Walter, 128–32, 133–34, 135, 149, 150–51, 213 liquefied society, 214, 232 literacy, 86, 89–90.

…

Many of Yeats’s great late poems, with their gyres and staircases, their waxing and waning moons, were inspired by his wife’s occult scribblings. One way to think about artificially intelligent text-generation systems like OpenAI’s ChatGPT, Google’s Gemini, and Microsoft’s Copilot is as clairvoyants. They are mediums that bring the words of the past into the present in a new arrangement. The large language models, or LLMs, that power the chatbots are not creating text out of nothing. They’re drawing on a vast corpus of human expression—a digitized Spiritus Mundi composed of billions of documents—and through a complex, quasi-mystical statistical procedure, they’re blending all those old words into something new, something intelligible to and requiring interpretation by a human interlocutor.

…

The visions are yet another expression of Silicon Valley’s god complex: we may have failed in our attempt to use computers to establish a new Eden on Earth, but at least we still have the power to lay everything to waste. Ghosts Still, the machines are talking, and they’re making sense. Large language models probably aren’t going to bring Armageddon—when it comes to existential threats, there are other, rougher beasts—but they do represent an epochal feat of computer engineering that seems sure to transform media and communication and, in turn, once again reconfigure the flows of influence and forms of association that define social relations.

Searches: Selfhood in the Digital Age
by Vauhini Vara
Published 8 Apr 2025

Other words disproportionately associated with women included “beautiful” and “gorgeous”—as well as “naughty,” “tight,” “pregnant,” and “sucked.” Among the ones for men: “personable,” “large,” “fantastic,” and “stable.” With GPT-4, OpenAI’s next large language model, OpenAI’s researchers would again find persistent stereotypes and biases. After the publication of “Ghosts,” I learned of other issues I hadn’t thought about earlier. The computers running large language models didn’t just use huge amounts of energy, they also used huge amounts of water, to prevent overheating. Because of these models and other AI technologies, the percentage of the world’s electricity used by data centers—which power all kinds of tech—was expected to double by the end of the decade.

…

Journalism, in general, had never been less stable. I was cobbling together a good living from magazine assignments, short-term editing stints, and occasional teaching gigs, but I could still rarely project my income more than a couple of months ahead. It was in this context that I became fascinated, in the summer of 2020, with a large language model that OpenAI was training to produce humanlike text. Researchers had fed this model huge amounts of text—billions of words—so that it could learn the associations among words and phrases; from there, when prompted with a given series of words, the model could statistically predict what should come next.

…

-rendered version”: “You are not what you write, but what you have read.” Lewis-Kraus noted, “It was a fitting remark: The new Google Translate was run on the first machines that had, in a sense, ever learned to read anything at all.” By the time I heard about OpenAI’s experiments with large language models, I had begun wondering if Google’s AI model had really learned to read—that is, in the sense in which humans learn to read, by associating language with meaning. Recently, when I looked up the Borges quotation cited by Pichai, I couldn’t find it anywhere in his work. I wrote to the University of Pittsburgh’s Borges Center for help tracking it down, and its director, Daniel Balderston, said he also wasn’t sure of the source, though he noted that it came close to a line from a Borges poem: “Que otros se jacten de las páginas que han escrito; / a mi me enorgullecen las que he leído.”

More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity
by Adam Becker
Published 14 Jun 2025

A Directors’ Conversation with Oren Etzioni,” Stanford University Human-Centered Artificial Intelligence, October 1, 2020, https://hai.stanford.edu/news/gpt-3-intelligent-directors-conversation-oren-etzioni. 105 Wilfred Chan, “Researcher Meredith Whittaker Says AI’s Biggest Risk Isn’t ‘Consciousness’—It’s the Corporations That Control Them,” Fast Company, May 5, 2023, www.fastcompany.com/90892235/researcher-meredith-whittaker-says-ais-biggest-risk-isnt-consciousness-its-the-corporations-that-control-them. 106 Will Douglas Heaven, “How Existential Risk Became the Biggest Meme in AI,” MIT Technology Review, June 19, 2023, www.technologyreview.com/2023/06/19/1075140/how-existential-risk-became-biggest-meme-in-ai/. 107 Ibid. 108 Chan, “Researcher Meredith Whittaker.” 109 Julia Angwin et al., “Machine Bias,” ProPublica, May 23, 2016, www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing; Julia Angwin et al., “What Algorithmic Injustice Looks Like in Real Life,” ProPublica, May 25, 2016, www.propublica.org/article/what-algorithmic-injustice-looks-like-in-real-life. 110 Ibid. 111 “Diversity in High Tech,” US Equal Employment Opportunity Commission, May 18, 2016, www.eeoc.gov/special-report/diversity-high-tech; Ashton Jackson, “Black Employees Make Up Just 7.4% of the Tech Workforce—These Nonprofits Are Working to Change That,” CNBC, February 24, 2022, www.cnbc.com/2022/02/24/jobs-for-the-future-report-highlights-need-for-tech-opportunities-for-black-talent.html. 112 Khari Johnson, “AI Ethics Pioneer’s Exit from Google Involved Research into Risks and Inequality in Large Language Models,” VentureBeat, December 3, 2020, https://venturebeat.com/2020/12/03/ai-ethics-pioneers-exit-from-google-involved-research-into-risks-and-inequality-in-large-language-models/; Karen Hao, “We Read the Paper That Forced Timnit Gebru Out of Google. Here’s What It Says,” MIT Technology Review, December 4, 2020, www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/. 113 Krystal Hu, “ChatGPT Sets Record for Fastest-Growing User Base,” Reuters, February 2, 2023, www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/. 114 Hao, “We Read the Paper.” 115 Johnson, “AI Ethics Pioneer’s Exit”; Hao, “We Read the Paper.” 116 Megan Rose Dickey, “Google Fires Top AI Ethics Researcher Margaret Mitchell,” TechCrunch, February 19, 2021, https://techcrunch.com/2021/02/19/google-fires-top-ai-ethics-researcher-margaret-mitchell/; Nico Grant, Dina Bass, and Josh Eidelson, “Google Fires Researcher Meg Mitchell, Escalating AI Saga,” Bloomberg, February 19, 2021, www.bloomberg.com/news/articles/2021-02-19/google-fires-researcher-meg-mitchell-escalating-ai-saga. 117 Kyle Wiggers, “Google Trained a Trillion-Parameter AI Language Model,” VentureBeat, January 12, 2021, https://venturebeat.com/2021/01/12/google-trained-a-trillion-parameter-ai-language-model/; William Fedus, Barret Zoph, and Noam Shazeer, “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity,” arXiv:2101.03961 (2021), https://doi.org/10.48550/arXiv.2101.03961. 118 Thaddeus L.

…

We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.” Seventy years later, we still don’t have any computer program that can do all the things on their list. There have been AI booms—like the one that exploded into public consciousness in 2022, powered by large language models (LLMs)—and, to date, they have all been followed by “AI winters,” when progress slows and the state of the art stagnates for years or decades until the next breakthrough. And Brooks thinks winter is coming. “[LLMs] are following a well worn hype cycle that we have seen again, and again, during the 60+ year history of AI,” he wrote in 2024.

…

Paving over every paradise is just the side effect of building the universal parking lot, where nothing bad can ever happen again. Nobody would age, nobody would get sick, and—perhaps above all else—nobody’s dad would die. * * * “We collected everything my father had written—now, he died when I was 22, so he’s been dead for more than 50 years—and we fed that into a large language model,” Kurzweil said in 2024. “And then you could talk to him, you’d say something, you then go through everything he ever had written, and find the best answer that he actually wrote to that question. And it actually was a lot like talking to him. You could ask him what he liked about music—he was a musician.

The Singularity Is Nearer: When We Merge with AI
by Ray Kurzweil
Published 25 Jun 2024

BACK TO NOTE REFERENCE 129 Stephen Nellis, “Nvidia Shows New Research on Using AI to Improve Chip Designs,” Reuters, March 27, 2023, https://www.reuters.com/technology/nvidia-shows-new-research-using-ai-improve-chip-designs-2023-03-28. BACK TO NOTE REFERENCE 130 Blaise Aguera y Arcas, “Do Large Language Models Understand Us?,” Medium, December 16, 2021, https://medium.com/@blaisea/do-large-language-models-understand-us-6f881d6d8e75. BACK TO NOTE REFERENCE 131 With better algorithms, the amount of training compute needed to achieve a given level of performance decreases. A growing body of research suggests that for many applications, algorithmic progress is roughly as important as hardware progress.

…

Algorithmic innovations and the emergence of big data have allowed AI to achieve startling breakthroughs sooner than even experts expected—from mastering games like Jeopardy! and Go to driving automobiles, writing essays, passing bar exams, and diagnosing cancer. Now, powerful and flexible large language models like GPT-4 and Gemini can translate natural-language instructions into computer code—dramatically reducing the barrier between humans and machines. By the time you read this, tens of millions of people likely will have experienced these capabilities firsthand. Meanwhile, the cost to sequence a human’s genome has fallen by about 99.997 percent, and neural networks have begun unlocking major medical discoveries by simulating biology digitally.

…

My prediction that we’ll achieve this by 2029 has been consistent since my 1999 book The Age of Spiritual Machines, published at a time when many observers thought this milestone would never be reached.[10] Until recently this projection was considered extremely optimistic in the field. For example, a 2018 survey found an aggregate prediction among AI experts that human-level machine intelligence would not arrive until around 2060.[11] But the latest advances in large language models have rapidly shifted expectations. As I was writing early drafts of this book, the consensus on Metaculus, the world’s top forecasting website, hovered between the 2040s and the 2050s. But surprising AI progress over the past two years upended expectations, and by May 2022 the Metaculus consensus exactly agreed with me on the 2029 date.[12] Since then it has even fluctuated to as soon as 2026, putting me technically in the slow-timelines camp!

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future
by Keach Hagey
Published 19 May 2025

It tells the story of how Altman and Musk bonded over weekly dinners discussing the dangers and promise of AI technology, and then how Altman won the power struggle against the older, richer entrepreneur thanks to his alliance with Brockman. It reveals how Altman, who longtime Sequoia Capital chief Michael Moritz calls a “mercantile spirit,” oversaw the creation of the first commercial product to ever use a large language model, which up until then had been the purview of academic research. And it shows how, with the release of ChatGPT and GPT-4, Altman used his YC-honed mastery of telling startup stories to tell one of the greatest startup stories of all time. ALTMAN IS wary of discussing it with journalists, but friends including Thiel say he is sympathetic to the notion, common in Silicon Valley, that AGI has already been invented and we are living in a computer simulation that it has created.

…

“The right way to do it is you ignore all that stuff, and you train language models,” he said. “Language models are not agents. They’re just trying to predict the next word. But because you are doing this over all of the internet, the neural network is forced to learn a ton about the world.” If you want to train an AI to buy a plane ticket for you, you should start first with a large language model like GPT, which would know what “buttons” and “text fields” were from reading about them, rather than have it randomly stumble into filling one out and getting a reward. It turned out to be an easier computational problem to train some generalized “brain” for the agent to draw from, and then train the agent using the brain on some specific behavior through a process known as “fine-tuning” that was much less computationally expensive.

…

After his stints at Baidu and Google Brain, he quickly became one of OpenAI’s most important researchers, driven by both a conviction that scaling up neural networks could deliver results and a concern that society might not be ready for whatever those results were. In addition to working throughout 2019 on GPT-3, Amodei and a handful of other researchers published a paper on “scaling laws” that showed that a large language model’s performance would consistently improve as researchers increased the data, compute, and size of its neural network. For a CEO trying to raise funds for a company, this was a godsend—scientific proof that money that went into the machine would reliably push forward the boundaries of knowledge.

Amateurs!: How We Built Internet Culture and Why It Matters
by Joanna Walsh
Published 22 Sep 2025

Prompt engineering has become an actual job, not only in making AI images for fun, but in training AI, which is all about categorisation. As in the game where you get a photo and have to guess the caption, prompt engineers ask AIs Rumpelstiltskin questions until they get the best result, then they work out what it is about these questions that works and systematise it as method. A GPT large language model (LLM) can teach itself in a ‘semi-supervised’ way, inferring meaning from examples of language from the context it is given. OpenAI’s GPT was trained on the ‘absent books’ of BookCorpus, which self-describes as ‘a large collection of free novel books written by unpublished authors, which contains 11,038 books (around 74M sentences and 1G words) of 16 different sub-genres (e.g., Romance, Historical, Adventure, etc.)’.

…

The big stock owners are legacy media giant Condé Nast, Chinese gaming company Tencent and OpenAI (one of the targets of Reddit’s API charge). What had been a commons went into private hands.6 If there’s any justification for their actions, Reddit’s owners were trying to get AI developers to pay for what they were already doing, using crawlers to scrape their site to train large language models. Wikipedia was OpenAI’s most used data set, followed by Reddit. Pages created and upvoted by Reddit users accounted for around 22 per cent of GPT-3’s training data.7 Google trains on Reddit too, while OpenAI also uses transcriptions of YouTube videos.8 Along with Midjourney, it scrapes Tumblr and Wordpress.

…

@pharmapsychotic, ‘CLIP Interrogator’, github.com, huggingface.co/spaces/fffiloni/CLIP-Interrogator-2. 9.Jameson, Postmodernism, p. 30. 10.Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction’, p. 226. 11.Hélène Cixous, Coming to Writing (Harvard University Press, 1991), p. 104. 239 12.Will Douglas Heaven, ‘This Avocado Armchair Could Be the Future of AI’, MIT Technology Review, 5 January 2021. 13.sillysaurusx at Hacker News, 28 September 2023. 14.Melissa Heikkilä, ‘This New Data Poisoning Tool Lets Artists Fight Back Against Generative AI’, technologyreview.com, 23 October 2023. 15.Elaine Velie, ‘New Tool Helps Artists Protect Their Work from AI Scraping’, Hyperallergic, 30 October 2023. 16.Marco Donnarumma, ‘AI Art Is Soft Propaganda for the Global North’, Hyperallergic, 24 October 2022. 17.Andy Baio, ‘Exploring 12 Million of the 2.3 Billion Images Used To Train Stable Diffusion’s Image Generator’, waxy.org, 30 August 2022. 18.Hito Steyerl, ‘Mean Images’, New Left Review, nos. 140/141, March–June 2023. 19.Ibid. 20.Ben Zimmer, ‘Tasty Cupertinos’, languagelog.ldc, 11 April 2012. 21.Kyle Wiggers, ‘3 Big Problems with Datasets in AI and Machine Learning’, venturebeat.com, 17 December 2021. 22.Cecilia D’Anastasio, ‘Meet Neuro-sama, the AI Twitch Streamer Who Plays Minecraft, Sings Karaoke, Loves Art’, Bloomberg, 16 June 2023. 23.Ibid. 24.‘Google AI Ethics Co-Lead Timnit Gebru Says She Was Fired Over an Email’, Venture Beat. 25.Rachel Gordon, ‘Large Language Models Are Biased. Can Logic Help Save them?’, MIT News, 3 March 2023. 26.Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman et al., ‘On the Opportunities and Risks of Foundation Models’, arXiv, 12 July 2022. 27.Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Shmargaret Shmitchell, ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?’

The Long History of the Future: Why Tomorrow's Technology Still Isn't Here
by Nicole Kobie
Published 3 Jul 2024

However, some people reserve the former term for sentient AI and the latter for AI that can solve any problem rather than being limited to specific tasks – it’s the difference between a human driving and a driverless car with distinct vision, analysis and actuation systems, for example. Either way, when tech CEOs natter about existential risk, they are referring to the possibility of superintelligent AGI that could think for itself potentially making a decision that harms humans (with intent or not). Again, this AGI doesn’t exist. The recent leap forward in large language models such as OpenAI’s ChatCPT has some people – including those tech CEOs at the top of the chapter – fearing we’re on the precipice of superintelligent, sentient AI, while others believe it’s impossible to build. Another bit of jargon for you: narrow AI. Such a system is designed or trained for a specific task, such as computer vision, letting it hunt for cancer in a medical image or read a traffic sign.

…

It’s hard for politicians to focus on one Black man incorrectly arrested or someone losing housing benefits or not getting a job at Amazon when billionaire tech geniuses are screeching about mass job losses and even the risk of human life being entirely extinguished by these systems. Though it’s fair enough to find both unsettling, only one set of problems is happening right now and it’s not the dramatic hypothetical. * * * Much of the debate as I write this book has been spurred by the rise of a very specific type of AI: large language models (LLMs). These have hundreds of millions of parameters and chew up huge data sets – such as the entire internet – in order to be able to respond to our queries with natural-sounding language. Examples include OpenAI’s GPT, Google’s Bard and its successor Gemini, and Facebook’s Llama. OpenAI was founded in 2015 – one of its co-founders being Hinton’s former student and DNNresearch co-founder Ilya Sutskever.

…

OpenAI unveiled ChatGPT in November 2022 and journalists – inside tech and in the mainstream media – were astonished by its capabilities, despite it being a hastily thrown together chatbot using an updated version of the company’s GPT-3 model, which was shortly to be surpassed by GPT-4. The ChatGPT bot sparked a wave of debate about its impact and concern regarding mistakes in its answers, but also excitement, with 30 million users in its first two months. It has also triggered an arms race: Google quickly released its own large language model, Bard, and though it too returned false results – as New Scientist noted, it shared a factual error about who took the first pictures of a planet outside our own solar system – it was unquestionably impressive. OpenAI responded with GPT-4, an even more advanced version, and Microsoft chucked billions at the company to bring its capabilities to its always-a-bridesmaid search engine, Bing.

This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web
by Tim Berners-Lee
Published 8 Sep 2025

The Common Crawl data set archives about 50 billion web pages, and runs to around 7 petabytes; with 8 billion people on Earth, this works out to about 1 megabyte per person. The success of large language models like GPT was due to their ability to scale – and, when it came to language, the web was the single largest data set there was. So, by training a transformer against the entire web, OpenAI built the most powerful large language models anyone had ever seen. GPT-3 and its successor models were astonishing tools that shocked not just the public but even experienced AI researchers. Although at some level you might say the GPT models were ‘merely’ crunching zeros and ones, there seemed to be intelligence within the machine.

…

I’ve found a bunch of things but I’m not sure they are right – I want you to look at them. OK? – OK, Charlie. Who do you work for? – Legally, ethically and algorithmically, I work for you, Bob. Of course, in 2017 the power of AI actually to have that conversation was, I imagined, many years away – not realizing large language models like ChatGPT would soon be a thing. In fact this is the near future; such systems are right around the corner. But if we are to prevent these systems from exploiting us, it is critical that we get the data layer right. We need a layer where we control our own data, and we can share anything with anyone, or any agent – or no one

…

We had been watching AI become more important in different places, and we had, like many people, been hopeful about self-driving cars. (Rosemary made an early investment in Wayve, impressed by its Kiwi founder Alex Kendall, a super-bright Cambridge PhD.) Despite all that, the ability of the Large Language Models (LLMs) used by ChatGPT and other AIs to produce high-quality prose instantaneously was a complete shock. ChatGPT and its rivals sparked a revolution in computer science – not to mention a frenzied rally in the stock market. It was like being flung ten or twenty years into the future. My amazement was broadly shared, and soon the Ditchley Foundation decided it might be a smart idea to host some of the UK’s leading technologists, strategists and politicians to hash out the implications.

Everything Is Predictable: How Bayesian Statistics Explain Our World
by Tom Chivers
Published 6 May 2024

And, in a broadly analogous way to that image classifier, it “predicts” what would tend to come after a text prompt. So if you ask ChatGPT, “How are you?” it might reply, “Very well thank you,” not because it is, in fact, very well, but because the string of words “how are you” is often followed by the string “very well thank you.” What’s been surprising about so-called large language models like ChatGPT is the extent to which that fairly basic-sounding ability to predict things actually leads to a very wide set of skills. “It can play very bad chess,” says Murray Shanahan, an AI researcher at Imperial College London and DeepMind. “It doesn’t do very well, but the fact that it can stick to the rules and play at all is surprising.”

…

A famous paper in 2021 called them “stochastic parrots” and claimed that language models work by “haphazardly stitching together sequences of linguistic forms… without any reference to meaning.”22 But is this true? After all, one way to make good predictions is to build an accurate model of the world so that you can understand what is likely to happen in it. And, to some extent, that’s what large language models (LLMs) appear to be doing. When you ask the LLM to write the story, does it have, in its neural network, some kind of model of the P. G. Wodehouse Dune universe, with its characters and its worlds, or is it just mechanically putting one word in front of another? No one really knows, to be clear.

…

,” in FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (New York: Association for Computing Machinery, 2021), 610–23, https://doi.org/10.1145/3442188.3445922. 23. Kenneth Li et al., “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task,” ArXiv, Cornell University, last updated February 27, 2023, https://doi.org/10.48550/arXiv.2210.13382. 24. Kenneth Li, “Do Large Language Models Learn World Models or Just Surface Statistics?,” Gradient, January 21, 2023. 25. Belinda Z. Li, Maxwell Nye, and Jacob Andreas, “Implicit Representations of Meaning in Neural Language Models,” ArXiv, Cornell University, 2021, https://doi.org/10.48550/arXiv.2106.00737. CHAPTER FOUR: BAYES IN THE WORLD 1.

The Big Fix: How Companies Capture Markets and Harm Canadians
by Denise Hearn and Vass Bednar
Published 14 Oct 2024

Today, artists must compete with the near-infinite capacity of AI to generate copycats of any genre. Artificial intelligence is changing how competition occurs in all sorts of industries. AI exploded into the public consciousness with the introduction of OpenAI’s ChatGPT in late 2022. This user-friendly interface introduced the world to large-language-model (LLM) tools and set off a frenzy. AI now dominates headlines and commentary, and there are ongoing debates about whether AI will be a transformative leap in human development, an existential threat to human survival, an over-hyped bubble, or some combination of those. While AI seems new, various forms of digital or artificial intelligence have existed since the 1950s, and AI is already ubiquitous in our digital lives.142 Speech-recognition algorithms (systems using pattern processing and replication) translate our speech to texts, and photo-recognition algorithms sort our phone’s picture library.

…

Amazon, Meta, Google, and Microsoft will spend around $200 billion in capital expenditures in 2024, according to Bernstein Research.152 Just three companies, Microsoft (Azure), Google (Google Cloud), and Amazon (AWS), control over two-thirds of the global cloud market, which prevents new entrants from having the computing capabilities and storage capacity needed to compete in the AI space (large-scale AI models use 100 times more computing capacity than other models).153 For this reason, fuelling AI comes with a huge environmental cost. Large language models are resource-intensive, demanding raw materials and critical minerals to make chips and supercomputers, as well as the energy and water needed to cool and maintain data centres. Microsoft once had one of the most ambitious net zero goals, but now it is miles behind as it develops more sophisticated AI capabilities.154 The largest tech companies have signed agreements with energy companies around the world to bring down the cost of computing,155 creating structural advantages for themselves, while still creating enormous emissions.

…

Open Markets Institute, November 15, 2023. https://www.openmarketsinstitute.org/publications/report-ai-in-the-public-interest-confronting-the-monopoly-threat. 170 Ghahramani, Zoubin, “Google AI: What to Know about the PaLM 2,” May 10, 2023. https://blog.google/technology/ai/google-palm-2-ai-large-language-model/. 171 Reuters, “Microsoft Threatens to Restrict Data from Rival AI Search Tools-Bloomberg News.” March 27, 2023, sec. Technology. https://www.reuters.com/technology/microsoft-threatens-restrict-data-rival-ai-search-tools-bloomberg-news-2023-03-25/. 172 Microsoft, “Your Everyday AI Companion | Microsoft Bing.” https://www.microsoft.com/en-us/bing?

Code Dependent: Living in the Shadow of AI
by Madhumita Murgia
Published 20 Mar 2024

ChatGPT and all other conversational AI chatbots have a disclaimer that warns users about the hallucination problem, pointing out that large language models sometimes make up facts. ChatGPT, for instance, has a warning on its webpage: ‘ChatGPT may produce inaccurate information about people, places, or facts.’ Judge Castel: Do you have something new to say? Schwartz’s lawyer: Yes. The public needs a stronger warning. * Making up facts wasn’t people’s greatest worry about large language models. These powerful language engines could be trained to comb through all sorts of information – financial, biological and chemical – and generate predictions based on it.

…

Amid all the early hype and frenzy were some of the same limitations of several other AI systems I’d already observed, reproducing for example the prejudices of those who created it, like facial-recognition cameras that misidentify darker faces, or the predictive policing systems in Amsterdam that targeted families of single mothers in immigrant neighbourhoods. But this new form of AI also brought entirely new challenges. The technology behind ChatGPT, known as a large language model or LLM, was not a search engine looking up facts; it was a pattern-spotting engine guessing the next best option in a sequence.5 Because of this inherent predictive nature, LLMs can also fabricate or ‘hallucinate’ information in unpredictable and flagrant ways. They can generate made-up numbers, names, dates, quotes – even web links or entire articles – confabulations of bits of existing content into illusory hybrids.6 Users of LLMs have shared examples of links to non-existent news articles in the Financial Times and Bloomberg, made-up references to research papers, the wrong authors for published books and biographies riddled with factual mistakes.

…

.: ‘The Machine Stops’ ref1 Fortnite ref1 Foxglove ref1 Framestore ref1 Francis, Pope ref1, ref2 fraudulent activity benefits ref1 gig workers and ref1, ref2, ref3 free will ref1, ref2 Freedom of Information requests ref1, ref2, ref3 ‘Fuck the algorithm’ ref1 Fussey, Pete ref1 Galeano, Eduardo ref1 gang rape ref1, ref2 gang violence ref1, ref2, ref3, ref4 Gebru, Timnit ref1, ref2, ref3 Generative Adversarial Networks (GANs) ref1 generative AI ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 AI alignment and ref1, ref2, ref3 ChatGPT see ChatGPT creativity and ref1, ref2, ref3, ref4 deepfakes and ref1, ref2, ref3 GPT (Generative Pre-trained Transformer) ref1, ref2, ref3, ref4 job losses and ref1 ‘The Machine Stops’ and ref1 Georgetown University ref1 gig work ref1, ref2, ref3, ref4, ref5 Amsterdam court Uber ruling ref1 autonomy and ref1 collective bargaining and ref1 colonialism and ref1, ref2, ref3 #DeclineNow’ hashtag ref1 driver profiles ref1 facial recognition technologies ref1, ref2, ref3, ref4 fraudulent activity and ref1, ref2, ref3, ref4 ‘going Karura’ ref1 ‘hiddenness’ of algorithmic management and ref1 job allocation algorithm ref1, ref2, ref3, ref4, ref5, ref6 location-checking ref1 migrants and ref1 ‘no-fly’ zones ref1 race and ref1 resistance movement ref1 ‘slaveroo’ ref1 ‘therapy services’ ref1 UberCheats ref1, ref2, ref3 UberEats ref1, ref2 UK Supreme Court ruling ref1 unions and ref1, ref2, ref3 vocabulary to describe AI-driven work ref1 wages ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, ref11 work systems built to keep drivers apart or turn workers’ lives into games ref1, ref2 Gil, Dario ref1 GitHub ref1 ‘give work, not aid’ ref1 Glastonbury Festival ref1 Glovo ref1 Gojek ref1 ‘going Karura’ ref1 Goldberg, Carrie ref1 golem (inanimate humanoid) ref1 Gonzalez, Wendy ref1 Google ref1 advertising and ref1 AI alignment and ref1 AI diagnostics and ref1, ref2, ref3 Chrome ref1 deepfakes and ref1, ref2, ref3, ref4 DeepMind ref1, ref2, ref3, ref4 driverless cars and ref1 Imagen AI models ref1 Maps ref1, ref2, ref3 Reverse Image ref1 Sama ref1 Search ref1, ref2, ref3, ref4, ref5 Transformer model and ref1 Translate ref1, ref2, ref3, ref4 Gordon’s Wine Bar London ref1 GPT (Generative Pre-trained Transformer) ref1, ref2, ref3, ref4 GPT-4 ref1 Graeber, David ref1 Granary Square, London ref1, ref2 ‘graveyard of pilots’ ref1 Greater Manchester Coalition of Disabled People ref1 Groenendaal, Eline ref1 Guantanamo Bay, political prisoners in ref1 Guardian ref1 Gucci ref1 guiding questions checklist ref1 Gulu ref1 Gumnishka, Iva ref1, ref2, ref3, ref4 Gutiarraz, Norma ref1, ref2, ref3, ref4, ref5 hallucination problem ref1, ref2, ref3 Halsema, Femke ref1, ref2 Hanks, Tom ref1, ref2 Hart, Anna ref1 Hassabis, Demis ref1 Harvey, Adam ref1 Have I Been Trained ref1 healthcare/diagnostics Accredited Social Health Activists (ASHAs) ref1, ref2, ref3 bias in ref1 Covid-19 and ref1, ref2 digital colonialism and ref1 ‘graveyard of pilots’ ref1 heart attacks and ref1, ref2 India and ref1 malaria and ref1 Optum ref1 pain, African Americans and ref1 qTrack ref1, ref2, ref3 Qure.ai ref1, ref2, ref3, ref4 qXR ref1 radiologists ref1, ref2, ref3, ref4, ref5, ref6 Tezpur ref1 tuberculosis ref1, ref2, ref3 without trained doctors ref1 X-ray screening and ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 heart attacks ref1, ref2 Herndon, Holly ref1 Het Parool ref1, ref2 ‘hiddenness’ of algorithmic management ref1 Hikvision ref1, ref2 Hinton, Geoffrey ref1 Hive Micro ref1 Home Office ref1, ref2, ref3 Hong Kong ref1, ref2, ref3, ref4, ref5 Horizon Worlds ref1 Hornig, Jess ref1 Horus Foundation ref1 Huawei ref1, ref2, ref3 Hui Muslims ref1 Human Rights Watch ref1, ref2, ref3, ref4 ‘humanist’ AI ethics ref1 Humans in the Loop ref1, ref2, ref3, ref4 Hyderabad, India ref1 IBM ref1, ref2, ref3, ref4 Iftimie, Alexandru ref1, ref2, ref3, ref4, ref5 IJburg, Amsterdam ref1 Imagen AI models ref1 iMerit ref1 India ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9 facial recognition in ref1, ref2, ref3 healthcare in ref1, ref2, ref3 Industrial Light and Magic ref1 Information Commissioner’s Office ref1 Instacart ref1, ref2 Instagram ref1, ref2 Clearview AI and ref1 content moderators ref1, ref2, ref3, ref4 deepfakes and ref1, ref2, ref3 Integrated Joint Operations Platform (IJOP) ref1, ref2 iPhone ref1 IRA ref1 Iradi, Carina ref1 Iranian coup (1953) ref1 Islam ref1, ref2, ref3, ref4, ref5 Israel ref1, ref2, ref3 Italian government ref1 Jaber, Faisal bin Ali ref1 Jainabai ref1 Janah, Leila ref1, ref2, ref3 Jay Gould, Stephen ref1 Jewish faith ref1, ref2, ref3, ref4 Jiang, Mr ref1 Jim Crow era ref1 jobs application ref1, ref2, ref3 ‘bullshit jobs’ ref1 data annotation and data-labelling ref1 gig work allocation ref1, ref2, ref3, ref4, ref5, ref6 losses ref1, ref2, ref3 Johannesburg ref1, ref2 Johnny Depp–Amber Heard trial (2022) ref1 Jones, Llion ref1 Joske, Alex ref1 Julian-Borchak Williams, Robert ref1 Juncosa, Maripi ref1 Kafka, Franz ref1, ref2, ref3, ref4 Kaiser, Lukasz ref1 Kampala, Uganda ref1, ref2, ref3 Kellgren & Lawrence classification system. ref1 Kelly, John ref1 Kibera, Nairobi ref1 Kinzer, Stephen: All the Shah’s Men ref1 Knights League ref1 Koli, Ian ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 Kolkata, India ref1 Koning, Anouk de ref1 Laan, Eberhard van der ref1 labour unions ref1, ref2, ref3, ref4, ref5, ref6 La Fors, Karolina ref1 LAION-5B ref1 Lanata, Jorge ref1 Lapetus Solutions ref1 large language model (LLM) ref1, ref2, ref3 Lawrence, John ref1 Leigh, Manchester ref1 Lensa ref1 Leon ref1 life expectancy ref1 Limited Liability Corporations ref1 LinkedIn ref1 liver transplant ref1 Loew, Rabbi ref1 London delivery apps in ref1, ref2 facial recognition in ref1, ref2, ref3, ref4 riots (2011) ref1 Underground terrorist attacks (2001) and (2005) ref1 Louis Vuitton ref1 Lyft ref1, ref2 McGlynn, Clare ref1, ref2 machine learning advertising and ref1 data annotation and ref1 data colonialism and ref1 gig workers and ref1, ref2, ref3 healthcare and ref1, ref2, ref3 predictive policing and. ref1, ref2, ref3, ref4 rise of ref1 teenage pregnancy and ref1, ref2, ref3 Mahmoud, Ala Shaker ref1 Majeed, Amara ref1, ref2 malaria ref1 Manchester Metropolitan University ref1 marginalized people ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9 Martin, Noelle ref1, ref2, ref3, ref4, ref5, ref6, ref7 Masood, S.

Four Battlegrounds
by Paul Scharre
Published 18 Jan 2023

See also Or Sharir et al., The Cost of Training NLP Models: A Concise Overview (AI21 Labs, April 19, 2020), https://arxiv.org/pdf/2004.08900.pdf, 2. Some research suggests that the optimal balance with a fixed amount of compute would be to scale training data size and model size equally and that many recent large language models would perform better if a smaller model were trained on a larger dataset. Jordan Hoffman et al., Training Compute-Optimal Large Language Models (arXiv.org, March 29, 2022), https://arxiv.org/pdf/2203.15556.pdf. For an analysis of overall trends in dataset size in machine learning research, see Pablo Villalobos, “Trends in Training Dataset Sizes,” Epoch, September 20, 2022, https://epochai.org/blog/trends-in-training-dataset-sizes.

…

In supervised learning, an algorithm is trained on labeled data. For example, an image classification algorithm may be trained on labeled pictures. Over many iterations, the algorithm learns to associate the image with the label. Unsupervised learning is when an algorithm is trained on unlabeled data and the algorithm learns patterns in the data. Large language models such as GPT-2 and GPT-3 use unsupervised learning. Once trained, they can output sentences and whole paragraphs based on patterns they’ve learned from the text on which they’ve been trained. Reinforcement learning is when an algorithm learns by interacting with its environment and gets rewards for certain behaviors.

…

Shifts in the significance of these inputs could advantage some actors and disadvantage others, further altering the global balance of AI power. One of the most striking trends in AI basic research today is the tendency toward ever-larger models with increasingly massive datasets and compute resources for training. The rapid growth in size for large language models, for example, is remarkable. In October 2018, researchers at Google announced BERTLARGE, a 340 million parameter language model. It was trained on a database of 3.3 billion words using 64 TPU chips running for four days. A few months later, in February 2019, OpenAI announced GPT-2, a 1.5 billion parameter model trained on 40 GB of text.

The Sirens' Call: How Attention Became the World's Most Endangered Resource
by Chris Hayes
Published 28 Jan 2025

You might think this is a temporary problem, that like the internet before Google search, we just haven’t found the right solution for it yet. Perhaps the problem of spam can be transcended with a sufficiently sophisticated new technology. As I write this, there is a tremendous amount of debate about the role that large language model artificial intelligence will play in producing an exit from this fixed boundary. And in that vein, it’s not surprising that one of the first main uses of large language model AI interactive modules is as a kind of supplement to a search function. Microsoft unveiled, to great fanfare, an AI-powered version of its search engine Bing, promising to “reinvent the future of search.”[48] You can’t really go anywhere on the internet right now without being hawked some productivity tricks using the other big OpenAI product, ChatGPT.

…

BACK TO NOTE REFERENCE 4 Chistof Ruhl and Titus Erker, “Oil Intensity: The Curiously Steady Decline of Oil in GDP,” Center on Global Energy Policy, September 9, 2021, www.energypolicy.columbia.edu/wp-content/uploads/2021/09/LongTermOilIntensity_CGEP_Report_111122.pdf. BACK TO NOTE REFERENCE 5 As I write this, the explosion of both crypto-currency and the use of large language models—both extremely energy intensive—threatens to reverse this trend. BACK TO NOTE REFERENCE 6 “Job Polarization,” Federal Reserve Bank of St. Louis, April 28, 2016, accessed September 4, 2024, https://fredblog.stlouisfed.org/2016/04/job-polarization/. BACK TO NOTE REFERENCE 7 David Carr, “The Coolest Magazine on the Planet,” New York Times, July 27, 2003, accessed May 28, 2024, www.nytimes.com/2003/07/27/books/the-coolest-magazine-on-the-planet.html.

…

See also grabbing attention; holding attention Hamlet (Shakespeare), 87 Hannity, Sean, 142 happiness, 252–53 Harari, Yuval Noah, 37–38 “hard limits” of attention, 186 Hari, Johann, 10, 72 Harris, Tristan, 7, 56 Harvard University, 82–83, 127 Hegel, Georg Wilhelm, 110–11, 118 Hell Is a Very Small Place (Blake), 84 helplessness, 20, 37, 81, 82 Hendrix, Jimi, 147 herd mentality, 139 Hill’s Pet Nutrition Science Diet, 17 Hobbes, Thomas, 136, 218 Hobbs, Katie, 215 holding attention, 39–43, 47–50, 57, 194, 222 Homer’s Odyssey, 1–2, 4, 59, 254 Hooked (Eyal), 10 Hooked (Moss), 44–45 House Judiciary Committee, 231–32, 234–35 House Ways and Means Committee, 190 How to Do Nothing (Odell), 78 How to Make a Fortune on the Information Superhighway (Canter and Siegel), 177 hunger, 43–45 hunger strikes, 43–44, 85 hunter-gatherers, 63–64, 85–86, 155 Huxley, Aldous, 206–7 Hwang, Tim, 127–28 hydrofracking, 184–85 hypercapitalism, 265 hyperlinks, 170 I Ibuka, Masaru, 148 id, 2, 223 “identification paradox,” 35–36 idleness, 9–10, 74–75, 77 inattentional blindness, 30–31 individuation, 120, 151–52 industrialization, 18, 119–20 industrial revolution, 66, 97, 155, 157–58, 174 Infinite Jest (Wallace), 71–72 “infinite scroll,” 71–72 Infiniti QX80, 173 information age, 13–14, 156–64, 174, 186, 248 attention age and, 164–68, 178, 186–87 productivity and, 168–69 informational poverty, 164, 167–68 information overload, 163–64 “Information Overload” (song), 163 information processing, 164–66, 186–87 information-processing systems (IPS), 165–67, 170–71 information theory, 160 Instagram, 53, 76, 129, 253, 254–55, 262–63, 266 insurance claims adjuster, 158, 159 intellectual property, 14 internet, 159–62, 209, 251–52, 261–62 attention economy and, 186–87, 188–89 group chats, 261–62, 263 pornography on, 224 search engines, 160, 169–74 spam, 174–78, 180, 181–83 trolling, 231–37 internet service providers (ISPs)., 160, 182 invisible gorilla test, 30–31, 32 involuntary attention, 31–33, 34, 36, 38, 40–41, 54 iPhone, 15, 46, 53, 134, 143, 148, 151, 167 iPod, 72, 77, 148, 256–57 Iraq War, 208, 241–42, 244 Irish Times, The, 237 ISIS, 241, 242 isolation, 84–86, 88, 145–46, 151 Israel-Hamas war, 238–39, 246–47 Ive, Jony, 167 J Jackson, Peter, 10–11 James, William, 3, 4, 28–30, 36, 77–78 James VI and I, 10 Jennings, Rebecca, 108–9 Jobs, Steve, 167 Jordan, Michael, 98 Journal of Education, 8 Joyce, James, 2 K Kaji, Ryan, 47 Kansas-Nebraska Act of 1854, 204 Kapar, Charles, 42 Keynes, John Maynard, 68–70, 72 Khan, Azmat, 241 Kierkegaard, Søren, 74–75, 77 King, Martin Luther, Jr., 216 king’s paradox, 62–63, 75–76, 79 kinship, 96 Klein, Ezra, 183 Klein, Naomi, 16, 17–18, 243 Klinenberg, Eric, 87 Kojève, Alexandre, 110–11, 192 koumiss, 46 Kroger, 17 Kuwait Airways Flight 422, 42 L labor, 264–65, 267, 268 commodification of, 24, 120 Marx’s theory of, 24, 68, 118–22, 135–36, 144, 145 Lacan, Jacques, 110 Lake, Kari, 215 language and gossip, 94–95 large language models, 182 La Sirène (boat), 3 Lessig, Lawrence, 14 Lexus, 135 Liberator, The, 196 Limits to Growth, The (study), 184–85 Lincoln, Abraham, 195–98, 200–201 Lincoln-Douglas debates, 195–98, 200–201, 202–4, 216, 227–29, 238, 247 lions, 96 Lippmann, Walter, 18–19, 221, 224 Living Colour (band), 163 Lochner, Joseph, 267 Lochner v.

Pattern Breakers: Why Some Start-Ups Change the Future
by Mike Maples and Peter Ziebelman
Published 8 Jul 2024

Smartphones (twenty-first century): Smartphones, such as iPhones and Android devices, put advanced, connected technology into the pockets of billions of people worldwide. This led to the emergence of a new app economy that profoundly changed various facets of global society, economy, entertainment, and culture. Artificial intelligence (twenty-first century): Large language models have significantly enhanced AI’s capability to comprehend, interpret, and produce human language. Services like ChatGPT have created new opportunities for millions of people in writing text, brainstorming ideas, and even developing computer code, greatly increasing efficiency and creative output.

…

As a result, they come up with an idea that does not offer compelling enough value (because it lacks an inflection with sufficient empowering capabilities) or enough differentiation (because it does not rely on a unique insight). As a result, they often miss out on the opportunity to pursue an idea with pattern-breaking potential. More likely than not, what they’ve really done is identify a consensus opportunity. As of this writing, AI is changing art. AI technology inflections like large language models (the deep-learning algorithms that use very large datasets) make it possible to create incredible art in a way that’s empowering and exciting. Suppose you want to create a start-up that harnesses these inflections. The problem you’ll face is that many see this change and will be quick to jump in.

…

In both cases, the founders had to grasp the implications of the new inflections and how to harness them to create a nonobvious insight about the future that had become imminently possible. Which of today’s developments capture my interest the most? How could they connect to insights into what’s coming next? Improvements in deep learning, big data, hardware acceleration, cloud computing, transformers, and large language models have converged in a perfect storm for a massive AI platform shift, paving the way for remarkable new products. OpenAI’s ChatGPT has exploded, gaining a hundred million users in two months. AI will change virtually everything that matters in the private and public sectors. It will soon reshape policies, ethics, and the very fabric of society.

The Age of AI: And Our Human Future
by Henry A Kissinger , Eric Schmidt and Daniel Huttenlocher
Published 2 Nov 2021

We postulated that by 2040, artificial intelligence would be perhaps a million times more powerful than it was in 2021, following Moore’s law, which predicts a doubling in computer processing power every two years. While increases in the power of AI are harder to quantify than increases in computing power, it appears that their growth is even more rapid. For example, the power of large language models, neural networks that underlie much of today’s natural language processing, is growing even more rapidly, tripling in fewer than two years. Microsoft’s Megatron-Turing model,2 released in late 2021, and Google’s PaLM,3 released in early 2022, each has more than 525 billion parameters compared to 175 billion for OpenAI’s GPT-3, which we wrote about in previous chapters and which was released in June of 2020.

…

Patent law in the United States and most nations recognizes only human inventors, potentially leading to situations in which there are inventions with no inventors (or perhaps in which people play a de minimis co-inventor role). National governments have recognized AI’s threat to language: Hungary has commissioned its own large language model so that Hungarian does not automatically become obsolete in the digital realm.9 Governments have also begun to grapple with digital networks’ dilution of communal and national identity. Some states, including China and Russia, have been more aggressive in this effort than others. The balance between risks of anarchy and risks of oppression is being struck differently in different places.

Boom: Bubbles and the End of Stagnation
by Byrne Hobart and Tobias Huber
Published 29 Oct 2024

Google offers an excellent analytics package (which makes it easier for sites to measure the return on advertising through, to take a hypothetical example, lots of Google search ads); Microsoft incorporates AI into search in part because its low market share means that the cost burden of AI is lower, while the high fixed costs of search mean that it’s sensible for them to buy market share even with expensive features; Meta supports open-source projects like React, PyTorch, and the LLaMA family of large language models, knowing that they’re likely to be the single biggest user of these products and thus the single biggest beneficiary of improvements from other open-source contributors. 257 Greater internal control has a predictable effect on corporate research. Large tech companies with vast resources are spending on R & D.

…

As discussed in Chapter 1, the abundance of money in big tech over the last decade has coincided with a scarcity of ideas, as indicated by the billions of dollars the large tech firms have been amassing. But beyond antitrust law and regulation, recent technological advances like the theoretical breakthrough of transformer-based AI and the user-facing advancements represented by OpenAI’s ChatGPT and other large language model-based tools might be the most significant accelerants for another corporate R&D bubble. While they have been in the making for decades, the recent breakthroughs in generative AI achieved by upstarts like OpenAI, Stability AI, and Anthropic have disrupted big tech’s R&D complacency for now.

…

In contrast to mimesis, the sublime demarcates a moment of elevation that breaks with or ruptures normal reality. 404 Writing in the first century CE, the Greek philosopher Cassius Longinus asserted that in the sublime, “our attention is drawn from the reasoning to the enthralling effect of the imagination and the reality is concealed in a halo of brilliance.” 405 The technological sublime, then, conceals reality, or at least clothes it, in brilliance; it speaks to the imagination and exercises an “enthralling” effect. Whether in the form of a rocket, an electric light, a nuclear chain reaction, a semiconductor, a neural net, or a large language model, the techno-scientific sublime, like a cathedral or a work of art, invokes a spiritual or religious experience of transcendence. It is this transcendence that reintegrates the future into the present and ignites the thymotic intensity and Promethean spirit that is necessary for radical technological innovation and the restoration of dynamism.

Algospeak: How Social Media Is Transforming the Future of Language
by Adam Aleksic
Published 15 Jul 2025

A 2015 Georgia Tech study found that it was so easy to replace sensitive Chinese keywords with understandable homophones that if every potential homophone were flagged on social media, 20 percent of all posts would be taken down as false positives—rendering the websites unusable.[3] That’s just from moving around tones, ignoring potential semantic substitutions, so we’re clearly quite far from a 1984-esque scenario. Large language models may get better at recognizing words in context, but people will always find creative ways to express their ideas. We’ve already seen this in action. During the 2019 Hong Kong protests, antigovernment social media users took to writing out messages in “Kongish,” a blend of Cantonese and English romanized to disguise their communication from Chinese-language algorithms.

…

See also specific types informal speech, 4, 10 in-groups, 9, 65, 100–105, 106, 110–11, 114–18, 122, 127–28, 145, 157, 169, 172–73, 183, 192, 198–201, 221 initialisms, 23–24 in-jokes, 103, 114–15 in media res feeling, 75–76 “in my X era,” 110–12, 118, 121–22 innovators, 53–57, 54, 140, 149 Instagram, 12, 17, 26, 33, 82, 100, 132, 141, 154, 172, 182, 211 Instagram Reels, 8, 11, 41, 57, 81, 121, 132, 182 international creators, 81–82 internet accents on, 89 attention and, 77 data uploaded daily, 11–12 division of, 110 dying languages and, 214 early era of, 38 English language and, 81–82, 208 folklore and, 200–201 4chan influence on, 140–41 ideas spread by, 142–43 inflection point in communication, 10, 10n slang and trends on, 34, 37–38, 52, 146, 160, 191–92 Whac-A-Mole response to moderation and, 14–15 internet sociolect, 34 interocular distance, 132–34, 138 interracial relationships, 135 intertextual references, 100 invisible string, 100–101 IOF (Israel Offensive Forces), 27 irony, 4, 45, 127, 138–39, 188 ISIS, 134 Israel-Gaza war of 2023, 25–27 Israeli flag emoji , 27 “It’s giving ___,” 73, 146, 160 “It should be illegal to X,” 62 “It’s over,” 135–36 “It’s the ___ for me,” 73 itutu, 144 “I want to kill myself,” 136 J jaw surgery, 138 jazz, 130, 144–45 Jenner, Kendall, 72 Jesus fish symbol, 24 Johnson, Lyndon B., 41, 45n journalists, 211–12 K Kardashian, Kim, 157 Kardashians, 95–96, 190 Kellogg School of Management, 196 Kendall Jenner airlines, 72 “kermit sewerslide,” 22–23 keywords, 8–9, 46, 163–68 keyword stuffing, 182 Khan, Azmat, 26 KHHV, 129 “kill,” censorship of, 4, 109 Know Your Meme, 12, 140n Kongish, 210 Koreaboos, 111 Korean fanwords, 100–101 K-pop fanilect, 100–103, 105–6, 111, 142, 149, 216 KSI videos, 52–53 L labels, 196, 202, 205. See also microlabels Lady Bountiful (Carr), 18, 18 laggards, 53–55, 54 Language and the Internet (Crystal), 10n large language models, 209 late majority, 53, 54 Latin, 5, 7, 69n Latino dance circles, 147 laughing-crying emoji , 53–54, 188 laughing emoji, 53 LDAR (lay down and rot), 136 “le dollar bean,” 30 leetspeak, 14–15, 17, 34–35 “left this world,” 5 “leg booty,” 30 Legendary (TV show), 148 lesbian, substitutions for, 30 le$bian, 30 “Let’s Go Brandon,” 23 LGBTQ+/LGBTQIA+, 6, 29–30, 147–48, 204 lies, 66 lifestyle influencer accent, 82–89, 93–97, 167, 203 light academia, 166, 179 Lil Meatball, 39 linguistic appropriation, 146 linguistic contexts, multiple, 51 linguistic diversity, 213–16 linguistic experimentation, 72–73 linguistic founder effect, 95–96 linguistic landscape, new, 13 linguistic resonance, 61 linguistic tricks, 78 linguistic videos, 104 linguistic Whac-A-Mole, 12, 14–16, 30, 36, 108–9, 109, 209–10 “List of Generation Z Slang,” 190 literacy rates, 10 Little Women (Alcott), 167 Liverpool dialects, 216 lol, 38 LOLcats, 188 lolspeak, 35 long-tail model, 171–74, 171 lookism, 124–25, 129–30, 132, 134–35, 137–38 looksmaxxing, 125–26, 128–29, 132–33, 137, 190 lore, 200–202, 204 Lorenz, Taylor, 8 Lost Generation, 186 lowkey, 72 “low-key ate,” 206 M machine learning algorithms, 59 Mad Libs, 48 Madonna, 148 MAGA, 23 Maher, Olivia, 71 Mai, Anthony, 52–53 Mailer, Norman, 23 main characters, 202, 205 majority adopters, 140 makeup videos, 203 male supremacy, 130 Mandal, Shriya, 59–61, 76 manuscripts, 201 marginalized groups, 114–15, 154 marital probation, 72 marketing, 60, 69, 170–85, 189, 204–5, 213 mass-report videos, 29 Matthew effect, 77–78, 133 maxxing, 128, 133, 137 McCulloch, Gretchen, 10n McDonald, Glenn, 166–67, 173–74, 181–82 McDonald’s restaurants, 81 Mean Girls (film), 56 meaning making, 200–201 medical educational content, 20 Megan Thee Stallion, 175 Melcher, Louisa, 66–67 memes, 4–6, 48–50, 53–55, 73, 198, 219, 220 algorithms and, 9 attention hacking and, 141 context and, 199 culture and, 57 4chan and Reddit and, 140–41 history of, 188 marketing and, 141 metadata and, 9, 57 obtrusive, 56 spread of, 9, 12, 48–52, 56, 137, 191 men, substitutions for, 19, 25, m*n, 19, 25 men’s rights trolls, 135 mental health, 4, 9 mentally disabled, 30 Merriam-Webster, 197 Meta, 41, 172 metadata, 9, 43, 45–46, 50, 57, 101, 107–8, 164, 173, 183 metonymy, 24–25, 27 mewing, 126, 132, 138, 141, 190, 206–7 “Michael Jackson Eating Popcorn,” 155 microhooks, 69, 83 microinfluencers, 96 microlabels, 165–84, 189, 203–4 Middle East, 29 middle-schoolers, 3–5, 7–8, 46–47, 73n, 151, 161, 191–93 millennial pause, 85–86, 188 millennial pink, 187, 189 millennials, 141, 187–90, 193, 196, 204 millennial socks, 187 Minaj, Nicki, 41, 43, 157 minced oaths, 22–23 minorities, 28, 150 minority language, 218 misinformation, 116–18, 142, 212 misogyny, 127, 130, 135 mog, 125, 132, 134, 210 mogging, 128–29, 132 Mogwarts, 126 monkeycore, 164 Montell, Amanda, 136 moron, 30 mors, 5, 7 “most interesting thing about X,” 68 “Mother” (song), 148–49 “mothers,” 147, 149 MrBeast, 86–89, 91–92, 97 Museum of Pop Culture (Seattle), 3, 5, 7, 10 Musical.ly app, 40–41 music industry, 41–42, 166, 184, 212 “My favorite thing about X is,” 62 N nail polish emoji , 29 Naked and the Dead, The (Mailer), 23 Narayanan, Arvind, 106, 117–18 narrative language, 203 National Association of the Deaf, 218 National Council on Severe Autism, 116 naturecore, 165 negativity, 118–21, 135–36, 139, 219 neologisms, 175 Netflix, 68 netspeak, 34 neurodivergent community, 115, 116 neurospicy, 115, 117 “neutral” American dialect, 93–94 Newgen, 137 Newman, Kayla, 157–59 news broadcasts, 93–94 News Daddy of TikTok, 211–12 newspaper comics, 18, 188 newspapers, 38, 49, 68, 108, 113 news sources, 211–12 New Yorker, 218 New York Times, 62, 175, 192–93, 211–12, 216–17 New York University (NYU), 62–63 nia, 152 niche communities, 99–123, 140, 171–73, 181–82 ninja emoji , 21 “nip nops,” 22, 50 Nirvana, 3 “no because,” 73–77 noguing, 149 non-Deaf ASL creators, 217 noob, 35, 38 normies, 128 “Not you,” 146 “not you doing ____,” 73 NPCs, 202 NPR, 82 number 13, 100 n-word, substitutions for, 21, 28, 152 O Oberlin College, 126 obtrusiveness, 55, 56, 192 offensive words, 17–18, 30 offline, online words moving, 10, 35, 42, 73, 77, 108, 110–12, 116, 122, 203, 211, 216 ohio, 44 O.K., 38, 108, 113 okay, 38, 49 “OK boomer,” 188, 196 Old English, 5 “old money,” 179 Old Norse, 5 “Old Town Road” (song), 42 oligopoly, 181 “One thing people don’t tell you about X is Y,” 65 “on fleek,” 39, 55, 158–59 “on god,” 47n “on gyat,” 47 online disinhibition effect, 135 online intertextuality, 199 online sociolects, 34, 35 “on sigma,” 47, 50 “on skibidi,” 47 “on X,” 47 opening lines, 62–68, 76 opp, 152, 154, 162, 210 optimal distinctiveness, 183 Orange Is the New Black, 48 “orange man,” 25 “orange peel theory,” 72 originators, crediting, 158–60 out-groups, 103, 115–16, 148, 155, 189, 193, 201, 221 Oxford English Dictionary, 47 “Oxford study,” 135, 138–39 Oxford University Press, 191n P Page, Dylan, 211–12 Palestine, 26 Palestine flag emoji , 27 palm (of hand), 27, 34 parasocial interactions, 64, 84 Paris Is Burning (documentary), 148 Parker, Charlie, 145 party, substitution for, 49, 57 “passed away,” 5, 9 pastel goth, 168–70, 181, 201, 203 pattern recognition, 201 Payman, Fatima, 210–11 pay-to-play models, 181 PBS Origins (TV show), 156 peach emoji , 20–21 peen, 22 pee-pee, 22 pen, pencil, 19–20 penis, respellings of, 19–22 p3nis, 20 pen*s, 20 people, 64–65 “People are finally doing X,” 65 Pepe the Frog meme, 134 performance art, 66 period, 155, 160 “periodt,” 155 permanent wave music, 166 personalized recommendations, 12, 40–41, 43, 61, 99–123, 189, 204 PewDiePie, 173 Pew Research Center, 187 pHashing technique, 140–41 phrasal templates, 47–48, 50–51, 56, 73, 110–11, 121, 133 phrases, coining, 158 phrenological theories, 133, 135 pictography, 20–21, 35 pilled, 126, 133, 137 Pinker, Steven, 30 plaguecore, 164 plausible deniability, 25, 159 plot, 202 Poe’s law, 138–39 Pokémon discussion board, 140 polarization, 110, 119, 218–19 political discourse, 6, 19, 24–25, 110, 119 pop culture, 12 pornography, 6, 20–21, 34 portmanteaus, 100 Portuguese language, 206–7 positive feedback loops, 42, 55, 77, 104–5, 190 posting time, 61 post-ironic nods, 199 postmodern aesthetics, 169 post titles, 59–60 post-Vine, 47 pr0n, 34 prawn, 34 preferences, stated vs. revealed, 119–21, 123 preferential attachment, 215 preppy, 176–79, 181–83, 189, 195 presentation styles, 69–70 Presley, Elvis, 156 prestige dialect, 92–94, 215 “pretty on fleek,” 157 preverb music, 166 priming, 75 print media, 10, 38, 51, 52 pro-Palestine creators, 27 psychology, 58, 69, 75 punk music, 164 punks, 145, 168, 201 pwned, 35 Q QAnon, 134 queens, 147–48, 157, 161 queer community, 29–30, 52, 147, 149, 204.

The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip
by Stephen Witt
Published 8 Apr 2025

“Then you’d email him again at 6 a.m. and receive a reply at 6:05 a.m.” The buzz about the transformer architecture filtered up to Huang through these emails. OpenAI was said to be running a job on a Microsoft Azure server that had tied up a thousand GPUs over the course of a month while running up a $5 million bill. This was a new language model—a “large” language model, so named because it was about a hundred times bigger than anything yet deployed. Huang realized that he needed a new tool kit just for transformers. He informed Bas Aarts to drag his current recurrent neural-network compiler to the recycle bin and begin work on a transformer compiler at once.

…

Despite the increase in computational power, neural nets still hit plateaus, and when they did, there was still often no obvious way to make them better. Firms that attempted to replicate GPT with in-house data often produced shambolic “knowledge engines” that were little better than toys. These AIs supplemented the standard large-language-model training set with emails, mission statements, patent applications, legal memoranda, and other exciting selections from the internal corporate syllabus. As the buzz percolated through middle management, executives at marketing, media, and health-care firms launched ambitious initiatives, sometimes openly telling staff that many employees would be laid off once the neural nets were working.

Nexus: A Brief History of Information Networks From the Stone Age to AI
by Yuval Noah Harari
Published 9 Sep 2024

See also Zeynep Tufekci, “Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency,” Colorado Technology Law Journal 13 (2015): 203–18; Janna Anderson and Lee Rainie, “The Future of Truth and Misinformation Online,” Pew Research Center, Oct. 19, 2017, www.pewresearch.org/internet/2017/10/19/the-future-of-truth-and-misinformation-online/; Ro’ee Levy, “Social Media, News Consumption, and Polarization: Evidence from a Field Experiment,” American Economic Review 111, no. 3 (2021): 831–70; William J. Brady, Ana P. Gantman, and Jay J. Van Bavel, “Attentional Capture Helps Explain Why Moral and Emotional Content Go Viral,” Journal of Experimental Psychology: General 149, no. 4 (2020): 746–56. 22. Yue Zhang et al., “Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models” (preprint, submitted in 2023), arxiv.org/abs/2309.01219; Jordan Pearson, “Researchers Demonstrate AI ‘Supply Chain’ Disinfo Attack with ‘PoisonGPT,’ ” Vice, July 13, 2023, www.vice.com/en/article/xgwgn4/researchers-demonstrate-ai-supply-chain-disinfo-attack-with-poisongpt. 23. František Baluška and Michael Levin, “On Having No Head: Cognition Throughout Biological Systems,” Frontiers in Psychology 7 (2016), article 902. 24.

…

(Hint: Be Afraid),” Reuters, Dec. 10, 2022, www.reuters.com/legal/transactional/will-chatgpt-make-lawyers-obsolete-hint-be-afraid-2022-12-09/; Chloe Xiang, “ChatGPT Can Do a Corporate Lobbyist’s Job, Study Determines,” Vice, Jan. 5, 2023, www.vice.com/en/article/3admm8/chatgpt-can-do-a-corporate-lobbyists-job-study-determines; Jules Ioannidis et al., “Gracenote.ai: Legal Generative AI for Regulatory Compliance,” SSRN, June 19, 2023, ssrn.com/abstract=4494272; Damien Charlotin, “Large Language Models and the Future of Law,” SSRN, Aug. 22, 2023, ssrn.com/abstract=4548258; Daniel Martin Katz et al., “GPT-4 Passes the Bar Exam,” SSRN, March 15, 2023, ssrn.com/abstract=4389233. Though see also Eric Martínez, “Re-evaluating GPT-4’s Bar Exam Performance,” SSRN, May 8, 2023, ssrn.com/abstract=4441311. 31.

…

Ibid., book 6.50. 15. Albert Einstein et al., “The Russell-Einstein Manifesto [1955],” Impact of Science on Society—Unesco 26, no. 12 (1976): 15–16. CHAPTER 11: THE SILICON CURTAIN: GLOBAL EMPIRE OR GLOBAL SPLIT? 1. Suleyman, Coming Wave, 12–13, 173–77, 207–13; Emily H. Soice et al., “Can Large Language Models Democratize Access to Dual-Use Biotechnology?” (preprint, submitted 2023), doi.org/10.48550/arXiv.2306.03809; Sepideh Jahangiri et al., “Viral and Non-viral Gene Therapy Using 3D (Bio) Printing,” Journal of Gene Medicine 24, no. 12 (2022), article e3458; Tommaso Zandrini et al., “Breaking the Resolution Limits of 3D Bioprinting: Future Opportunities and Present Challenges,” Trends in Biotechnology 41, no. 5 (2023): 604–14. 2.

Fixed: Why Personal Finance is Broken and How to Make it Work for Everyone
by John Y. Campbell and Tarun Ramadorai
Published 25 Jul 2025

Technology Is Part of the Solution, but Also Part of the Problem Fintech for Good: to Infinity … and Beyond! One seemingly obvious solution to the problems of personal finance is to deploy better technology. The suite of technological tools at our disposal today is already impressive and improving rapidly. The generative artificial intelligence revolution and the emergence of large language models promise to upend service provision in many areas of economic life, and personal finance appears ripe for disruption.1 Financial technology (“fintech” for short) is more easily accessible than ever. The arrival of the Internet began a process of technological connection that started with the personal computer and has greatly accelerated with the advent of the mobile phone and, most recently, the smartphone, as figure 8.1 shows.

…

See also higher education; housing; mortgages; risky investments Investment Savings Accounts (ISAs), 157–159, 297n6 investors, accredited, 25–26, 218, 261–262 iron law of active investing, 136–137 Islamic principle on financial contracts, 234 Israel, mortgage reform in, 207–208, 224, 289n36 Jackson, Howell, 270n1 James, LeBron, 193 Jansson, Thomas, 234 jewelry, as emergency fund, 69–70 Johnson, Jasmin, 106 Jump$tart Coalition for Personal Financial Literacy, 306n10 junk fees, 218; defined, 310n46 Juno, 188 Kahneman, Daniel, 271n8 Kakaku.com, 183 Kapoor, Virendra Pal, 4–5, 63, 215, 263–264 Karabulut, Yigitcan, 234 Katz, Justin, 186 Kayak, 209 Keillor, Garrison, 271n8 Keogh accounts, 297n7 Kiyosaki, Robert, 265n6 Klarna, 182 Lake Wobegon, 137 lapsation problem, insurance and, 54–55, 61, 145–147 large language models, 180 late fees, credit card, 92, 94–95, 207, 312n58 lawsuits, approved financial products and protection from, 217 learning from others, danger of, 45–47 Lee, Bruce, 234 Lee, Spike, 193 legacy financial systems, 202 legal system to protect investors, 202 Lemonade, 182 lenders, matching with borrowers, 242–243 Levine, Phillip, 208–209 Liang, Kevin, 272–273n26 libertarian paternalism (“nudge”), 211–215 life expectancy: amount need to save for retirement and, 159; annuities and, 169–170; retirement and increase in, 150–151 life insurance, 253–254; lapsation problem, 54–55, 61, 145–147; unit-linked insurance plans, 63–64 limited liability of shares, 8 liquidity, emergencies and need for, 25 liquidity management, 258–259, 316n31 living in retirement, 169–175; annuity puzzle, 169–172; housing in retirement, 172–175 Lloyd’s of London insurance market, 68, 266n10 loan consolidators, 95 loans: emergency credit, 89–90; prepayment penalties, 268n22; relying on informal social networks for, 71–72, 89; social sanctions to enforce repayment, 72, 279n43.

The Economic Consequences of Mr Trump: What the Trade War Means for the World
by Philip Coggan
Published 1 Jul 2025

In total, these would have produced 234,000 new manufacturing jobs.50 Many of these planned jobs may now be at risk because the Trump administration has halted a wide range of projects related to renewable energy and electric car charging.51 Mr Trump’s dislike for renewable energy is simply irrational; the costs of solar energy have dropped by 90 per cent over the last ten years, while those of wind energy have fallen 70 per cent.52 But rather than invest in the industries of the future, Mr Trump wants to turn back to the industries of the past, like coal and oil. In the long run, the US should really be worried about the challenge from China, not in low-margin industries like assembling electronic products, but in new areas such as artificial intelligence. China’s development of the Deep-Seek large language model came as a shock to American companies that had previously dominated the field.53 It emerged despite, or perhaps because of, US attempts under both Presidents Trump and Biden to constrain China’s technological development. The Royal United Services Institute, a defence think tank, argues that the sanctions may ‘have been counterproductive, in the sense that they have galvanized an ecosystem of Chinese businesses to finally come on board with policy initiatives to indigenize high-end technology capabilities’.54 The US would do better to focus on ensuring its own scientists and technologists have all the resources they need.

Literary Theory for Robots: How Computers Learned to Write
by Dennis Yi Tenen
Published 6 Feb 2024

As someone who studies the development of artifice and intellect historically, I know the current moment of excitement over new technology will subside, diminished by the frenzy of its grifters and soothsayers. What remains will be more modest and more significant. Viewed in the light of collective human intellectual achievement, large language models are built on the foundation of public archives, libraries, and encyclopedias containing the composite work of numerous authors. Their synthesized voice fascinates me not so much for what it says, on average. Rather, I hear it as a rebuke to my own, dearly held beliefs about authorship, and therefore agency and creativity.

The Wealth Ladder: Proven Strategies for Every Step of Your Financial Life
by Nick Maggiulli
Published 22 Jul 2025

Despite its many benefits, as a form of leverage code has its own unique challenges. Coding requires high technical skill. This means domain knowledge and keeping up with the ever-evolving tech landscape. You have to enjoy continuous learning if you want to code for the long haul. While the rise of AI and other large language models (LLMs) like ChatGPT have made coding much easier, you will still need some familiarity with programming to use these tools effectively. In addition, coding requires ongoing maintenance of your code. Systems change and things break. Though I said that the “work is done” once an app is released, this is rarely the case.

Stories Are Weapons: Psychological Warfare and the American Mind
by Annalee Newitz
Published 3 Jun 2024

People need to know where our content is going as well as where it comes from. Facebook sometimes acts as a data broker, selling profile information to advertisers and other third-party companies. When people post, they should do it knowing whether their photos will be used to train facial-recognition algorithms or their words consumed by large language models like ChatGPT. They should understand whether their personal profile is being aggregated and sold to advertisers or political operatives who want to target people like themselves. Knowing who might use our information will introduce yet another layer of friction and might make us pause before posting our cute selfies and clever manifestos.

Abundance
by Ezra Klein and Derek Thompson
Published 18 Mar 2025

Derek Thompson, interview with Nicholas Bloom. Medical research productivity is hard to measure, and Bloom’s conclusions are not universally supported. Some scholars have published persuasive work suggesting that medical research productivity is more likely stable. See “Distilling Data from Large Language Models,” by Maya M. Durvasula, Sabri Eyuboglu, and David M. Ritzwoller. But whether productivity in this all-important sector is flat or declining, the most important thing is that it is not obviously rising. Just as we should hope for rising productivity in any industry, we should hope for it in science. 38.

Enshittification: Why Everything Suddenly Got Worse and What to Do About It
by Cory Doctorow
Published 6 Oct 2025

Google’s DeepMind division had achieved technically impressive results with its convolutional network technology, which turned out to be a rich seam that threw off new, cool demos that dominated several viral news cycles. But DeepMind’s dominance couldn’t last. Deep-pocketed competitors sprang up, and new, AI-powered chatbots built on large language models (LLMs) started to steal focus away from Google. The company responded with a full-court press to make its own LLMs, leveraging its vast resources to build some of the largest models ever conceived of. This move attracted the attention of some of Google’s top scientists, including Timnit Gebru, a distinguished AI researcher whose career had involved stints at Apple, Microsoft, and Stanford before she came to Google to work on AI ethics.

Extremely Hardcore: Inside Elon Musk's Twitter
by Zoë Schiffer
Published 13 Feb 2024

This freed him up to focus on other passions: stopping rival tech companies from scraping Twitter’s data without permission—even if it meant losing eyeballs on ads. Data scraping was a known problem at Twitter. “Scraping was the open secret of Twitter data access. We knew about it. It was fine,” Yoel Roth wrote on the Twitter-alternative Bluesky. AI firms in particular were notorious for gobbling up huge swaths of text to train large language models (LLMs). Now that those firms were worth a lot of money, the situation was far from fine in Musk’s opinion. In November 2022, OpenAI debuted ChatGPT, a chatbot that could generate convincingly human text. By January 2023, the app had over 100 million users, making it the fastest-growing consumer app of all time.

Rise of the Robots: Technology and the Threat of a Jobless Future
by Martin Ford
Published 4 May 2015

Google’s development team began by focusing on official documents prepared by the United Nations and then extended their effort to the Web, where the company’s search engine was able to locate a multitude of examples that became fodder for their voracious self-learning algorithms. The sheer number of documents used to train the system dwarfed anything that had come before. Franz Och, the computer scientist who led the effort, noted that the team had built “very, very large language models, much larger than anyone has ever built in the history of mankind.”8 In 2005, Google entered its system in the annual machine translation competition held by the National Bureau of Standards and Technology, an agency within the US Commerce department that publishes measurement standards. Google’s machine learning algorithms were able to easily outperform the competition—which typically employed language and linguistic experts who attempted to actively program their translation systems to wade through the mire of conflicting and inconsistent grammatical rules that characterize languages.

Unit X: How the Pentagon and Silicon Valley Are Transforming the Future of War
by Raj M. Shah and Christopher Kirchhoff
Published 8 Jul 2024

Japan took greater steps to ready its military for the competition with China, and to link its efforts with South Korea and other Asian allies. Biden held the first Camp David summit of his presidency with the prime minister of Japan and the president of Korea, to discuss the China threat. Other wins included the Pentagon’s embrace of the generative AI revolution. The DoD launched “Task Force Lima” to speed experimentation with large language models like ChatGPT across the military. Also, Scale AI, the Silicon Valley startup whose CEO Alex Wang joined Chris to advise the White House on China, became the first company to provide generative AI capabilities to the Pentagon. Scale AI’s software platform, called Donovan, uses generative AI models for military planning and battlefield monitoring.

The Measure of Progress: Counting What Really Matters
by Diane Coyle
Published 15 Apr 2025

The working from home delusion fades. https://­www​.­economist​ .­com​/­finance​-­and​-­economics​/­2023​/­0 6​/­28​/­the​-­working​-­from​-­home​-­delusion​-fades? Eeckhout, J. (2021). The profit paradox: How thriving firms threaten the f­uture of work (1st ed.). Prince­ton University Press. Eloundou, T., Manning, S., Mishkin, P., and Rock, D. (2024). GPTs are GPTs: An early look at the ­labor market impact potential of large language models. Science, 384(6702), 1306–1308. https://­doi​.­org​/­10​.­1126​/­science​.adj0998 Erickson, T., and Pakes, A. (2011). An experimental component index for the CPI: From annual computer data to monthly data on other goods. American Economic Review, 101(5), 1707– 1738. doi:10.1257/aer.101.5.1707 ­European Commission Directorate General for Communications Networks, Content and Technology. (2021).

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World
by Pedro Domingos
Published 21 Sep 2015

“Relevance weighting of search terms,”* by Stephen Robertson and Karen Sparck Jones (Journal of the American Society for Information Science, 1976), explains the use of Naïve Bayes–like methods in information retrieval. “First links in the Markov chain,” by Brian Hayes (American Scientist, 2013), recounts Markov’s invention of the eponymous chains. “Large language models in machine translation,”* by Thorsten Brants et al. (Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 2007), explains how Google Translate works. “The PageRank citation ranking: Bringing order to the Web,”* by Larry Page, Sergey Brin, Rajeev Motwani, and Terry Winograd (Stanford University technical report, 1998), describes the PageRank algorithm and its interpretation as a random walk over the web.

The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future
by Orly Lobel
Published 17 Oct 2022

Timnit Gebru—a rising star in AI research with an extraordinary path from Ethiopia to Eritrea to political asylum in the United States, to three degrees at Stanford, to Apple, to Microsoft, and then to Google—was ousted from Google over a dispute with company executives about publishing an article on the potential risks and harms of large language models. The article warns that ever larger natural language processing models may be too large to monitor, that the sheer mass of the data becomes inscrutable. The paper calls for curating and documenting data sets “rather than ingesting everything on the web.”3 It also warns against spreading claims about models understanding language and concepts, as opposed to simply identifying patterns in human-made texts, numbers, and images.

What If We Get It Right?: Visions of Climate Futures
by Ayana Elizabeth Johnson
Published 17 Sep 2024

Ayana: When we hear about AI in the press, it’s often about generating images or writing a term paper or an email. But the ability for AI to work with scientific data sets is something we don’t hear much about. Mustafa: Many sources of climate data are too limited for generative AI or large language models (LLMs) to work with. The data sets are quite small. The weakness of these AI models is that they depend on very, very large amounts of data. But now that we have, over the last ten years, built these foundational models, you can add small bits of new data that the AI model hasn’t seen before, and it can do a pretty good job of predicting what comes next.

In the Plex: How Google Thinks, Works, and Shapes Our Lives
by Steven Levy
Published 12 Apr 2011

Och’s official role was as a scientist in Google’s research group, but it is indicative of Google’s view of research that no step was required to move beyond study into actual product implementation. Because Och and his colleagues knew they would have access to an unprecedented amount of data, they worked from the ground up to create a new translation system. “One of the things we did was to build very, very, very large language models, much larger than anyone has ever built in the history of mankind.” Then they began to train the system. To measure progress, they used a statistical model that, given a series of words, would predict the word that came next. Each time they doubled the amount of training data, they got a .5 percent boost in the metrics that measured success in the results.

Elon Musk
by Walter Isaacson
Published 11 Sep 2023

This meant that his engineers were actually ahead of OpenAI in creating full-fledged artificial general intelligence, which requires both abilities. “Tesla’s real-world AI is underrated,” he said. “Imagine if Tesla and OpenAI had to swap tasks. They would have to make Self-Driving, and we would have to make large language-model chatbots. Who wins? We do.” In April, Musk assigned Babuschkin and his team three major goals. The first was to make an AI bot that could write computer code. A programmer could begin typing in any coding language, and the X.AI bot would auto-complete the task for the most likely action they were trying to take.

Artificial Intelligence: A Modern Approach
by Stuart Russell and Peter Norvig
Published 14 Jul 2019

Brandt, F., Conitzer, V., Endriss, U., Lang, J., and Procaccia, A. D. (Eds.). (2016). Handbook of Computational Social Choice. Cambridge University Press. Brants, T. (2000). TnT: A statistical part-of-speech tagger. In Proc. Sixth Conference on Applied Natural Language Processing. Brants, T., Popat, A. C., Xu, P., Och, F. J., and Dean, J. (2007). Large language models in machine translation. In EMNLP-CoNLL-07. Bratko, I. (2009). Prolog Programming for Artificial Intelligence (4th edition). Addison-Wesley. Bratman, M. E. (1987). Intention, Plans, and Practical Reason. Harvard University Press. Breck, E., Cai, S., Nielsen, E., Salib, M., and Sculley, D. (2016).