Generative Pre-trained Transformer

16 results

pages: 169 words: 41,887

Literary Theory for Robots: How Computers Learned to Write
by Dennis Yi Tenen
Published 6 Feb 2024

W., 110 Bloomfield, Leonard, 83 Bobrow, Daniel, 92 body posture, 4 Boeing, 11, 97 Boston, Mass., 70 bots, 136–37 Brecht, Bertolt, 61 Brin, Sergey, 113 Browning, I., 110 brute force computing, 110 Byron, Lady Anne Isabella, 51, 52 Byron, Lord George, 12, 51, 61 bytes, 6–9, 88 Calculus of Probabilities (Tenen), 103 Cambridge University, 60, 92, 113 care, 13 cars, self-driving, 14 Cast Away (film), 36 Cauchy, Augustin-Louis, 44 cells, 27 Center for Communications Sciences, 87 Centre for Data Ethics and Innovation, 131 chains, 105 Characteristica universalis (Leibniz), 44 characters, 9 Charles Joseph, Archduke, 32 Charniak, Eugene, 92 charts, 9, 19, 22, 25, 76 chatbots, 9, 11, 16, 20, 113–15, 118, 120, 122–24, 129 Chautauqua Literary File, 74 chemistry, 82, 85 China, 10 Chinese language, 43 Chomsky, Carol, 92 Chomsky, Noam, 21, 86–87, 90, 92, 93, 97, 102, 114 Clark, Peter, 97–99 clock-making, 52 cognition, distributed, 123–25 Colby, Kenneth, 92 collective intelligence, 133 collective labor, 123 collective nouns, 128 college papers, 137 Collegio Romano, 30 Columbia University, 83 combinatorial sonnets, 34 command and control, 88 communication(s), 105–6, 109–10, 119 comparative philology, 80, 83 computers, 8, 12, 37, 48, 87, 88, 91–93, 97, 105, 109 computer science, 121, 135 consciousness, 3 context, and meaning, 115 controlled taxonomies, 23 conversational AI, 135 conversational intelligence, 139 Cook, William Plotto: A New Method of Plot Suggestion for Writers of Creative Fiction, 71, 75–77, 81, 94 Cornell University, 110 corporate personhood, 125, 127 corporations, 128 corpuses, 12 counterprogramming, 136 Crane, Gregory, 28 creativity (creative process), 14, 21, 61, 63, 67, 68, 133, 134, 140–41 Darwin, Charles, 51 databases, 3, 24 Defoe, Daniel Robinson Crusoe, 36, 65 De Morgan, Sophia Elizabeth, 51 Descartes, Réné, 44 Meditations, 35 de Stael, Germaine, 80 Detroit, Mich., 74 Diagnostic and Statistical Manual of Mental Disorders (DSM-5), 23–24 Dial, 74 Dickens, Charles, 51, 65, 67 dictionaries, 29 Diderot, Denis Encyclopédie, 122 Difference Engine, 48–49, 52, 60 disinformation, 132 distributed intelligence, 123–24 distributed thought, 123–25 divination circles, 24 document retrieval, 110 Donbas region, 121 Dostoevsky, Fyodor, 65, 121 Douglas, Mary How Institutions Think, 127 Downey, June Plots and Personalities, 71 Dowst, Robert Saunders The Technique of the Fiction Writing, 71 Dumas, Alexandre, 65 Dwarf Fortress, 100 eclipses, 52 Editor: Journal of Information for Literary Workers, 70, 72, 74 educational reform, 67 Educational Specialty Company, 74 ELIZA (chatbox therapist), 20, 92, 93 emergence, 16 encryption, 119 Encyclopédie (Diderot), 122 English, Thomas, 71 Skeleton Essays, or Authorship in Outline, 71, 72 Enlightenment, 46, 67 epistemology, 84 equality, concept of, 116 Esenwein, Joseph Writing the Short Story, 71 Esperanto, 45 Essay Towards a Real Character, and a Philosophical Language, An (Wilkins), 10, 41–45 Eugene Onegin (Pushkin), 104, 117 Eureka Pocket Scrap Book, 74 event horizons, 7 exceptions and exceptionalism, 38, 39, 52, 59, 61, 62, 79–80, 133 FAA Incident Data System (FIDES), 98, 99 failure, concept of, 116 fairness, concept of, 116 fairytales, See folktales Fansler, Harriott Types of Prose Narratives, 71 Faraday, Michael, 51 Faulkner, Mary, 66 Federal Aviation Administration (FAA), 98 Fiction House, 66 fictions, real effect of, 127 FIDES (FAA Incident Data System), 98, 99 film, 61 Finns, 80 Firth, John, 114, 115 “flora and fauna,” 129 folklore studies, 80–82, 93–96 folktales, 79–83, 88, 93, 96–98 Franklin, Ohio, 70 freedom, concept of, 116 Garcia-Molina, Hector, 113 gears, 14, 49, 52–53 general intelligence, 37–38 generative grammars, 87, 88, 91, 94, 97 generative pre-trained transformers (GPT), 139 German language, 121 Germany, 26, 34 Gibson, Kevin, 127 global positioning system (GPS), 15 Godard, Jean-Luc, 93 Goethe, Johann Wolfgang von, 65 Gore, Al, 17 GPS (global positioning system), 15 GPT (generative pre-trained transformers), 139 GPT-4 algorithm, 129, 130 grammar(s), 21, 38, 40–42, 85–88, 90–95, 97, 102, 108, 113–14 Green, Bert, 92 Grimm, Jacob, 80 ground truth, 40 GRU, 132, 136 Gulliver’s Travels (Swift), 65 GUS (story generator), 92–93 Gutenberg printing press, 39 Habsburgs, 30 Hanks, Tom, 36 Harris, Zellig, 83, 86, 114 Harry Potter, 107 Harvard University, 72–73, 113 Hayy ibn Yaqzan (Ibn Tufail), 36 “Heavenly Love-Kiss XLI” (Kuhlmann), 31, 40 Hegelianism, 6–7 Herder, Johann Gottfried von, 80 hermeneutics, 2 Hill, Wycliffe, 75, 77 Plot Genie, 75–77, 81, 94 history, 5–9, 12, 91 Hobbes, Thomas Leviathan, 127 Hollywood, 71, 94–95, 134 Home Correspondence School, 70–71 Hopi, 80 Horne, Charles The Technique of the Novel, 71 How Institutions Think (Douglas), 127 human intelligence, 116, 122, 123, 134 humanities, 121 Humboldt, Wilhelm von, 50 IBM, 8, 87, 113 IBM 709 computer, 88, 109 Ibn Khaldun, 9, 18–21, 25, 45–46, 48, 118, 139 Muqaddimah, 18–19, 26 Ibn Tufail Hayy ibn Yaqzan (“Self-Taught Philosopher”), 36 indexers and indexing, 113 induction, 52 industrial age, 2, 64 information, flow of, 105 innovation, 8 inputs, 5 Inquisition, 31 inscription, 12 Institute for Ethics in AI, 131 intellectual work, 61–62 intelligence, 34–37; See also artificial intelligence (AI) of AI, 14–16, 21, 125 collective, 133 concept of, 116 conversational, 139 defining, 4 distributed, 123–24 general, 37–38 human, 116, 122, 123, 134 as labor, 61, 124 linguistic, 101 machine, 37, 115–16 military, 132 spectrum of, 17 statistical, 114 super-, 21 unevenness of, 139 universal, 37–38, 139 internet, 17, 136 Italy, 30 Ivan V, Tsar, 34 Jacquard loom, 60, 67 Jakobson, Roman, 84 Jones, Karen Spärck, 113 journalism, 61 Journal of Business Ethics, 127 justice, concept of, 116 Kafka, Franz, 61 Kane, William R., 70 Kaplan, Ronald, 92 Kay, Martin, 92 Kazakhstan, 139 Keeler, Harry Web-Work, 73, 95 Kepler, Johannes, 30 keywords, weighted, 113 keywords in context, 113 Khwarizmi, Muhammad ibn Musa al-, 9 Kircher, Athanasius, 30–34, 36, 39–40, 44, 48, 49, 56, 60, 75 Kissinger, Henry, 16 Klein, Sheldon, 92 knowledge, 11, 21, 31, 32, 35–36, 46, 132 knowledge work, 5, 62, 133–36 KPMG, 131 Kubrick, Stanley, 93 Kuhlmann, Quirinus, 30–32, 34, 40 “Heavenly Love-Kiss XLI,” 31, 40 Kyrgyzstan, 8 labor, 123 collective, 123 intellectual, 61–62 and knowledge work, 133–36 Latin, 39 Latin Word Study Tool, 28 Laughery, Kenneth, 92 law, 127 Law & Order (television series), 79 “Laws of Literary Invention,” 81 learning, 115–16, 125–26 Lehert, Wendy, 93 Leibniz, Gottfried, 10, 43–46, 48, 55, 60, 118 Characteristica universalis, 44 Plus Ultra, 44–45 Leningrad State University, 82 Lenski, Lois The Little Train, 87, 88, 90, 109 Lermontov, Mikhail, 121 letter magic, 20–22 Leviathan (Hobbes), 127 Li, Robin, 113 Lindsay, Kathleen, 66 linguistic intelligence, 101 linguistic proficiency, 115–16 linguistics, 2, 83–87, 92–93, 101, 103, 113, 114, 119 literacy, 38, 64, 67 literary markets, 65, 70 literary production, 64–67, 69, 71–72, 79 literature, 1, 9–11, 59, 61, 65, 80–82, 120–21 Little, Paul, 66 Little Train, The (Lenski), 87, 88, 90, 109 Llull, Ramon, 9, 10, 24–27, 31, 46, 48, 91, 118 Ars Brevis, 24, 31 local values, 38 London, Jack, 74 Longfellow, Henry, 80 Lost (television series), 36 Lovelace, Ada, 12, 43, 48, 51–52, 54–60, 64, 91, 118 Lucas, George, 93 Luhn, Peter, 113 machine intelligence, 37, 115–16 machine learning, 115–16, 125–26 machine translation, 119 mainframe computers, 87, 88 malicious agents, 136 manual transmissions, 14 manufacturing mass, 133–34 template-based, 60–61, 64 MARGIE (story generator), 92 markets, literary, 65, 70 Markets and Methods for Writers, 71 Markov, Andrey Andreyevich, 103–5, 109–10, 112 Markov chains, 10, 101, 106, 108–9 Marx, Karl, 29 Massachusetts Institute of Technology (MIT), 21, 87, 92, 105 mass literacy, 38 mass manufacturing, 133–34 Masterman, Margaret, 92, 113, 119 Mathematical Organ, 32–34, 39–40, 44, 48, 75 matrix, 64 meaning (meaning-making), 8–9, 57, 91, 93, 101–3, 105, 110–12, 114, 115, 138 Meaning of Meaning, The (Ogden and Richards), 102 mechanical notation, 53–54 Meditations (Descartes), 35 Meehan, James, 92, 94–97, 99 memorization, 3 memory, 8–9, 28–29 MESSY (story generator), 92 metaphors, 6, 13, 17, 29, 111, 115, 125–28 military, 10, 87–88, 93, 97, 119, 132 mind, 3, 4, 6, 16, 62, 84, 92–94, 96, 102, 114–15 missile defense systems, 10, 87 MIT, See Massachusetts Institute of Technology MITRE Corporation, 10, 88 mobile phone assistants, 123 moral agency, 131–32 Morphology of the Folktale (Propp), 80–82, 93, 96 Morse code, 7 Moscow, Russia, 34 MS.

…

There, “universal” intelligence—some of the things ever written—finally joins the list of global energy resources, to be fought over, colonized, extracted, sold, bought, pirated, liberated, and exhausted. 9. General intelligence leads to generic intelligence. At each stage of its development, from Ibn Khaldun to the early versions of the generative pre-trained transformers (GPT), we have often found our rather narrow set of technical concerns to veer toward the spiritual. As the gold rush around this latest wave of AI research subsides, I cannot help but wonder about this noninstrumental excess. Even the most rudimentary of word mechanisms has the immediate capacity to delight, in the way children still play with simple origami fortune tellers.

pages: 439 words: 125,379

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future
by Keach Hagey
Published 19 May 2025

The elfin thirty-eight-year-old was wrapping up the best year of a charmed career, a year when he became a household name, had senators eating out of his hand, met with presidents and prime ministers around the world, and—most important within the value system of Silicon Valley—delivered a new technology that seemed like it was very possibly going to change everything. When OpenAI launched its uncannily humanlike chatbot, ChatGPT (short for generative pre-trained transformer) the previous November, it was an instant smash, reaching 100 million users in less than three months, the fastest-growing app in the world to date.4 When OpenAI, only a few months later, unveiled a more formidable successor, GPT-4—it could pass the bar exam and ace the AP biology test—the dizzying rate of progress suggested that the company’s audacious mission to safely create the world’s first artificial general intelligence, or AGI, might indeed be within reach.

…

At OpenAI, we thought, ‘We are a dirty work company.’ ” WHEN THEY finished training the model, they found that it could not only beat benchmarks when answering questions from the data it was trained on, but that it seemed able to answer questions about things it wasn’t trained on, a phenomenon known as “zero-shot.” Years later, Altman would describe the result as “somewhat impressive, but no deep understanding of how it worked or why it worked.” Radford, Sutskever, and team called the model a “generatively pre-trained transformer,” or GPT for short.9 They showed a new path forward for the company. The agent approach was wrong because it started training from scratch, Karpathy explained. It took a gargantuan amount of machine learning to teach it what it needed to know. “The right way to do it is you ignore all that stuff, and you train language models,” he said.

…

Louis, 26, 53 Delicious (website), 75 Deming, Peter, 59–60, 64 democracy, 152–53, 225 Democratic Party, 21, 204, 207–8, 234, 296 Deployment Safety Board (DSB), 279–80, 287 Deshpande, Alok, 58–61, 64, 76, 85, 89, 102, 111, 127 Deshpande, Sheila, 58, 91 Desmond-Hellmann, Sue, 303 Deutsch, David, 314–15 Dewey, John, 45 deworming charities, 211, 213 Diamandis, Peter, 144, 210 diffusion model trained by adding digital “noise,” 262–63 Digital Chocolate mobile gaming company, 97 Diller, Barry, 229 Ditton, Andy, 34–35 DJ Kay Slay, 102 DJI drones, 231 DLA Piper lobbying firm, 300 DNNresearch, 182 Doblet phone-charging startup, 149, 177 Dodgeball location-based app, 104, 105, 118 Donahue, Nick, 260–61 Dota 2 (video game), 215–18, 221–22, 242, 284 dot-com bubble of the late 1990s, 73, 87, 93 Dowling, Steve, 287 Dropbox, 123, 139, 150, 152, 157, 158–59, 247 drop-outs, tech bros as mostly, 62, 88, 91–92, 96, 108, 113, 124, 132, 175, 179, 217–18 Dubai, 274 eBay, 82, 168 economic growth democracy and, 152–53, 225 dot-com bubble of the late 1990s, 73, 87, 93 financial crisis of 2008, 4, 116–17, 131, 137, 150 as a kind of spiritual hack, 153 real wages, 132 Edge web browser, 272 Edison, Thomas, 46 effective altruism, 2, 4, 6 189–214, 224, 265–66, 276, 300 “bed nets” era of, 212 deworming charities, 211, 213 GreaterWrong forum, 301 Open Philanthropy Project, 213–14, 241, 266–67, 276–77, 299–301 see also AI safety Effective Ventures, 276 Efficient Market Hypothesis, 144 Eichler, Mike, 30, 40 80,000 Hours, 211–12 Elbaz, Gil, 243 Electronic Arts, 97 Electronic Frontier Foundation (EFF), 59, 98 energy production, 13, 134, 231, 280, 313–14 nuclear energy, 12–13, 57, 108, 134–36, 154, 177, 205, 230, 259, 280 solar energy, 28 see also compute Enigma machine, 139 Enlightenment, 314–15 Enron, 60 esports, 215–16 Evans, Jon, 142 Extropians, 141–42, 144–45, 164–65, 199, 214 Fabolous (DJ), 102 Facebook, 60, 63, 73, 79–80, 96, 101, 116–17, 123, 127, 131, 137, 161, 169, 184, 188, 204, 207–8, 229 fact-checking, 265 Fairchild Semiconductor, 72, 87 fake news, 265 Federal Communications Commission (FCC), 59, 106 Federal Trade Commission, 285 Feldman, Ellen, 173 Fellow Robots robotics company, 210 Fermi’s paradox, 170 “few shot” learning, 244 “Feynman method of being a genius,” 219 Filan, Daniel, 305 financial crisis of 2008, 4, 116–17, 131, 137, 150 “finding product-market fit,” 107, 156 “finding your tribe,” 157 Finney, Hal, 142 Firefox web browser, 63 Flexport logistics platform, 138 Florida, 40 “Flowers for Algernon” (Keyes), 140 Foo Camp, 63 Forbes (magazine), 257 Foreign Affairs (journal), 267 Foresight Institute, a technology think tank, 144 Forstall, Scott, 112, 114, 116 “Founder Mode” (Graham), 263 founders as kings, 6, 60, 65, 68, 70–75, 95 Founders at Work (Livingston), 63, 64 Founders Fund venture firm, 2, 6, 132, 139, 147, 226 Foursquare, 118–20, 126 Francis, Peggy, 32, 36 “Free Oceana,” 141 Freemasonry, 31 Fridman, Lex, 17, 304 Friend, Tad, 201 From Zero to One (Thiel and Masters), 132, 230 frontier in American history, 153 FroSoCo (short for Freshman Sophomore College), 55, 57 FTX crypto market, 212, 257 Furstenberg, Diane von, 229 Future of Humanity Institute, 5, 165–66, 241–42, 267 Future of Life conference, Puerto Rico (2015), 167–70, 207, 211 Future of Life Institute, 145, 168, 208–9, 272 Galef, Julia, 225 Galois, Évariste, 174 game theory, 166, 285 Gates, Bill, 65, 90, 212, 216, 267–68 Gates, Melinda, 212 Gauss, Carl Friedrich, 174 Gawker Media, 137, 204–5 gay marriage rights, 56, 296 Gay Straight Alliance, 52 Gaza, war in, 289 GE (General Electric), 210 Gebbia, Joe, 263 Gebru, Timnit, 252–53, 270–71 Gemini AI model, 307 general artificial intelligence (AGI), see AI (artificial intelligence) generative AI, 1, 3, 9, 219, 221, 270 generative pre-trained transformers (GPTs), 3, 221, see also various GPTs under OpenAI genius, human, 77, 81, 127, 140, 156, 219 “Gentle Seduction, The” (Stiegler), 199–200 George, Henry, 256 Georgia, 23–25, 58, 104, 142 GeoSim 3D modeling company, 209–10 Germany, 274 Gibney, Bruce, 132–33 Gibstine, Connie (Sam’s mother), 15, 31–38, 39–50, 53–54, 91, 201–2, 227–28, 248–50, 261, 295, 312 Gil, Elad, 136 Gillette, 62, 71 Ginsberg, Allen, 166 Girard, René, 131 Github Copilot, 262 Gittell, Ross, 35 GiveWell, 212–13, 266 Giving Pledge, 212 Giving What We Can, 211 global positioning system (GPS) chips in mobile phones, 57–58, 99 Gmail, 123, 150, 286–87 Go (game), 191–92, 216–17 “godfathers of AI,” 188, 312 Goertzel, Ben, 145 Goetz, Jim, 88 Goldman Sachs, 59, 64, 150, 225 Good Ventures foundation, 212–13 Google AdSense, 243 Alphabet, 194, 271 “Attention Is All You Need” (“the transformer paper”), 218–19, 270 Bard conversational model, 271 Chrome web browser, 272 DeepMind acquisition, 146–48, 154, 165, 168–69, 171–72, 184, 189–94, 208, 211, 217, 221, 270 Dodgeball location-based app acquisition, 104, 105, 118, 104 Gemini AI model, 307 Gmail, 123, 150, 286–87 recent initial public offering (IPO) of, 87 YouTube acquisition, 93–94 see also Anthropic Google assistant, 271 Google Brain, 82, 169, 184, 243, 270 Google Colab Notebook, 247 Google I/O annual developer conference, 307 Google Maps, 59 Google Search, 270 Gordon-Levitt, Joseph, 210, 225 GotNews (website), 204 GPTs (generative pre-trained transformers), see chatbots; various GPTs under OpenAI Graham, Paul, 3, 13–16, 62–65, 67–76, 81–82, 94, 136, 149, 151, 186, 263 “A Unified Theory of VC Suckage,” 72–73 “Collison installation,” 246 “Founder Mode,” 263 “How to Start a Startup,” 69–70 painting’s influence on, 67–69 see also Y Combinator (YC) graphical user interfaces, 195 graphics processing units (GPUs), 176, 181–82, 219, 247, 255 Gras, Mike, 249 GreaterWrong forum, 301 Green Dot prepaid debit card company, 126–27, 133 Green-Lowe, Jason, 301–2 Grey, Aubrey de, 144, 259 Groom, Lachy, 151–52, 154, 157 Gross, Daniel, 309 Groupon, 119, 116 Guardian, The (newspaper), 220 Gurevich, Mikhail, 77, 82 Gurson, Doktor, 149, 177 Hacker News message board, 132, 151 hackers/hacking, 3, 57, 63, 68–70, 160n, 162 Haffner & Gibstine Real Estate, 32 Halcyon Molecular, 257–59 Hall, Ed, 26 Halo 3 (video game), 95, 109 ham radios, 32–33, 48 Hanson, Robin, 141–42, 144 Harder, Josh, 207 “hardware strategy, the,” 226, see also compute Harris, Kamala, 273, 313 Harris, Sylvia, 24–25 Hartford Courant (newspaper), 28 Hartford, CT, 28–29 Hartford Institute of Criminal and Social Justice, 28 Hartmann, Frank, 28–29 Harvard Computer Society, 69 Hassabis, Demis, 145–47, 169, 171–72, 192, 209 Hassenfeld, Elie, 212 Hawaii, 227, 250, 260, 295, 311–12 Hawking, Stephen, 169, 211 Hawkins, Trip, 97 HBO’s Silicon Valley, 101, 258–59 HBO’s Westworld, 199 heads, frozen, 141 Health Extension Foundation, 258 Helion Energy nuclear fusion startup, 13, 136, 207, 259, 280, 298 Helo telepresence robot, 210 Her (film), 307 Herzog, Isaac, 274 Hill, Daniel, 165 Hilton, Jacob, 265 Hinduism, 17 Hinton, Geoff, 178–84, 188, 266, 312–13 hip-hop, 102 Hipmunk travel search company, 162 HIV/AIDS, 33, 43, 49 Hoffman, Reid, 109, 172, 223, 231, 234–35, 277, 296 Hogan, Hulk, 205 Holocaust, attempted analogies to the, 135 Homejoy, 195 Horizon Institute for Public Service, 301 housing, see affordable housing Houston, Drew, 157 “How to Start a Startup” (Graham), 69–70 Howard, James, 110–11 Howl (Ginsberg), 166 Huffman, Steve, 69–70, 75, 76–78, 81, 162–63 Hui, Fan, 192 Human Advancement Research Community (HARC), 195, 197–98, 236 humans in a democratic society, 135 human genius, 77, 81, 127, 140, 156, 219 human-computer interfaces, 210 learning to truly produce knowledge, 314 neuroscience and studying the human brain, 145–48, 202 “only good for hugs or sex,” 269 their brains as the original neural nets, 181 see also chatbots Hunnewell, H.

pages: 336 words: 91,806

Code Dependent: Living in the Shadow of AI
by Madhumita Murgia
Published 20 Mar 2024

OpenAI took a hefty investment of more than $10bn from Microsoft and converted itself into what was, for all intents and purposes, a for-profit enterprise that sold AI technologies to large corporations and governments around the world.4 OpenAI’s crown jewel was an algorithm called GPT – the Generative Pre-trained Transformer – software that could produce text-based answers in response to human queries. One of the authors of the ‘Attention Is All You Need’ paper, Lukasz Kaiser, had ended up working there and helping to build it. It was an impressive piece of technology but until November in 2022 it was small-scale, clunky and mostly in the hands of tech-savvy programmers.

…

.: ‘The Machine Stops’ ref1 Fortnite ref1 Foxglove ref1 Framestore ref1 Francis, Pope ref1, ref2 fraudulent activity benefits ref1 gig workers and ref1, ref2, ref3 free will ref1, ref2 Freedom of Information requests ref1, ref2, ref3 ‘Fuck the algorithm’ ref1 Fussey, Pete ref1 Galeano, Eduardo ref1 gang rape ref1, ref2 gang violence ref1, ref2, ref3, ref4 Gebru, Timnit ref1, ref2, ref3 Generative Adversarial Networks (GANs) ref1 generative AI ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 AI alignment and ref1, ref2, ref3 ChatGPT see ChatGPT creativity and ref1, ref2, ref3, ref4 deepfakes and ref1, ref2, ref3 GPT (Generative Pre-trained Transformer) ref1, ref2, ref3, ref4 job losses and ref1 ‘The Machine Stops’ and ref1 Georgetown University ref1 gig work ref1, ref2, ref3, ref4, ref5 Amsterdam court Uber ruling ref1 autonomy and ref1 collective bargaining and ref1 colonialism and ref1, ref2, ref3 #DeclineNow’ hashtag ref1 driver profiles ref1 facial recognition technologies ref1, ref2, ref3, ref4 fraudulent activity and ref1, ref2, ref3, ref4 ‘going Karura’ ref1 ‘hiddenness’ of algorithmic management and ref1 job allocation algorithm ref1, ref2, ref3, ref4, ref5, ref6 location-checking ref1 migrants and ref1 ‘no-fly’ zones ref1 race and ref1 resistance movement ref1 ‘slaveroo’ ref1 ‘therapy services’ ref1 UberCheats ref1, ref2, ref3 UberEats ref1, ref2 UK Supreme Court ruling ref1 unions and ref1, ref2, ref3 vocabulary to describe AI-driven work ref1 wages ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10, ref11 work systems built to keep drivers apart or turn workers’ lives into games ref1, ref2 Gil, Dario ref1 GitHub ref1 ‘give work, not aid’ ref1 Glastonbury Festival ref1 Glovo ref1 Gojek ref1 ‘going Karura’ ref1 Goldberg, Carrie ref1 golem (inanimate humanoid) ref1 Gonzalez, Wendy ref1 Google ref1 advertising and ref1 AI alignment and ref1 AI diagnostics and ref1, ref2, ref3 Chrome ref1 deepfakes and ref1, ref2, ref3, ref4 DeepMind ref1, ref2, ref3, ref4 driverless cars and ref1 Imagen AI models ref1 Maps ref1, ref2, ref3 Reverse Image ref1 Sama ref1 Search ref1, ref2, ref3, ref4, ref5 Transformer model and ref1 Translate ref1, ref2, ref3, ref4 Gordon’s Wine Bar London ref1 GPT (Generative Pre-trained Transformer) ref1, ref2, ref3, ref4 GPT-4 ref1 Graeber, David ref1 Granary Square, London ref1, ref2 ‘graveyard of pilots’ ref1 Greater Manchester Coalition of Disabled People ref1 Groenendaal, Eline ref1 Guantanamo Bay, political prisoners in ref1 Guardian ref1 Gucci ref1 guiding questions checklist ref1 Gulu ref1 Gumnishka, Iva ref1, ref2, ref3, ref4 Gutiarraz, Norma ref1, ref2, ref3, ref4, ref5 hallucination problem ref1, ref2, ref3 Halsema, Femke ref1, ref2 Hanks, Tom ref1, ref2 Hart, Anna ref1 Hassabis, Demis ref1 Harvey, Adam ref1 Have I Been Trained ref1 healthcare/diagnostics Accredited Social Health Activists (ASHAs) ref1, ref2, ref3 bias in ref1 Covid-19 and ref1, ref2 digital colonialism and ref1 ‘graveyard of pilots’ ref1 heart attacks and ref1, ref2 India and ref1 malaria and ref1 Optum ref1 pain, African Americans and ref1 qTrack ref1, ref2, ref3 Qure.ai ref1, ref2, ref3, ref4 qXR ref1 radiologists ref1, ref2, ref3, ref4, ref5, ref6 Tezpur ref1 tuberculosis ref1, ref2, ref3 without trained doctors ref1 X-ray screening and ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 heart attacks ref1, ref2 Herndon, Holly ref1 Het Parool ref1, ref2 ‘hiddenness’ of algorithmic management ref1 Hikvision ref1, ref2 Hinton, Geoffrey ref1 Hive Micro ref1 Home Office ref1, ref2, ref3 Hong Kong ref1, ref2, ref3, ref4, ref5 Horizon Worlds ref1 Hornig, Jess ref1 Horus Foundation ref1 Huawei ref1, ref2, ref3 Hui Muslims ref1 Human Rights Watch ref1, ref2, ref3, ref4 ‘humanist’ AI ethics ref1 Humans in the Loop ref1, ref2, ref3, ref4 Hyderabad, India ref1 IBM ref1, ref2, ref3, ref4 Iftimie, Alexandru ref1, ref2, ref3, ref4, ref5 IJburg, Amsterdam ref1 Imagen AI models ref1 iMerit ref1 India ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9 facial recognition in ref1, ref2, ref3 healthcare in ref1, ref2, ref3 Industrial Light and Magic ref1 Information Commissioner’s Office ref1 Instacart ref1, ref2 Instagram ref1, ref2 Clearview AI and ref1 content moderators ref1, ref2, ref3, ref4 deepfakes and ref1, ref2, ref3 Integrated Joint Operations Platform (IJOP) ref1, ref2 iPhone ref1 IRA ref1 Iradi, Carina ref1 Iranian coup (1953) ref1 Islam ref1, ref2, ref3, ref4, ref5 Israel ref1, ref2, ref3 Italian government ref1 Jaber, Faisal bin Ali ref1 Jainabai ref1 Janah, Leila ref1, ref2, ref3 Jay Gould, Stephen ref1 Jewish faith ref1, ref2, ref3, ref4 Jiang, Mr ref1 Jim Crow era ref1 jobs application ref1, ref2, ref3 ‘bullshit jobs’ ref1 data annotation and data-labelling ref1 gig work allocation ref1, ref2, ref3, ref4, ref5, ref6 losses ref1, ref2, ref3 Johannesburg ref1, ref2 Johnny Depp–Amber Heard trial (2022) ref1 Jones, Llion ref1 Joske, Alex ref1 Julian-Borchak Williams, Robert ref1 Juncosa, Maripi ref1 Kafka, Franz ref1, ref2, ref3, ref4 Kaiser, Lukasz ref1 Kampala, Uganda ref1, ref2, ref3 Kellgren & Lawrence classification system. ref1 Kelly, John ref1 Kibera, Nairobi ref1 Kinzer, Stephen: All the Shah’s Men ref1 Knights League ref1 Koli, Ian ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9, ref10 Kolkata, India ref1 Koning, Anouk de ref1 Laan, Eberhard van der ref1 labour unions ref1, ref2, ref3, ref4, ref5, ref6 La Fors, Karolina ref1 LAION-5B ref1 Lanata, Jorge ref1 Lapetus Solutions ref1 large language model (LLM) ref1, ref2, ref3 Lawrence, John ref1 Leigh, Manchester ref1 Lensa ref1 Leon ref1 life expectancy ref1 Limited Liability Corporations ref1 LinkedIn ref1 liver transplant ref1 Loew, Rabbi ref1 London delivery apps in ref1, ref2 facial recognition in ref1, ref2, ref3, ref4 riots (2011) ref1 Underground terrorist attacks (2001) and (2005) ref1 Louis Vuitton ref1 Lyft ref1, ref2 McGlynn, Clare ref1, ref2 machine learning advertising and ref1 data annotation and ref1 data colonialism and ref1 gig workers and ref1, ref2, ref3 healthcare and ref1, ref2, ref3 predictive policing and. ref1, ref2, ref3, ref4 rise of ref1 teenage pregnancy and ref1, ref2, ref3 Mahmoud, Ala Shaker ref1 Majeed, Amara ref1, ref2 malaria ref1 Manchester Metropolitan University ref1 marginalized people ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8, ref9 Martin, Noelle ref1, ref2, ref3, ref4, ref5, ref6, ref7 Masood, S.

…

pages: 347 words: 100,038

This Is for Everyone: The Captivating Memoir From the Inventor of the World Wide Web
by Tim Berners-Lee
Published 8 Sep 2025

In the late 2010s, a research team at Google developed a powerful new neural net architecture termed a ‘transformer’. Following this, researchers at OpenAI, which was then a non-profit start-up, adopted Google’s transformer architecture. They began training it on large quantities of data, using it to generate new text. They called this tool a Generative Pre-trained Transformer, or GPT. The earliest versions of GPT were not very impressive – partly because the data sets OpenAI researchers were using weren’t large enough. GPT-1, for example, trained on a corpus of self-published e-books, mostly from the romance and science fiction categories, and its output was closer to nonsense than meaningful human prose.

…

The printing, copying, redistribution, or retransmission of this Content without express written permission is prohibited Index Aadhaar ref1 Aaron, Swartz ref1 Abou-Zahra, Shadi ref1, ref2 Abramatic, Jean-François ref1 academic papers, JSTOR ref1 accessibility ref1, ref2, ref3, ref4, ref5, ref6 ActiveX ref1 activism, hostile (Edelman Trust Barometer) ref1 Adam Smith lecture ref1 addiction, social media ref1, ref2, ref3, ref4 Addis, Louise ref1 Adelman, Len ref1 Adobe ref1 advertisements browsers ref1 cookies ref1 first clickable ref1, ref2 microtargeting ref1, ref2 pop-up ref1 privacy ref1 social media ref1, ref2, ref3 third-party distribution networks ref1, ref2 affordability ref1 Africa ref1, ref2, ref3 agents ref1, ref2, ref3, ref4 AJAX platform ref1 Akamai Technologies ref1, ref2 al-Sisi, Abdel Fattah ref1 Alexa ref1, ref2 Alexa Internet ref1 Alexander, Helen ref1 algorithms consistent hashing ref1 PageRank ref1 public key cryptography ref1 social media ref1, ref2, ref3, ref4, ref5, ref6 Alibaba ref1, ref2 Alice in Wonderland (Carroll) ref1 ‘alignment problem’ ref1 AlphaFold ref1, ref2 AlphaGo ref1 AlphaZero ref1 AltaVista ref1, ref2 ‘always on’ ref1 Amazon ref1, ref2, ref3, ref4, ref5, ref6 Andreessen Horowitz venture-capital fund ref1 Andreessen, Marc ref1, ref2, ref3, ref4, ref5, ref6, ref7 Android ref1 Anklesaria, Farhad ref1 Anonymous ref1 AOL ref1, ref2, ref3, ref4 AOL hometown ref1 Apache HTTP servers ref1 Apollo naming system ref1, ref2 Apple anti-trust lawsuits ref1 apps ref1 business model ref1 HyperCard ref1 interoperability ref1 iPhone ref1, ref2, ref3 Jobs leaves ref1 Jobs returns ref1 partnerships ref1 Siri ref1 standards ref1 WHATWG ref1, ref2, ref3 Applied Semantics ref1 apps interoperability ref1, ref2 killer apps ref1 smartphones ref1 web apps ref1 Arab Spring ref1 Archer, Mary ref1 archives ref1, ref2 Arena browser ref1, ref2 ARPANET ref1 Arroyo, James ref1 artichokes ref1 artificial intelligence (AI) AI ‘agents’ ref1, ref2, ref3 ‘AI winter’ ref1, ref2 authors and musician’s concerns ref1 autonomy ref1 Charlie ref1, ref2 copyright infringement ref1 DeepMind ref1 Ditchley Summit ref1, ref2, ref3 early development ref1 future possibilities ref1, ref2 global summits ref1 GOFAI ref1 GPTs (Generative Pre-trained Transformers) ref1, ref2, ref3, ref4, ref5 ‘human in the loop’ ref1 Inflection.AI ref1 intention economy ref1 military applications ref1 need for inclusivity ref1 neural networks ref1, ref2, ref3, ref4 OpenAI ref1, ref2, ref3, ref4, ref5 paradigm shift ref1 RAGs (Retrieval-Augmented Generation systems) ref1 reinforcement learning from human feedback ref1 search engines ref1 semantic web ref1 simplified text ref1 singularity ref1 speed of development ref1 superintelligence ref1 trust ref1 see also ChatGPT Asimov, Isaac ref1, ref2, ref3, ref4 Association for Computing Machinery (ACM) ref1 atheism ref1, ref2 Athumi ref1 Atkinson, Bill ref1 Attenborough, David ref1 attention economy ref1, ref2, ref3, ref4 attention spans ref1 audio descriptions ref1 audiobooks ref1 augmented reality ref1 Australia ref1, ref2, ref3 authentication ref1 authoritarians ref1, ref2, ref3, ref4 Autodesk ref1 Baidu ref1 bar-code scanners ref1 Barabasi, Albert-Laszlo ref1 Barlow, John Perry ref1, ref2, ref3 Barton, Nick ref1, ref2 BBC ref1, ref2, ref3 Beihang University, Beijing ref1, ref2 Beijing ref1 Belgium ref1, ref2 Bell Labs ref1 Bellingcat organization ref1 Bengio, Yoshua ref1 Berkman Klein Center for Internet and Society ref1, ref2 Berners-Lee, Alice (daughter) ref1, ref2, ref3, ref4 Berners-Lee, Ben (son) ref1, ref2, ref3, ref4, ref5 Berners-Lee, Conway (father) ref1, ref2, ref3, ref4, ref5 Berners-Lee, Mary Lee (mother) ref1, ref2, ref3, ref4, ref5, ref6, ref7 Berners-Lee, Rosemary see Leith, Rosemary Berners-Lee, Tim awards ref1, ref2, ref3, ref4, ref5, ref6 character ref1, ref2, ref3 childhood and education ref1, ref2, ref3, ref4 children ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8 cottage in Wales ref1 D.G.

…

Nash ref1, ref2 digital commons ref1 digital divide ref1 Digital Equipment Corporation (DEC) ref1 digital signatures ref1 dishwasher poem (GPT-4) ref1 disinformation ref1, ref2 Ditchley Foundation ref1, ref2, ref3, ref4 DNS (Domain Name System) ref1, ref2, ref3 documentation systems ref1, ref2 documents compared to data ref1 doomscrolling ref1, ref2 dot-com bubble ref1, ref2, ref3 dot-matrix printers ref1 Doubleclick ref1 Dougherty, Dale ref1 Dow Jones ref1 Dropbox ref1 e-commerce ref1 E-Trade ref1 East Sheen, London ref1 ECMAScript ref1 Edelman, Richard ref1 Edelman Trust Barometer ref1 Edge ref1 Egypt ref1 eigenvectors 151n Elbaz, Gil ref1 electromagnets ref1 Electronic Frontier Foundation (EFF) ref1, ref2, ref3 Elizabeth II ref1, ref2, ref3, ref4 email ref1, ref2 Emanuel School ref1, ref2, ref3 encryption ref1, ref2, ref3, ref4 engagement, algorithms ref1, ref2, ref3, ref4 Engelbart, Douglas ref1, ref2 Enigma cipher ref1 ‘Enquire-within’ program ref1, ref2, ref3 Enquire Within Upon Everything ref1 equality ref1 Equifax ref1 error codes ref1 Erwise ref1 Eternal September ref1 Ethiopia ref1 Euler’s formula ref1 European Commission ref1 European Computer Manufacturers Association (ECMA) ref1 European Council for Nuclear Research see CERN European Semiconductor Equipment Company ref1 European Union Brexit ref1 GDPR (General Data Protection Regulation) ref1, ref2 Ex Machina (film, 2014) ref1 Excite ref1 Expedia ref1 Facebook advertisements ref1 Africa ref1 AI training ref1 Arab Spring ref1 collaborative filtering and polarization ref1, ref2 data breaches ref1 data ownership ref1 microtargeting ref1 users as the product ref1, ref2 facts, encoding ref1, ref2 farming ref1 Fediverse ref1, ref2, ref3 Fermilab ref1 Ferranti ref1, ref2, ref3 Ferranti, Basil de ref1 File Transfer Protocol (FTP) ref1, ref2 Filo, David ref1 Finland ref1 Firefox ref1, ref2 First Parish Church, Lexington ref1 Flametree ref1 Flanders ref1, ref2 Fora ref1 Ford Foundation ref1 Forth (programming language) ref1 forums ref1 Foster, Norman ref1 free speech ref1 Freud, Lucien ref1 Friendster ref1 Fry, Stephen ref1 Gal, Yarin ref1 Gallaudet University ref1, ref2 garbage in, garbage out ref1 Gates, Bill ref1, ref2, ref3, ref4 GDPR (General Data Protection Regulation) ref1, ref2 GEC ref1 Gemini ref1 generalization ref1 Geneva ref1, ref2, ref3, ref4, ref5 Geneva Amateur Operatic Society (GAOS) ref1 genome-sequencing providers ref1 Geocities ref1 geospatial mapping ref1 Ghana ref1 Gifford, David ref1 GIFs ref1, ref2 Gilliat, Bruce ref1 Gilmore, John ref1 Gilyard-Beer, Peter ref1 Glasswing ref1, ref2 Global News Network (GNN) ref1 Gmail ref1 Go (game) ref1, ref2 Gods of Literature ref1, ref2 Goldstein, Jono ref1, ref2 Google AJAX ref1 anti-trust lawsuits ref1 Applied Semantics purchase ref1 China ref1 Chrome ref1 DeepMind ref1 Gemini ref1, ref2 Gmail ref1 Google Docs ref1 Google Maps ref1, ref2 Google Meet ref1 hosted web anniversary dinner ref1 HTML and WHATWG ref1 passkeys ref1 search functionality ref1, ref2 standards ref1 start-up ref1 transformers ref1 users as the product ref1 Web Index ref1 Zeitgeist network event ref1 Google Maps ref1, ref2 Gopher ref1, ref2 Gore, Al ref1, ref2, ref3, ref4 government data ref1, ref2, ref3 governments and Contract for the Web ref1, ref2, ref3 GPS data ref1, ref2 GPTs (Generative Pre-trained Transformers) ref1, ref2, ref3, ref4 GPUs (Graphical Processing Units) ref1 Grail ref1 graph structures ref1, ref2 Great Firewall of China ref1, ref2 Greif, Irene ref1 Grisham, John ref1 Grok AI ref1 Grundy, Frank ref1, ref2 hacktivism ref1 Hall, Justin ref1 Halonen, Tarja ref1 Hanamura, Wendy ref1 haptic touch ref1 Harari, Yuval Noah ref1, ref2 Hardin, Joseph ref1, ref2, ref3 Harris, Kamala ref1 Harris, Tristan ref1 Harrison, George ref1 Harvard Berkman Klein Center for Internet and Society ref1, ref2 Institute for Rebooting Social Media ref1 Hassabis, Demis ref1, ref2 Hawke, Sandro ref1 Helsinki ref1 Helsinki University of Technology ref1 Hendler, Jim ref1 Herzberg, Frederick ref1 Heywood, Jeremy ref1 Hickson, Ian (Hixie) ref1, ref2 hierarchies ref1 Higgins, Eliot ref1 Higgs boson ref1 Higgs, Peter ref1 Hinton, Geoffrey ref1 holiday bookings ref1, ref2 Home Depot ref1 home pages ref1 Hoogland, Walter ref1, ref2 Hoschka, Philipp ref1, ref2 HTML (Hypertext Markup Language) ref1, ref2, ref3, ref4, ref5, ref6 HTTP (Hypertext Transfer Protocol) ref1, ref2, ref3, ref4, ref5 HTTPS standard ref1 human first systems ref1 human rights ref1, ref2, ref3, ref4, ref5, ref6 see also civil liberties humans as social animals ref1 HyperCard ref1 hyperlinks ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8 hypertext ref1, ref2, ref3 see also HTML Hypertext ‘91 conference ref1 Hypertext ‘93 conference ref1 Ibargüen, Alberto ref1 IBM, Watson ref1 identity theft ref1 IETF (Internet Engineering Task Force) ref1, ref2, ref3 images GIFs ref1, ref2 IMG tag ref1 PNG (Portable Network Graphics) ref1 Imitation Game, The (film, 2014) ref1 ‘Imitation Game’ (Turing) ref1, ref2 inclusivity ref1, ref2 India ref1, ref2 indigenous communities, land rights ref1 Industrial Revolution ref1 Inflection.AI ref1 Infosys ref1 infrastructure evolution ref1 INRIA (National Institute for Research in Digital Science and Technology), France ref1, ref2 Inrupt ref1, ref2, ref3, ref4 Instagram ref1, ref2, ref3 Institute for Rebooting Social Media ref1 integrated circuits ref1 intellectual property rights ref1, ref2 intelligence agencies ref1, ref2 intention economy ref1, ref2, ref3, ref4 intercreativity ref1, ref2, ref3, ref4 International Standards Organization (ISO) ref1 internet Al Gore funding bill ref1, ref2 blackouts ref1 deep web ref1 early development ref1 early functionality ref1 free ethos ref1, ref2, ref3 Map of Everything ref1, ref2, ref3 protocols ref1, ref2 size ref1 universal access ref1, ref2 see also World Wide Web Internet Archive ref1, ref2 Internet Explorer ref1, ref2, ref3, ref4, ref5 internet service providers (ISPs) ref1, ref2, ref3 interoperability ref1, ref2, ref3, ref4, ref5 Intuit ref1 invisible pixels ref1 IP addresses ref1, ref2, ref3 iPhone ref1, ref2, ref3 Iran ref1 ITT ref1 Jambon, Jan ref1 Java ref1 JavaScript ref1, ref2, ref3 Jitsi ref1 Jobs, Steve ref1, ref2, ref3, ref4, ref5, ref6, ref7 Jones, Peter ref1 JScript ref1 JSTOR ref1 Justin’s Links from the Underground ref1, ref2 Kagame, Paul ref1 Kahle, Brewster ref1, ref2 Kahn, Bob ref1, ref2 Kapor, Mitch ref1 Keio University, Tokyo ref1 Kendall, Alex ref1 Kenya ref1 killer apps ref1 Kirk, Anna ref1 Kirk, Matthew ref1 Knight Foundation ref1 Kotok, Alan ref1 Krotoski, Aleks ref1 Kubrick, Stanley ref1 Kunz, Paul ref1 Kurzweil, Ray ref1 labelling reality or fakes ref1 land rights ref1 LANs (Local Area Networks) ref1 Larkin, Philip ref1 Lassila, Ora ref1 Last.fm ref1 Le Corbusier ref1 LeCun, Yann ref1 Legal Information Institute (LII) ref1, ref2 Legg, Shane ref1 legislation Data Use and Access Bill ref1 GDPR (General Data Protection Regulation) ref1, ref2 mobile phone spectrum ref1 open data ref1 Leighton, Tom ref1 Leith, Rosemary (wife) Berkman Klein Center for Internet and Society ref1 investments ref1 marriage to Tim ref1 sailing ref1, ref2 visits Beijing ref1 web anniversary speech ref1 Web Foundation ref1, ref2, ref3 Lenat, Doug ref1 Lessig, Lawrence ref1 Lewin, Daniel ref1 Li, Angel ref1 liability of hosts ref1 Library of Alexandria ref1, ref2 Library of Congress ref1 LibreOffice ref1 Libya ref1 Lie, Hakon ref1, ref2, ref3 lifeloggers ref1, ref2 lists ref1 Literature, Gods of ref1, ref2 LLMs (Large Language Models) ref1, ref2, ref3, ref4 LocalFirst Community ref1 location data ref1 logic agents ref1, ref2 logic gates ref1, ref2, ref3 London 2012 Olympics ref1, ref2 Lovett, Adrian ref1 Lycos ref1 Lynx ref1 Ma, Jack ref1, ref2 Mac OS X ref1 MacArthur Fellowship ref1 McBryan, Oliver ref1 McCahill, Mark ref1 McCartney, Paul ref1, ref2 McCourt, Frank ref1 machine learning collaborative filtering ref1 neural networks ref1, ref2 see also artificial intelligence machine translation ref1 McManus, Richard ref1 MacWWW ref1 Malamud, Carl ref1 Mandelbrot ref1 maps geospatial mapping ref1 Google Maps ref1, ref2 Map of Everything on the Internet ref1, ref2, ref3 OpenStreetMap ref1, ref2 World Wide Web Middle Earth ref1 Marcos, Bongbong ref1 Markey, Ed ref1, ref2 Martin, George R.

pages: 194 words: 57,434

The Age of AI: And Our Human Future
by Henry A Kissinger , Eric Schmidt and Daniel Huttenlocher
Published 2 Nov 2021

Even after the antibiotic was discovered, humans could not articulate precisely why it worked. The AI did not just process data more quickly than humanly possible; it also detected aspects of reality humans have not detected, or perhaps cannot detect. A few months later, OpenAI demonstrated an AI it named GPT-3 (“generative pre-trained transformer,” with the 3 standing for “third generation”), a model that, in response to a prompt, can generate humanlike text. Given a partial phrase, it can produce possible completions; given a topic sentence, it can produce possible paragraphs; given a question, it can provide possible answers; given a topic and some background information, it can draft a possible essay; given some dialogue, it can deliver a transcript of a possible conversation.

pages: 660 words: 179,531

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by Karen Hao
Published 19 May 2025

But after pursuing it further, he was surprised by the results. The Transformer had improved quickly and performed much better on a range of language processing tasks, such as summarizing or answering questions about a document, than anything else he had tried before. In 2018, OpenAI released the first version of that model, called Generative Pre-Trained Transformer, later nicknamed GPT-1. The second word in the name—pre-trained—is a technical term within AI research that refers to training a model on a generic pool of data as a prerequisite for it to learn more specific tasks later. GPT-1, in other words, had been trained on a generic pool of English to create a rough approximation of how the language worked.

…

See also Doomers p(doom) (probability of doom), 232, 250, 317, 319–20, 377 expected values, 229–30 expert systems, 94–95 Exploratory Research, 149, 151–52 extinction, 24, 26–27, 55, 232, 378 extractivism, 104, 417 in Chile, 272, 273–74, 281–85, 296–99 in Uruguay, 291–96 use of term, 104n F Facebook, 11, 15, 16, 51–52, 105, 154, 159, 162, 192, 209, 230, 321, 334 facial recognition, 57, 103, 104, 115, 161, 435n Fact Factory, 261 fair use, 91 Fairwork, 202, 206, 416 Federal Trade Commission (FTC), 239, 308, 358 Fedus, Liam, 247, 406 “Feel the AGI,” 120, 255 Feynman, Richard, 121–22 firefighting, 237, 260 first mover’s advantage, 103 Flamingo Generation, 220–21 Floyd, George, 152–53 “fluid data territory,” 299 Formula One, 1, 231 Founders Fund, 38 Foursquare, 32 fraud, 25, 250, 267 free speech, 368–69 Friar, Sarah, 404 Fridman, Lex, 383 Friedman, Milton, 272–73 friendly AI, 57, 319–20 Friend, Tad, 26–27, 31 frontier model, 305–11 Frontier Model Forum, 305–6, 309 FTX, 231–32, 233 bankruptcy, 257–58, 322, 380 FTX Future Fund, 231–32 Fuentes Anaya, Oskarina Veronica, 197–202, 415–17 Future Perfect, 388 Futures of Artificial Intelligence Research, 273–74 G Gates, Bill, 68 congressional testimony of, 311 GPT-4, 245–48 OpenAI demo, 71–72, 132–33, 246 Gates Demo, 71–72, 132–33, 246 Gawker Media, 38 GDPR (General Data Protection Regulation), 136 Gebru, Timnit, 24, 52–53, 108, 160–70, 171–73, 414 Generative Pre-Trained Transformers. See GPT Genius Makers (Metz), 80 Geometric Intelligence, 110 Ghost Work (Gray and Suri), 193–94 Gibstine, Connie, 29–31, 44, 327–28, 331–32, 333, 337 Gibstine, Marvin, 29 GitHub, 135–36, 182–84, 237, 243, 336 Codex, 184, 243, 247, 269, 318 Copilot, 184, 237, 336 GiveWell, 230–31, 322 Global South, 16, 89, 165, 186, 190, 193, 222, 278, 291, 416.

pages: 260 words: 82,629

The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip
by Stephen Witt
Published 8 Apr 2025

He had seen Shazeer and Kaiser’s proof-of-concept with the fake Wikipedia articles and thought it could be expanded upon. First, the model would “pretrain” on a large collection of text. Then it would generate text of its own. Combining the purpose, the method, and the architecture, you arrived at the “Generative Pre-Trained Transformer,” or GPT. GPT-1 launched in June 2018. It learned to read using BookCorpus, a collection of around seven thousand free, self-published ebooks. (Sci-fi, romance, and fantasy were the predominant genres—many of the books were Twilight knockoffs.) Drawing from this curriculum of third-rate vampire fiction, the first GPT was as bad as you might expect, answering users’ queries with streams of Dadaist nonsense.

pages: 285 words: 86,858

How to Spend a Trillion Dollars
by Rowan Hooper
Published 15 Jan 2020

Rather than program all the possible outcomes into the software – which is what software engineers used to try to do, with inevitable shortcomings – in machine learning with a neural network, the computer learns on its own. There has been spectacular success with a turbo form of machine learning called deep learning; it’s behind the ability of DeepMind’s AlphaGo and AlphaZero, and it’s the basis of a system developed by OpenAI called Generative Pre-trained Transformer, or GPT. A publicly available version called GPT-2 can generate original text, perhaps a sports report, a movie review, or maybe even poetry, when given a prompt. It is a kind of neural network that relies on what’s called unsupervised learning. That is, it has been exposed to lots of data (in this case, some 8 million text documents scraped off the internet), but like AlphaZero had to learn chess by itself, GPT-2 had to figure out what it all means by itself.

pages: 418 words: 102,597

Being You: A New Science of Consciousness
by Anil Seth
Published 29 Aug 2021

the humans failed: The description of the Turing test as a test of ‘human gullibility’ comes from a 2015 New York Times article by John Markoff, ‘Software is smart enough for SAT, but still far from intelligent’, New York Times, 21 September 2015. See www.nytimes.com/2015/09/21/technology/personaltech/software-is-smart-enough-for-sat-but-still-far-from-intelligent.html. vast artificial neural network: GPT stands for ‘Generative Pre-trained Transformer’ – a type of neural network specialised for language prediction and generation. These networks are trained using an unsupervised deep learning approach essentially to ‘predict the next word’ given a previous word or text snippet. GPT-3 has an astonishing 175 billion parameters and was trained on some 45 terabytes of text data.

pages: 284 words: 96,087

Supremacy: AI, ChatGPT, and the Race That Will Change the World
by Parmy Olson

Sutskever started asking people the same thing when he walked around the office, according to someone who worked there at the time: “Can you make it bigger?” Thanks to the transformer, Radford was making more progress with his language model experiments in two weeks than over the previous two years. He and his colleagues started working on a new language model they called a “generatively pre-trained transformer” or GPT for short. They trained it on an online corpus of about seven thousand mostly self-published books found on the internet, many of them skewed toward romance and vampire fiction. Plenty of AI scientists had used this same dataset, too, known as BooksCorpus, and anyone could download it for free.

pages: 301 words: 105,209

Searches: Selfhood in the Digital Age
by Vauhini Vara
Published 8 Apr 2025

Researchers had fed this model huge amounts of text—billions of words—so that it could learn the associations among words and phrases; from there, when prompted with a given series of words, the model could statistically predict what should come next. The most recent version was called GPT-3, short for Generative Pre-trained Transformer 3. I found examples of GPT-3’s work, and they astonished me. Some of them could easily be mistaken for texts written by a human hand. In others, the language was weird, off-kilter—but often poetically so, almost truer-seeming than writing any human would produce. When The New York Times asked GPT-3 to generate a piece in the style of its Modern Love column, where people share stories about their love lives, it wrote, “We went out for dinner.

System Error: Where Big Tech Went Wrong and How We Can Reboot
by Rob Reich , Mehran Sahami and Jeremy M. Weinstein
Published 6 Sep 2021

—Sheila Jasanoff, The Ethics of Invention, 2016 Chapter 8 Can Democracies Rise to the Challenge? In early 2019, a young company called OpenAI made an announcement that immediately caused waves in the scientific community. OpenAI had created an extremely powerful AI-driven tool called GPT-2 (Generative Pre-trained Transformer, model 2) capable of generating surprisingly high-quality text. It does so with nothing more than a minimal prompt; a single sample sentence will do, such as “Write an essay about Toni Morrison’s Beloved.” The GPT-2 language model is extremely flexible, able to translate, answer questions, and summarize and synthesize other texts in addition to generating text of many different kinds, including remarkably plausible poetry, journalism, fiction, academic papers, essays for middle school, and even computer code.

pages: 416 words: 118,522

Why Machines Learn: The Elegant Math Behind Modern AI
by Anil Ananthaswamy
Published 15 Jul 2024

“Initially, we thought it was a fluke and dug deeper into it. It turned out to be something that happens pretty reliably.” The neural network that Power and colleagues were using was called a transformer, a type of architecture that’s especially suited to processing sequential data. LLMs such as ChatGPT are transformers; GPT stands for “generative pre-trained transformer.” Given a sequence of, say, ten words and asked to predict the next most plausible word, a transformer has the ability to “pay attention” to all the words at once and also to the order of the words and not just treat them as some arbitrary jumble. Of course, commercial LLMs are behemoths, with tens or even hundreds of billions of parameters.

pages: 444 words: 117,770

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma
by Mustafa Suleyman
Published 4 Sep 2023

Based on this, it then suggests which tokens should come next in the sequence, what output logically follows the input. In other words, it autocompletes what might come next. These systems are called transformers. Since Google researchers published the first paper on them in 2017, the pace of progress has been staggering. Soon after, OpenAI released GPT-2. (GPT stands for generative pre-trained transformer.) It was, at the time, an enormous model. With 1.5 billion parameters (the number of parameters is a core measure of an AI system’s scale and complexity), GPT-2 was trained on 8 million pages of web text. But it wasn’t until the summer of 2020, when OpenAI released GPT-3, that people started to truly grasp the magnitude of what was happening.

AI 2041: Ten Visions for Our Future
by Kai-Fu Lee and Qiufan Chen
Published 13 Sep 2021

There is none of the special-purpose labeling we described in the previous section. With enough natural data and sufficient processing power, the system can learn on its own to detect arrival and departure times, and a great deal more. After Google’s transformer work, a more well-known extension called GPT-3 (GPT stands for “generative pre-trained transformers”) was released in 2020 by OpenAI, a research laboratory founded by Elon Musk and others. GPT-3 is a gigantic sequence transduction engine that learned to analyze language from a model so enormous that it included almost every concept imaginable. Leveraging one of the most powerful supercomputers in the world, GPT-3 was trained on more than 45 terabytes of text, which would take 500,000 lifetimes for a human to read.

pages: 619 words: 177,548

Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity
by Daron Acemoglu and Simon Johnson
Published 15 May 2023

From the Field of AI Dreams People are right to be excited about advances in digital technologies. New machine capabilities can massively expand the things we do and can transform many aspects of our lives for the better. And there have also been tremendous advances. For example, the Generative Pre-trained Transformer 3 (GPT-3), released in 2020 by OpenAI, and ChatGPT released in 2022 by the same company, are natural-language processing systems with remarkable capabilities. Already trained and optimized on massive amounts of text data from the internet, these programs can generate almost human-like articles, including poetry; communicate in typical human language; and, most impressively, turn natural-language instructions into computer code.