stochastic parrot

Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by Karen Hao
Published 19 May 2025

“I can kind of see a paper taking shape here,” she continued, “using large language models as a case study for ethical pitfalls and what can be done better.” “Would you be interested in co-authoring such a thing?” she asked. Within two days, Bender had sent Gebru an outline. They later came up with a title, adding a cheeky emoji for emphasis: “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? “” * * * — Gebru assembled a research team for the paper within Google, including her colead Mitchell. In response to the encouraging words in Dean’s annual review, she flagged the paper as an example of the work she was pursuing. “Definitely not my area of expertise,” Dean said, “but would definitely learn from reading it.”

…

This would make people prone not only to believing the text to be factual information but also to consider the model a competent adviser, a trustworthy confidant, and perhaps even something sentient. In November, per standard company protocol, Gebru sought Google’s approval to publish the “Stochastic Parrots” paper at a leading AI ethics research conference. Samy Bengio, who was now her manager, approved it. Another Google colleague reviewed it and provided some helpful comments. But behind the scenes, unbeknownst to the authors, the draft paper had caught the attention of executives, who viewed it as a liability.

…

It highlighted myriad other issues, including the complete concentration of talent, resources, and technologies in for-profit environments that allowed companies to act so audaciously because they knew they had little chance of being fact-checked independently; the continued abysmal lack of diversity within the spaces that had the most power to control these technologies; and the lack of employee protections against forceful and sudden retaliation if they tried to speak out about unethical corporate practices. The “Stochastic Parrots” paper became a rallying cry, driving home a central question: What kind of future are we building with AI? By and for whom? * * * — For Jeff Dean, the dissolution of the ethical AI team delivered a direct blow to his reputation. As one of Google’s earliest employees, he had helped build the initial software infrastructure that made it possible for the company’s search engine to scale to billions of users.

These Strange New Minds: How AI Learned to Talk and What It Means
by Christopher Summerfield
Published 11 Mar 2025

Here’s a respected cognitive scientist exhorting us all to avoid #AIhype: With disbelief and discontent, I have since watched academics […] jumping on the bandwagon and enthusiastically surfing the AI hype wave, e.g., by talking enthusiastically about ChatGPT on national television or in public debates at universities, and even organizing workshops on how to use this stochastic parrot in academic education.[*4] The term ‘stochastic parrot’ is, of course, a reference to the famous paper, discussed in Part 3, which argues that claims of LLM capability are massively overblown – that they are simply parroting excerpts from their training data, and not doing anything remotely intelligent.[*5] As we have heard, this is a misconception, recycled from failed Chomskyan critiques of statistical modelling in NLP.

…

It is often used to argue that criteria for machine intelligence based on language output (such as the Turing Test) are irreparably flawed. The operator in the Chinese room is a parrot. They repeat the words found in the rulebook, without ever grasping their meaning. Many people have found this to be a compelling metaphor for LLMs. One highly influential paper describes LLMs as ‘stochastic parrots’ – where ‘stochastic’ indicates that they encode probabilities of transitions between words.[*5] An LLM is, the authors argue, a system for ‘haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning’.

…

This ability to deal with novel prompts distinguishes LLMs from the student who memorizes the answers to an exam (who will be stumped by a curveball question) and from the choirboy who learns to make Latin sounds that he cannot decipher (who will never be able to make polite chit-chat with the Pope). So, although it is a witty rebuke, LLMs are not just ‘stochastic parrots’. They resemble parrots in that any words they use are ultimately copied from humans. But humans also learn language from other humans, so this is hardly a fatal flaw. LLMs can put words and concepts together in ways that they have never observed before, allowing sophisticated answers to unexpected queries.

The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future
by Keach Hagey
Published 19 May 2025

As the Anthropic crew was heading for the door at OpenAI in late 2020, a similar fight over safety had broken out at Google over the publication of a controversial paper by lead researchers Emily Bender and Timnit Gebru called “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”14 The title’s menacing image of a plumed colossus combines the talking birds’ famous knack for imitation and an uncommon word—“stochastic,” derived from the Greek stokhastikos, which is related to English’s “conjecture.” The phrase “stochastic parrots,” then, refers to the propensity of large language models to produce guesswork and mimicry, as opposed to thoughtful analysis and human communication.

…

Google said reviewers found the paper too critical and asked the co-authors to retract it. Gebru claims she was then fired, while Google maintains she resigned.15 The affair exploded into the press and turned “Stochastic Parrots” into one of the most-cited critiques of AI and a cultural meme. (Shortly after OpenAI released ChatGPT the following year, Altman would cheekily tweet, “I am a stochastic parrot and so r u.”)16 But in many ways, the paper validated and gave voice to the sort of fears that Amodei, Brundage, Clark, and others at OpenAI had felt just a couple years earlier, which had led the company to hesitate to release the full source code for GPT-2.

…

It Has Learned to Code (and Blog and Argue),” The New York Times, November 24, 2020. 7.Paul Graham, “Do Things That Don’t Scale,” PaulGraham.com, July 2013. 8.Annie Altman, “How I Started Escorting,” Medium, March 27, 2024. 9.Weil, “Oppenheimer of Our Age.” 10.Annie Altman, “How I Started Escorting.” 11.Sam Altman, “Please Fund More Science,” Sam Altman blog, March 30, 2020. 12.Greg Brockman, Mira Murati, Peter Welinder, OpenAI, “OpenAI API,” OpenAI blog, June 11, 2020. 13.Tom Simonite, “OpenAI’s Text Generator Is Going Commercial,” Wired, June 11, 2020. 14.Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Margaret Mitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021. 15.Emily Bobrow, “Timnit Gebru Is Calling Attention to the Pitfall of AI,” The Wall Street Journal, February 24, 2023. 16.Sam Altman @sama, “I am a stochastic parrot and so r u,” Twitter, December 4, 2022. CHAPTER 15CHATGPT 1.Tom Simonite, “It Began as an AI-Fueled Dungeon Game. It Got Much Darker,” Wired, May 5, 2021. 2.Ibid. 3.Sam Altman, “Moore’s Law for Everything,” Sam Altman, March 16, 2021. 4.Ibid. 5.Richard Nieva, “Sam Altman’s Eyeball-Scanning Crypto Project Worldcoin Is Having an Identity Crisis,” Forbes, August 10, 2023. 6.Chafkin, 138. 7.Antonio Regalado, “A Startup Pitching a Mind-Uploading Service That Is ‘100 Percent Fatal,’ ” MIT Technology Review, March 13, 2018. 8.Antonio Regalado, “Sam Altman Invested $180 Million into a Company Trying to Delay Death,” MIT Technology Review, March 8, 2023. 9.Haje Jan Kamps, “Helion Secures $2.2B to Commercialize Fusion Energy,” TechCrunch, November 5, 2021. 10.Friend, “Manifest Destiny.” 11.Katherine Long, Hugh Langley, “OpenAI CEO Sam Altman Went on an 18-Month, $85-Million Real Estate Shopping Spree—Including a Previously Unknown Hawaii Estate,” Business Insider, November 30, 2023. 12.Samson Zhang, “Donahue,” Postulate, July 20, 2021. 13.Annie Altman, @anniealtman108, “I experienced sexual, physical, emotional, verbal, financial, and technological abuse from my biological siblings, mostly Sam Altman and some from Jack Altman,” X, November 13, 2021. 14.Annie Altman, “How I Started Escorting.” 15.Annie Altman, @anniealtman108, “If the multiverse is real, I want to meet the version of me who did run away to the circus at age 5 years old about wanting to end this life thing and being touched by older siblings, and said ‘mother’ decided to instead protect her sons and demand to receive therapy and chores only from her female child.”

Supremacy: AI, ChatGPT, and the Race That Will Change the World
by Parmy Olson

Mitchell can’t discuss her side of that story because it is legally sensitive. The Stochastic Parrots paper hadn’t been all that earth-shattering in its findings. It was mainly an assemblage of other research work. But as word of the firings spread and the paper got leaked online, it took on a life of its own. Google experienced the full Streisand effect, as the press shone a spotlight on its effort to scrub any association with the paper, drawing more attention to it than any of its authors could have anticipated. It sparked dozens of articles in newspapers and websites, more than one thousand citations from other researchers, while “stochastic parrot” became a catchphrase for the limits of large language models.

…

Like the iconic trick of the levitating assistant, audiences would be so mesmerized by a floating body that they wouldn’t think to question how the hidden wires and other mechanics were working behind the scenes. Bender couldn’t stand the way GPT-3 and other large language models were dazzling their early users with what was, essentially, glorified autocorrect software. So she suggested putting “stochastic parrots” in the title to emphasize that the machines were simply parroting their training. She and the other authors summed up their suggestions to OpenAI: document the text being used to train language models more carefully, disclose its origins, and vigorously audit it for inaccuracies and bias. Gebru and Mitchell quickly submitted the paper for review through Google’s internal process, through which the company checked its researchers weren’t leaking any sensitive material.

…

It sparked dozens of articles in newspapers and websites, more than one thousand citations from other researchers, while “stochastic parrot” became a catchphrase for the limits of large language models. Sam Altman would later tweet, “I am a stochastic parrot and so r u” days after the release of ChatGPT. Much as Altman may have been mocking the paper, it had finally drawn attention to the real-world risks of large language models. At surface level, it seemed like Google’s approach to AI was “do no evil.” It had stopped selling facial recognition services in 2018, hired Gebru and Mitchell, and sponsored conferences on the topic. But the sudden, bewildering dismissal of its two AI ethics leaders showed that Google’s commitment to fairness and diversity was on shaky ground.

Escape From Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It
by Erica Thompson
Published 6 Dec 2022

Similarly, a model that is produced in a certain cultural context, such as a British tech company consisting mainly of Oxbridge engineering graduates, would draw on the mathematical and social traditions of that environment both for the conceptual and the technical construction of the model and the definition of adequacy-for-purpose. And this often happens without the makers even realising that any other perspectives exist. Clever horses and stochastic parrots Clever Hans was a performance act at the beginning of the twentieth century, a horse that was able to tap a hoof to indicate the correct answer to basic arithmetic questions. The show was very convincing, but further investigation revealed that the horse was responding to subtle changes in body language in his handler and could only give the right answer if his handler knew it.

…

But it’s difficult, because most of the researchers are WEIRD and educated at elite schools that emphasise a dominant paradigm of science where a model’s accuracy is the highest virtue. Questions of power, bias and implications for marginalised communities do not arise naturally because they do not personally affect the majority of these researchers. Linguist Emily Bender, AI researcher Timnit Gebru and colleagues have written about ‘the dangers of stochastic parrots’, referring to language models that can emulate English text in a variety of styles. Such models can now write poetry, answer questions, compose articles and hold conversations. They do this by scraping a huge archive of text produced by humans – basically most of the content of the internet plus a lot of books, probably with obviously offensive words removed – and creating statistical models that link one word with the probability of the next word given a context.

…

This is the reason why it is so important to ensure that test data are as different as possible from training data, to remove the potential confounding factors that make it look like the model is doing well when it is not. Of course, both Clever Hans and the tank detector were doing interesting things – just not what their handlers thought they were doing. The ‘stochastic parrot’ language models are doing very interesting things too, and their output may be indistinguishable from human-generated text in some circumstances, but that by itself is not sufficient to justify their use. Explainability and ‘Just So’ stories Apart from finding as much different test data as possible, how can we spot our model horses before they produce horse manure?

The Long History of the Future: Why Tomorrow's Technology Still Isn't Here
by Nicole Kobie
Published 3 Jul 2024

A third challenge concerns the very nature of LLMs: the paper dubs them ‘stochastic parrots’, meaning they mimic without understanding. The AI system is not applying meaning, that’s up to readers to do. This is why so many people were surprised by ‘hallucinations’ or ‘lies’ in responses from ChatGPT – they didn’t have the AI literacy to understand that such systems do not understand meaning or context, they merely spit out text that matches the pattern of our language. They do it incredibly well, and it fools us. These stochastic parrots create language without meaning. Any meaning in the output is supplied by the reader, and is therefore an illusion.

…

ProPublica, May 23, 2016. https://biturl.top/2UZ36j Lighthill Controversy Debate, BBC, 1973. https://biturl.top/N7nIje Bender, Emily M.; Gebru, Timnit; McMillan-Major, Angelina; Mitchell, Margaret. ‘Statement from the listed authors of Stochastic Parrots on the “AI pause” letter.’ DAIR Institute, March 31, 2023. https://biturl.top/neqMJr Bender, Emily M.; Gebru, Timnit; McMillan-Major, Angelina; Shmitchell, Shmargaret. ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?’ FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, March 2021, pp. 610–623. https://doi.org/10.1145/3442188.3445922 Bernhardt, Chris.

Why Machines Learn: The Elegant Math Behind Modern AI
by Anil Ananthaswamy
Published 15 Jul 2024

GO TO NOTE REFERENCE IN TEXT “scientific papers from the arXiv preprint server”: Ethan Dyer and Guy Gur-Ari, Google Research, Blueshift Team, “Minerva: Solving Quantitative Reasoning Problems with Language Models” Google Research (blog), June 30, 2022, https://blog.research.google/2022/06/minerva-solving-quantitative-reasoning.html. GO TO NOTE REFERENCE IN TEXT EPILOGUE At a public talk I gave on ChatGPT: Anil Ananthaswamy, “ChatGPT and Its Ilk,” YouTube video, n.d., https://www.youtube.com/watch?v=gL4cquObnbE. GO TO NOTE REFERENCE IN TEXT “stochastic parrots”: Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, N.Y., March 2021, pp. 610–23. GO TO NOTE REFERENCE IN TEXT “Google Photos, y’all f*** [sic] up.

…

Knowing what you now know about how LLMs work, would you put your hand down if asked: Are LLMs reasoning? If you lowered your hand, you wouldn’t be alone. Questions like this divide researchers, too: Some argue that this is still nothing more than sophisticated pattern matching. (Emily Bender of the University of Washington and colleagues coined a colorful phrase for LLMs; they called them “stochastic parrots.”) Others see glimmers of an ability to reason and even model the outside world. Who is right? We don’t know, and theorists are straining to make mathematical sense of all this. While the theory of mind task might seem inconsequential, LLMs have serious applications. For example, LLMs fine-tuned on web pages containing programming code, are excellent assistants for programmers: Describe a problem in natural language, and the LLM will produce the code to solve it.

The Shame Machine: Who Profits in the New Age of Humiliation
by Cathy O'Neil
Published 15 Mar 2022

So if a camera picked up the face of a Black woman at a crime scene, the system would likely match it to a large number of innocent people. These disturbing findings led Amazon and Microsoft to stop selling the software to law enforcement. In 2020, Gebru and her growing team at Google turned their attention to the biases of “immense language models”—the raw material for much of Google’s AI. The paper—“On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”—worked to establish the statistical likelihood that racism and other biases would be embedded in Google’s automated services. The paper suggested, pointedly, that more culturally sensitive models could be developed by directing the machine learning on more focused data sets.

…

GO TO NOTE REFERENCE IN TEXT led Amazon and Microsoft to stop selling the software to law enforcement: Nitasha Tiku, “Google Hired Timnit Gebru to Be an Outspoken Critic of Unethical AI. Then She Was Fired for It,” The Washington Post, December 23, 2020, https://www.washingtonpost.com/​technology/​2020/​12/​23/​google-timnit-gebru-ai-ethics/. GO TO NOTE REFERENCE IN TEXT “On the Dangers of Stochastic Parrots”: Karen Hao, “We Read the Paper That Forced Timnit Gebru out of Google. Here’s What It Says,” MIT Technology Review, December 4, 2020, https://www.technologyreview.com/​2020/​12/​04/​1013294/​google-ai-ethics-research-paper-forced-out-timnit-gebru/. GO TO NOTE REFERENCE IN TEXT she denounced the company for censoring her: Casey Newton, “The Withering Email That Got an Ethical AI Researcher Fired at Google,” Platformer, December 3, 2020, https://www.platformer.news/​p/the-withering-email-that-got-an-ethical.

Four Battlegrounds
by Paul Scharre
Published 18 Jan 2023

Brooks, “Mistaking Performance for Competence Misleads Estimates Of AI’s 21st Century Promise And Danger,” in What to Think About Machines That Think: Today’s Leading Thinkers on the Age of Machine Intelligence (Edge Question Series) (New York: Harper Perennial, October 6, 2015), https://www.edge.org/response-detail/26057. 234gender, racial, and religious biases: Brown et al., Language Models are Few-Shot Learners; Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT ‘21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (March 2021), 610–623, https://dl.acm.org/doi/pdf/10.1145/3442188.3445922. 234Google Translate: James Kuczmarski, “Reducing Gender Bias in Google Translate,” The Keyword (blog), Google, December 6, 2018, https://blog.google/products/translate/reducing-gender-bias-google-translate/. 234gender distribution of nurses and doctors: “Professionally Active Physicians by Gender,” Kaiser Family Foundation, updated September 2021, https://www.kff.org/other/state-indicator/physicians-by-gender/; “Labor Force Statistics from the Current Population Survey: Household Data Annual Averages,” U.S.

…

FAccT ‘21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (March 2021), 610–623, https://dl.acm.org/doi/pdf/10.1145/3442188.3445922. 234Google Translate: James Kuczmarski, “Reducing Gender Bias in Google Translate,” The Keyword (blog), Google, December 6, 2018, https://blog.google/products/translate/reducing-gender-bias-google-translate/. 234gender distribution of nurses and doctors: “Professionally Active Physicians by Gender,” Kaiser Family Foundation, updated September 2021, https://www.kff.org/other/state-indicator/physicians-by-gender/; “Labor Force Statistics from the Current Population Survey: Household Data Annual Averages,” U.S. Bureau of Labor Statistics, updated January 22, 2021, https://www.bls.gov/cps/cpsaat11.htm. 234existing social biases: Bender et al., “On the Dangers of Stochastic Parrots.” 234option to choose the gender: Kuczmarski, “Reducing Gender Bias in Google Translate.” 234résumé-sorting model: Jeffrey Dastin, “Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women,” Reuters, October 10, 2018, https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G. 234Learning systems will sometimes find shortcuts: Ortega, Maini, and the DeepMind safety team, “Building Safe Artificial Intelligence.” 234learned to alternate from the previous input: Joel Lehman et al., “The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities,” Artificial Life 26, no. 2 (2020), 281–282, https://doi.org/10.1162/artl_a_00319. 234Simulated digital creatures: Lehman et al., “The Surprising Creativity of Digital Evolution,” 279–281. 235deception and concealment tactics: Lehman et al., “The Surprising Creativity of Digital Evolution,” 288–289. 235optimal scoring strategy was not to race at all: Jack Clark and Dario Amodei, “Faulty Reward Functions in the Wild,” OpenAI Blog, December 21, 2016, https://openai.com/blog/faulty-reward-functions/. 235Q*bert: Lehman et al., “The Surprising Creativity of Digital Evolution,” 285. 235win by crashing opposing algorithms: Lehman et al., “The Surprising Creativity of Digital Evolution,” 284. 235exploiting bugs in the simulation: Lehman et al., “The Surprising Creativity of Digital Evolution,” 283–285. 235evolved circuit on an FPGA chip: Adrian Thompson, “An Evolved Circuit, Intrinsic in Silicon, Entwined with Physics,” in: Tetsuya Higuchi, Masaya Iwata, and Liu Weixin, eds., Evolvable Systems: From Biology to Hardware (Berlin: Springer, 1996), https://doi.org/10.1007/3-540-63173-9_61. 235“game” or “hack” their reward functions: Victoria Krakovna et al., “Specification Gaming: the Flip Side of AI Ingenuity,” Deepmind Blog, April 21, 2020, https://deepmind.com/blog/article/Specification-gaming-the-flip-side-of-AI-ingenuity; Clark and Amodei, “Faulty Reward Functions in the Wild”; Amodei et al., Concrete Problems in AI Safety. 235deleted the files containing the “correct” answers: Lehman et al., “The Surprising Creativity of Digital Evolution,” 281. 235take credit for other rules: Douglas B.

…

Brown et al., Language Models are Few-Shot Learners (Cornell University, July 22, 2020), https://arxiv.org/pdf/2005.14165.pdf. 294Switch-C: William Fedus, Barret Zoph, and Noam Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (arXiv.org, January 11, 2021), https://arxiv.org/pdf/2101.03961.pdf. 294745 GB of text: Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” FAccT ‘21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (March 2021), 610–623, https://dl.acm.org/doi/pdf/10.1145/3442188.3445922. 294Megatron-Turing NLG: Ali Alvi and Paresh Kharya, “Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, the World’s Largest and Most Powerful Generative Language Model,” Microsoft Research Blog, October 11, 2021, https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/. 294825 GB of text: Leo Gao et al., The Pile: An 800GB Dataset of Diverse Text for Language Modeling (arXiv, December 31, 2020), https://arxiv.org/pdf/2101.00027.pdf. 294270 billion “tokens”: Alvi and Kharya, “Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B.” 294sixth-largest supercomputer: “SELENE,” Top500, https://www.top500.org/system/179842/. 294over 4,000 GPUs: Alvi and Kharya, “Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B.” 294PaLM: The 540 billion parameter version of PaLM was trained using “6144 TPU v4 chips running for 1,200 hours and 3072 TPU v4 chips running for 336 hours including some downtime and repeated steps.”

Everything Is Predictable: How Bayesian Statistics Explain Our World
by Tom Chivers
Published 6 May 2024

Even if it looks much more complicated than that—say, following the prompt “Write me a short story set in the Dune universe in the style of P. G. Wodehouse” with a two-thousand-word piece about Duke Augustus “Gussy” Atreides trying to stop his aunt marrying him off to a Harkonnen—it is still, skeptics say, just predicting what comes next. A famous paper in 2021 called them “stochastic parrots” and claimed that language models work by “haphazardly stitching together sequences of linguistic forms… without any reference to meaning.”22 But is this true? After all, one way to make good predictions is to build an accurate model of the world so that you can understand what is likely to happen in it.

…

Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 3rd ed. (Chennai, India: Pearson, 2010), 9. 21. Most of this section is taken from a conversation I had with Dr. William Woof of UCL, who uses AI and machine learning techniques to improve diagnosis of retinal diseases. 22. Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?,” in FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (New York: Association for Computing Machinery, 2021), 610–23, https://doi.org/10.1145/3442188.3445922. 23. Kenneth Li et al., “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task,” ArXiv, Cornell University, last updated February 27, 2023, https://doi.org/10.48550/arXiv.2210.13382. 24.

Searches: Selfhood in the Digital Age
by Vauhini Vara
Published 8 Apr 2025

It was better received, by far, than anything else I’d ever written. I thought I should feel proud, and to an extent I did. But I felt unsettled, too. Five months before the publication of “Ghosts,” the researchers Emily M. Bender and Timnit Gebru had written, with colleagues, a paper called “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In it, they made a convincing case that the methods used to train AI language models, in addition to requiring huge amounts of energy, could lead them to produce biased, even racist or misogynistic, language. Though AI companies didn’t disclose a lot about how they trained their models, OpenAI had described some of its training processes.

…

A philosopher might consider the question of whether AI can be conscious by asking whether it matters that GPT-3 doesn’t have a hand if it can produce credible text about having a hand. A literary critic might consider it similarly, in the context of Roland Barthes’s influential essay “The Death of the Author,” which argues for favoring a reader’s interpretation of a text over what the author might have intended. In “On the Dangers of Stochastic Parrots,” Bender and Gebru provide another useful lens through which to understand the question. “Human language use takes place between individuals who share common ground and are mutually aware of that sharing (and its extent), who have communicative intents, which they use language to convey, and who model each other’s mental states as they communicate,” they write.

AI in Museums: Reflections, Perspectives and Applications
by Sonja Thiel and Johannes C. Bernhardt
Published 31 Dec 2023

In this context, it is always important to keep in mind that the results generative AI technologies produce can be factual, but might also be speculative. For this reason, generative text production as it occurs in the context of large language models such as ChatGPT or GPT-4 is often likened to the figure of the ‘stochastic parrot’ (Bender et al. 2021, 610–23): like a parrot, AI technology is not capable of reflecting on what has been blended together from the data pool that has been fed into it. It is not able to check its own results for factuality, which is why the results must be critically questioned upon the input of a prompt.6 6 As they are able to imitate the human cultural performance of speaking, talking parrots are known mainly as linguistic curiosities.

…

In conclusion, Wishing Well can be taken as an interesting example of such an agenda, which facilitates co-creativity between humans and machines in the exhibition space, as well as conveying ethical dilemmas that are to be expected in any use of generative AI. Yannick Hofmann and Cecilia Preiß: Say the Image, Don’t Make It References Bender, Emily M. et al. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In: FACCT Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 610–23. https://doi.org/10.1145/3442188 .3445922 (all URLs here accessed in August 2023). Bonet, Lluis/Négrier, Emmanuel (2018). The Participative Turn in Cultural Policy: Paradigms, Models, Contexts.

Co-Intelligence: Living and Working With AI
by Ethan Mollick
Published 2 Apr 2024

Narayanan, “Quantifying ChatGPT’s Gender Bias,” AISnakeOil.com, April 26, 2023, https://www.aisnakeoil.com/p/quantifying-chatgpts-gender-bias. GO TO NOTE REFERENCE IN TEXT create a distorted and biased representation: E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (New York: Assocation for Computing Machinery, 2021), 610–23. GO TO NOTE REFERENCE IN TEXT Some of them just cheat: T. H. Tran, “Image Generators Like DALL-E Are Mimicking Our Worst Biases,” Daily Beast, September 15, 2022, https://www.thedailybeast.com/image-generators-like-dall-e-are-mimicking-our-worst-biases.

Amateurs!: How We Built Internet Culture and Why It Matters
by Joanna Walsh
Published 22 Sep 2025

Can Logic Help Save them?’, MIT News, 3 March 2023. 26.Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman et al., ‘On the Opportunities and Risks of Foundation Models’, arXiv, 12 July 2022. 27.Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Shmargaret Shmitchell, ‘On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?’, FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 610–23, see p. 615. 240 28.Bommasani et al., ‘On the Opportunities and Risks of Foundation Models’. 29.Maurice Merleau-Ponty, ‘Eye and Mind’, in The Primacy of Perception: And Other Essays on Phenomenological Psychology (Northwestern University Press, 1964), p. 189. 30.Lyotard, ‘The Sublime and the Avant-Garde’, p. 255. 31.Jameson, Postmodernism, p. 46. 32.Merleau-Ponty, ‘Eye and Mind’, p. 165. 33.Ibid., p. 162. 34.Yuriko Saito, Everyday Aesthetics (Oxford University Press, 2008), p. 24. 35.Geoff Dyer, ‘Diary: Why Can’t I See You?’

Enshittification: Why Everything Suddenly Got Worse and What to Do About It
by Cory Doctorow
Published 6 Oct 2025

This move attracted the attention of some of Google’s top scientists, including Timnit Gebru, a distinguished AI researcher whose career had involved stints at Apple, Microsoft, and Stanford before she came to Google to work on AI ethics. In 2021, Gebru and several outside peers wrote a paper titled “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” that was accepted for the Association for Computing Machinery’s highly selective Conference on Fairness, Accountability, and Transparency. This was a perfectly normal thing to happen at Google. Much of Google’s success, after all, was down to its ability to lure top academic researchers into industry, where they were offered a corporate version of tenure—the freedom to pursue their research interests and publish their results.

The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future
by Orly Lobel
Published 17 Oct 2022

Alex Mar, “Love in the Time of Robots,” Wired, October 17, 2017, https://www.wired.com/2017/10/hiroshi-ishiguro-when-robots-act-just-like-humans. 2. Hiroshi Ishiguro and Shuichi Nishio, “Building Artificial Humans to Understand Humans,” Journal of Artificial Organs 10, no. 3 (February 2007): 133–142. 3. Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?,” in FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (New York: Association for Computing Machinery, 2021), 610, https://doi.org/10.1145/3442188.3445922. 4. Sundar Pichai, “AI at Google: Our Principles,” The Keyword (blog), Google, June 7, 2018, https://www.blog.google/technology/ai/ai-principles/. 5.

More Everything Forever: AI Overlords, Space Empires, and Silicon Valley's Crusade to Control the Fate of Humanity
by Adam Becker
Published 14 Jun 2025

Google had initially developed the “transformer” architecture that LLMs are based on; OpenAI had just released GPT-3, using that architecture, in a private beta, and it was already making waves within the field. It was becoming clear that transformer LLMs were a hot area in AI. Seeing this, Gebru, her colead Margaret Mitchell, and two linguists at the University of Washington authored a paper about the problems with such LLMs. Titled “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” the paper laid out four major areas of concern. First, actually training these models can be very computationally intensive, leading to a huge carbon footprint. Training a model the size of GPT-3 has a carbon footprint roughly equivalent to flying an airplane across the United States and back three times, and other phases in the development of these models increase that footprint further before they are finished and released.