Supremacy: AI, ChatGPT, and the Race That Will Change the World
by
Parmy Olson
If you were a young AI researcher like Illia Polosukhin, you were sitting next to the people who’d invented these techniques. In early 2017, Polosukhin was getting ready to leave Google and was willing to take some risks. Inside one of the Google canteens, two floors below the office of Larry Page, the twenty-five-year-old Ukrainian was spitballing with two other researchers, Ashish Vaswani and Jakob Uszkoreit. His lunch mates also didn’t like following the conventions of other scientists in the building. Vaswani was hungry to work on a big project. Uszkoreit had been at Google for more than ten years and was wary of the way Google Brain’s incentive structure had morphed into something like a glorified academic institution; after hiring dozens of new graduates and academics, he was surrounded by people who mainly cared about being first author on a paper or getting published at a conference.
…
Google was loathe to release any new technology that could end up disrupting the success of its search business. Its executives and publicity team framed that approach as being one of caution, but more than anything, the company was obsessed with maintaining its reputation and the status quo. Soon, Google was going to experience what Ashish Vaswani describes as a “biblical moment.” As Google continued printing money from its advertising business, OpenAI was taking what looked like a monumental step toward AGI, and it wasn’t keeping anything under wraps. ACT 3 THE BILLS CHAPTER 10 Size Matters If you walked out of the headquarters of Google in sunny Mountain View, California, and drove north for about an hour, you’d eventually hit San Francisco, step out of your car, and shiver.
The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip
by
Stephen Witt
Published 8 Apr 2025
Polosukhin recognized that “self-attention” could be deployed in a similarly maximal way, probabilistically linking each word in the tree not just to other words in the sentence but also to thousands of other words throughout an entire text. Even a word that had appeared many paragraphs earlier might provide a contextual clue to what the next word meant. Polosukhin and Uszkoreit were joined by Ashish Vaswani, another Google researcher, and by early 2017 the three had built a rudimentary English-to-German translator based on the self-attention mechanism. Polosukhin and Uszkoreit had previously contributed to an internal Google program called “autobot,” which attempted to automatically write Wikipedia pages.
Code Dependent: Living in the Shadow of AI
by
Madhumita Murgia
Published 20 Mar 2024
This awakening was at first gradual, and then happened all at once, largely due to a single event, a watershed moment of AI entering our public square: the launch of ChatGPT. Like many breakthroughs in scientific discovery, the one that spurred this latest artificial intelligence advance came from a moment of serendipity. In early 2017, two Google research scientists, Ashish Vaswani and Jakob Uszkoreit, were in a hallway of the search giant’s Mountain View campus, discussing a new idea for how to improve machine translation, the AI technology behind Google Translate.1 The AI researchers had been working with another colleague, Illia Polosukhin, on a concept they called ‘self-attention’ that could radically speed up and augment how computers understand language.
The Singularity Is Nearer: When We Merge with AI
by
Ray Kurzweil
Published 25 Jun 2024
BACK TO NOTE REFERENCE 93 Rachel Syme, “Gmail Smart Replies and the Ever-Growing Pressure to E-Mail Like a Machine,” New Yorker, November 28, 2018, https://www.newyorker.com/tech/annals-of-technology/gmail-smart-replies-and-the-ever-growing-pressure-to-e-mail-like-a-machine. BACK TO NOTE REFERENCE 94 For a more detailed explainer on how transformers work, and the original technical paper, see Giuliano Giacaglia, “How Transformers Work,” Towards Data Science, March 10, 2019, https://towardsdatascience.com/transformers-141e32e69591; Ashish Vaswani et al., “Attention Is All You Need,” arXiv:1706.03762v5 [cs.CL], December 6, 2017, https://arxiv.org/pdf/1706.03762.pdf. BACK TO NOTE REFERENCE 95 Irene Solaiman et al., “GPT-2: 1.5B Release,” OpenAI, November 5, 2019, https://openai.com/blog/gpt-2-1-5b-release. BACK TO NOTE REFERENCE 96 Tom B.
Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI
by
Karen Hao
Published 19 May 2025
One of the applications I’m most eagerly awaiting.,” Twitter (now X), September 27, 2023, x.com/ilyasut/status/1707027536150929689. GO TO NOTE REFERENCE IN TEXT Sutskever would get up: A photo of Sutskever at the event. GO TO NOTE REFERENCE IN TEXT In August 2017, that changed: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez et al., “Attention Is All You Need,” in NIPS ’17: Proceedings of the 31st International Conference on Neural Information Processing Systems (December 2017): 6000–10, dl.acm.org/doi/10.5555/3295222.3295349. GO TO NOTE REFERENCE IN TEXT But Sutskever, who had focused: Sutskever’s PhD thesis work focused on recurrent neural networks.
On the Edge: The Art of Risking Everything
by
Nate Silver
Published 12 Aug 2024
GO TO NOTE REFERENCE IN TEXT has testified about his concerns: Cat Zakrzewski, Cristiano Lima-Strong, and Will Oremus, “CEO Behind ChatGPT Warns Congress AI Could Cause ‘Harm to the World,’ ” The Washington Post, May 17, 2023, washingtonpost.com/technology/2023/05/16/sam-altman-open-ai-congress-hearing. GO TO NOTE REFERENCE IN TEXT “Attention Is All”: Ashish Vaswani et al., “Attention Is All You Need,” arXiv, August 1, 2023, arxiv.org/abs/1706.03762. GO TO NOTE REFERENCE IN TEXT most rapidly adopted technologies: Krystal Hu, “ChatGPT Sets Record for Fastest-Growing User Base—Analyst Note,” Reuters, February 2, 2023, sec. Technology, reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01.