Reflections on Trusting Trust

Your Computer Is on Fire
by Thomas S. Mullaney , Benjamin Peters , Mar Hicks and Kavita Philip
Published 9 Mar 2021

In his acceptance speech for this award, titled “Reflections on Trusting Trust,”2 Thompson reminds the audience that he hadn’t worked actively on Unix in many years. After doing this act of performative humility, Thompson devotes his speech to a description of “the cutest program he ever wrote.” This program was an implementation of a method of creating Trojan horse programs—software with secret backdoors, which the nefarious creator of the Trojan horse could then use to hijack computer systems on which the software was installed. I will refer to the method that Thompson outlines in “Reflections on Trusting Trust” as the “Thompson hack” for the remainder of this chapter.3 Nevertheless, it is useful to note that what I call the Thompson hack was not wholly the result of Thompson’s original research.

…

However, it also holds its own inherent interest, since this layer of translation standing between human-friendlier programming languages and code that computers can run is one of the deepest elements in the deep-layered software stack that allows for the creation of software platforms. The primer on programming language hierarchies is followed by a close read of “Reflections on Trusting Trust.” Although this material is fairly technical, enough apparatus is provided for nonprogramming readers to understand both the key turning points of Thompson’s explanation of the hack and the significance of the hack in understanding software platforms. Once the technical work of explicating the Thompson hack’s methodology is finished, I turn to analyzing the social implications of the hack, and how those implications may differ from the ones that Thompson himself proposes in “Reflections.”

…

This game involved writing programs that would, when compiled and run, produce complete listings of their own source code as output. The winner would be the programmer who had produced the shortest self-replicating program. The method Thompson used to produce a self-replicating pronoun that he presents in “Reflections on Trusting Trust” is inspired by analytic philosopher W. V. O. Quine’s variant of the liar’s paradox. Most versions of this paradox, like the well-known “this sentence is false” formulation, contain demonstrative words referring to the sentences themselves. In “this sentence is false,” the demonstrative word is “this.”

Turing's Vision: The Birth of Computer Science
by Chris Bernhardt
Published 12 May 2016

“Symbolic Analysis of Relay and Switching Circuits,” Transactions American Institute of Electrical Engineers, vol. 57, 1938, pp. 38–80. [45] Sipser, Michael. Introduction to the Theory of Computation, Cengage Learning, 2012. [46] Soare, Robert. “Formalism and intuition in computability,” Phil. Trans. R, soc. A, (2012) 370, pp. 3277–3304. [47] Thompson, Ken. “Reflections on Trusting Trust,” Communications of the ACM, August 1984, vol. 27, no. 8, pp. 761–763. [48] Tibor, Radó. “On non-computable functions,” Bell System Technical Journal 41 (3) pp. 877–884, 1962. [49] Turing, Alan. “Computing machinery and intelligence,” Mind 1950, 59, 433–460. [50] Turing, Alan. “On Computable Numbers, with an Application to the Entscheidungsproblem,” Proceedings of the London Mathematical Society, Series 2, 42 (1936–7), pp. 230–265

…

See also Acceptance problem; Blank tape problem; Halting problem Universal computer/machine, 12, 87, 91 u-substitution, 62 von Neumann, John, 26, 97, 148, 155, 164 von Neumann architecture, 97, 155 Whitehead, Alfred North, Principia Mathematica, 7, 8, 10, 16 Wiener, Norbert, 26 Williams, Frederick, 156 Wolfram, Stephen, 85, 103, 164 Zuse, Konrad, 154 1 “Reflections on Trusting Trust” was presented by Ken Thompson in 1983. It was published in the Communications of the ACM and is widely available on the web.

This Is How They Tell Me the World Ends: The Cyberweapons Arms Race
by Nicole Perlroth
Published 9 Feb 2021

In dissecting those chips down to the very bit, Gosler could see that these advancements—and the complexity they introduced—would only create more room for error, malfunction, and eventually enemy subversion and attack. The previous year, Gosler had heard a famous lecture by Ken Thompson. Thompson, who had won the 1983 Turing Award for cocreating the Unix operating system, used his turn at the lectern to share his concerns on where technology was headed. He’d titled his lecture “Reflections on Trusting Trust,” and his conclusion was this: unless you wrote the source code yourself, you could never be confident that a computer program wasn’t a Trojan horse. Thompson had perfectly articulated what Gosler knew to be true. But by the time Gosler listened to Thompson’s lecture, he could see that the predicament was getting exponentially worse.

…

The reference to Sandia’s role in developing 97 percent of America’s non-nuclear weapons components is available on Sandia’s website: “Evaluating Nuclear Weapons: A Key Sandia Mission.” Eric Schlosser provided an entertaining, and disturbing, account of America’s nuclear weapons accidents in his 2013 book, Command and Control: Nuclear Weapons, the Damascus Accident and the Illusion of Safety (Penguin Press). Ken Thompson’s 1984 Turing Award speech, “Reflections on Trusting Trust,” is available here: www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf. Gosler’s Chaperon Experiments were also detailed in a 2016 dissertation by Craig J. Weiner, at George Mason University, titled: “Penetrate, Exploit, Disrupt, Destroy: The Rise of Computer Network Operations as a Major Military Innovation.”

…

., warnings to the, here Pwn2Own hacking contest, here Qatar, here Q Group (NSA), here Qualcomm, here ransomware attacks, here, here, here Ratcliffe, John, here Rather, Dan, here Raymond, Eric S., here Raytheon, here Reagan, Ronald, here, here, here, here Reckitt Benckiser, here Reddit, here “Reflections on Trusting Trust” (Thompson), here Retz, Dave, here Rhodes, Benjamin, here Rice, Alex, here, here, here Richarte, Gerardo (Gera), here, here Rizzo, Juliano, here, here Robertson, Pat, here Rogen, Seth, here, here Rogers, Michael, here Romney, Mitt, here room taps, here Rosenberg, Paul, here Rosneft (Russia), here, here RSA, here, here, here Russia break-up of the Soviet Union, here cyberespionage, here disinformation campaigns, here economy, here election interference (2016), here, here, here, here, here, here, here, here election interference (2020), here EternalBlue, use of, here kompromat, here, here ransomware attacks, here Ukraine, invasion of, here U.S. cyberattacks on, here U.S. grid, vulnerability to, here, here U.S. grid attacks in, here U.S. sanctions, here voter registration system hacks, here, here, here WannaCry ransomware in, here Russia, cyberattacks DNC, here, here, here, here, here, here, here, here outsourcing, here range of, here State Department, here, here TrickBot, here, here, here, here, here warning issued by, here White House, here Russia, cyberattacks in Ukraine election interference, here, here elections, here election systems, here factors limiting, here individuals, here infrastructure, here, here, here, here, here, here, here media attacks, here nuclear plants, here purpose, here al-Saadan, Abdullah, here Sabien, Jimmy, here, here, here Safari, here, here, here Safari (Apple), here Sagan, Carl, here Sahin, Tamer, here Said, Missoum, here Salehi, Ali Akbar, here Samba, here Sandberg, Sheryl, here Sanders, Bernie, here, here, here, here Sandia National Labs, here, here, here Sandler, Adam, here Sands casinos, here, here Sandworm (Russia), here, here Sanger, David, here, here, here, here, here, here, here, here, here, here, here, here Saudi Arabia, here, here, here, here, here Schlesinger, James, here Schmidt, Eric, here, here, here, here, here Schmidt, Howard, here Schneier, Bruce, here Schroeder, Gerhard, here Schulte, John, here SecurityFocus, here September here, 2001 terrorist attacks, here, here, here, here, here Sequoia Capital, here Shadow Brokers, here, here, here, here, here, here, here, here Shane, Scott, here, here, here, here, here, here, here Shwedo, Bradford “B.

Fancy Bear Goes Phishing: The Dark History of the Information Age, in Five Extraordinary Hacks
by Scott J. Shapiro

This statute limited the criminal offense to three specific scenarios—unauthorized access to obtain national security secrets, personal financial records from financial institutions or credit agencies, and hacking into government computers. devoted his lecture to cybersecurity: Kenneth Thompson, “Reflections on Trusting Trust,” Communications of the ACM, August 1984, https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ ReflectionsonTrustingTrust.pdf. The Turing lecture series was inaugurated in 1967. air force testers: Karger and Schell provided the first public description of the problem that compilers can insert malicious code into themselves.

…

do the same to UNIX: David Wheeler proposed a countermeasure against the Thompson attack using two different compilers, in David Wheeler, Fully Countering Trusting Trust Through Diverse Double-Compiling (PhD diss., George Mason University, 2009), https://dwheeler.com/trusting-trust/dissertation/html/wheeler-trusting-trust-ddc.html. “only program you can truly trust”: Thompson, “Reflections on Trusting Trust.” appearing on: Patrick was also a witness at the congressional cybersecurity hearings. When asked by a member of the subcommittee whether WarGames was an inspiration, Patrick disappointed: “That didn’t instigate us at all.” Many hackers, however, have since claimed that the movie was indeed their first inspiration.

Protocol: how control exists after decentralization
by Alexander R. Galloway
Published 1 Apr 2004

Douglas McIlroy, head of the Computing Techniques Research Department at Bell Labs, and a program called Worm created by John Shoch (and Jon Hupp) of Xerox Palo Alto Research Center. See A. K. Dewdney, “Computer Recreations,” Scientific American, March 1984, p. 22. For more on Shoch and Hupp, see “The Worm Programs,” Communications of the ACM, March 1982. Many attribute the worm concept to the science fiction novel Shockwave Rider by John Brunner. 20. Ken Thompson, “Reflections on Trusting Trust,” in Computers Under Attack: Intruders, Worms, and Viruses, ed. Peter Denning (New York: ACM, 1990), p. 98. 21. Dewdney, “Computer Recreations,” p. 14. 22. Jon A. Rochlis and Mark W. Eichin, “With Microscope and Tweezers: The Worm from MIT’s Perspective,” in Computers Under Attack: Intruders, Worms, and Viruses, ed.

Dark Mirror: Edward Snowden and the Surveillance State
by Barton Gellman
Published 20 May 2020

See also Lawrence Joffe, “Abu Musab al-Zarqawi Obituary,” Guardian, June 8, 2006, at https://perma.cc/8T2C-NZFP. fist-bumping status report: On file with author. taking part in a criminal conspiracy: See chapter 7. Eventually he agreed to breakfast: James R. Clapper, interview with author, August 17, 2018. as long ago as 1984: Kenneth Thompson, “Reflections on Trusting Trust,” Turing Award lecture, reproduced in Communications of the ACM, August 1984, at https://perma.cc/NL2L-7JX3. the Gemalto gambit: This story came to light in Jeremy Scahill and Josh Begley, “The Great SIM Heist,” Intercept, February 19, 2015, https://theintercept.com/2015/02/19/great-sim-heist/.

Dawn of the Code War: America's Battle Against Russia, China, and the Rising Global Cyber Threat
by John P. Carlin and Garrett M. Graff
Published 15 Oct 2018

Nikki Finke, “A University Professor’s ‘Startling’ Experiments Began It All,” Los Angeles Times, January 31, 1988, articles.latimes.com/1988-01-31/news/vw-39340_1 _computer-virus; and “When Did the Term ‘Computer Virus’ Arise?” Scientific American, September 2, 1997, scientificamerican.com/article/when-did-the-term-compute/. 43. Michelle Slatalla and Joshua Quittner, Masters of Deception: The Gang That Ruled Cyberspace (HarperCollins, 1995), 16. 44. Ken Thompson, “Reflections on Trusting Trust,” Turing Award Lecture, Communications of the ACM, vol. 1, no. 8, 1984, www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf. 45. Linda Greenhouse, “House Approves Measure to Make Computer Fraud a Federal Crime,” New York Times, June 4, 1986, www.nytimes.com/1986/06/04/us/house-approves-measure-to-make-computer-fraud-a-federal-crime.html; Josephine Wolff, “The Hacking Law That Can’t Hack It,” Slate, September 27, 2016, www.slate.com/articles/technology/future_tense/2016/09/the_computer_fraud_and _abuse_act_turns_30_years_old.html; and Scott Mace, “Computer Bills in Works,” InfoWorld, October 14, 1985, books.google.com/books?

New York 2140
by Kim Stanley Robinson
Published 14 Mar 2017

A low discount rate makes the future more important, a high discount rate is dismissive of the future. —Frank Ackerman, Can We Afford the Future? The moral is obvious. You can’t trust code that you did not totally create yourself. Misguided use of a computer is no more amazing than drunk driving of an automobile. —Ken Thompson, “Reflections on Trusting Trust” A bird in the hand is worth what it will bring. noted Ambrose Bierce c) Franklin Numbers often fill my head. While waiting for my building’s morose super to free my Jesus bug from the boathouse rafters where it had spent the night, I was looking at the little waves lapping in the big doors and wondering if the Black-Scholes formula could frame their volatility.