Mathematical Theory of Communication

The Information State: Politics in the Age of Total Control

by Jacob Siegel  · 24 Mar 2026  · 348pp  · 103,246 words

theory was born in 1948 when Shannon published a nonsecret version of the paper in The Bell System Technical Journal under the revised title, “A Mathematical Theory of Communication.” Contrary to our psychological projections, Shannon argued, communication is a physical event, not an exchange of meaning. Messages may incidentally convey meaning insofar as they

…

Computer,” Atlantic, March 20, 2017, https://www.theatlantic.com/technology/archive/2017/03/aristotle-computer/518697/. “These semantic aspects of communication” Claude E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal 27, no. 3 (1948): 379–423, https://doi.org/10.1002/j.1538-7305.1948.tb01338.x. “The machines of which

In Our Own Image: Savior or Destroyer? The History and Future of Artificial Intelligence

by George Zarkadakis  · 7 Mar 2016  · 405pp  · 117,219 words

and stored in the memory. Digital information is a long, long sequence of zeros and ones. Shannon’s breakthrough idea in his seminal paper ‘A Mathematical Theory of Communication’25 was to borrow the probabilistic mathematics of thermodynamics and apply them to the new field of telecommunications. Thermodynamics describes how molecules move as they

…

. Norbert Wiener was the grand visionary of cybernetics. Inspired by mechanical control systems, such as artillery targeting and servomechanisms, as well as Claude Shannon’s mathematical theory of communication and information, he articulated the theory of cybernetics in his landmark book, Cybernetics, of 1948.4 Godfather number two, Claude Shannon, was the genius who

…

(IOOI), and transform them to binary numbers (1001), you get the decimal result 9 (whatever that means …)! 25Shannon, C. E., and Weaver W. (1948), The Mathematical Theory of Communication. Champaign: University of Illinois Press. Shannon co-wrote the book with Warren Weaver, a pioneer in machine translation. 26I am rephrasing here an example given

Between Human and Machine: Feedback, Control, and Computing Before Cybernetics

by David A. Mindell  · 10 Oct 2002  · 759pp  · 166,687 words

narrative ends in 1945, when the Office of Scientific Research and Development (OSRD) closed down, and with the subsequent publication of Cybernetics , Claude Shannon’s “Mathematical Theory of Communications,” and the Massachusetts Institute of Technology (MIT) Radiation Laboratory series of textbooks on radar, electronics, and servomechanisms. These and other publications helped spread the results

…

records, stock market prices, production statistics, and the like.” 21 Just a few years later Claude Shannon expanded these ideas in his 1948 paper, “A Mathematical Theory of Communication,” which famously created information theory. 22 Following Nyquist’s and Hartley’s analyses years earlier, Shannon defined the act of communication as transferring a given

…

addresses Wiener’s prediction in more detail; and Blackman, Linear Data-Smoothing and Prediction , an extension of the 1948 work. 22. Shannon, “Mathematical Theory of Communication”; Shannon and Weaver, Mathematical Theory of Communication . 23. Shannon and Weaver, Mathematical Theory of Communication . For Shannon’s early work see Shannon to Bush, 16 February 1939, in Shannon, Claude Elwood Shannon , 455–56. 24. Shannon

…

, “Mathematical Theory of Communication,” 53n. The relationship between Shannon’s and Wiener’s work is more complex than described here. In a 1987 interview Shannon said, “I don’t

…

Shannon , xix). Semantic confusion sometimes exists over the “Weaver-Shannon” or the “Wiener-Shannon” theory of communication. The former, which derives from Shannon and Weaver, Mathematical Theory of Communication , is inaccurate because Weaver only translated Shannon’s work to make it more accessible (Weaver claimed no more). 25. Wiener, Cybernetics , 6. Bibliography Major Archival

…

): 531–49. Shannon, Claude Elwood. Claude Elwood Shannon: Collected Papers . Ed. N. J. A. Sloane and A. D. Wyner. New York: IEEE Press, 1993. ———. “A Mathematical Theory of Communication.” Bell System Technical Journal 27 (July–October 1948): 379–423, 623–56. ———. “Mathematical Theory of the Differential Analyzer.” Journal of Mathematics and Physics 20 (December

…

1941): 337–54. ———. “A Symbolic Analysis of Relay Switching Circuits.” Transactions of the AIEE 57 (1938): 713–23 . Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication . Chicago: University of Illinois Press, 1949. Shapin, Steven, and Simon Schaffer. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life . Princeton: Princeton University

…

, 227 and servomechanism theory, 168 , 208 –9 and Sperry, 213 , 221 and stability problem, 144 –45, 147 –49. See also Radiation Laboratory; Servomechanisms Laboratory “A Mathematical Theory of Communication” (Shannon), 7 , 320 mathematics: at Bell Labs, 113 Bush on, 145 –46 and dynamic tester, 241 and feedback, 141 and fire control, 196 , 235 and

Artificial Intelligence: A Modern Approach

by Stuart Russell and Peter Norvig  · 14 Jul 2019  · 2,466pp  · 668,761 words

. Shankar, N. (1986). Proof‑Checking Metamathematics. Ph.D. thesis, Computer Science Department, University of Texas at Austin. Shannon, C. E. and Weaver, W. (1949). The Mathematical Theory of Communication. University of Illinois Press. Shannon, C. E. (1950). Programming a computer for playing chess. Philosophical Magazine, 41, 256–275. Shapley, S. (1953b). Stochastic games. PNAS

The Singularity Is Near: When Humans Transcend Biology

by Ray Kurzweil  · 14 Jul 2005  · 761pp  · 231,902 words

it will require. The mathematician Claude Shannon, often called the father of information theory, defined the basic theory of data compression in his paper "A Mathematical Theory of Communication," The Bell System Technical Journal 27 (July–October 1948): 379–423, 623–56. Data compression is possible because of factors such as redundancy (repetition) and

The Information: A History, a Theory, a Flood

by James Gleick  · 1 Mar 2011  · 855pp  · 178,507 words

The Bell System Technical Journal in July and October. No one bothered with a press release. It carried a title both simple and grand—“A Mathematical Theory of Communication”—and the message was hard to summarize. But it was a fulcrum around which the world began to turn. Like the transistor, this development also

…

HEADQUARTERS OF BELL LABORATORIES, WITH TRAINS OF THE HIGH LINE RUNNING THROUGH Few libraries carried The Bell System Technical Journal, so researchers heard about “A Mathematical Theory of Communication” the traditional way, by word of mouth, and obtained copies the traditional way, by writing directly to the author for an offprint. Many scientists used

…

year the two pieces—Weaver’s essay and Shannon’s monograph—were published together as a book, now titled with a grander first word The Mathematical Theory of Communication. To John Robinson Pierce, the Bell Labs engineer who had been watching the simultaneous gestation of the transistor and Shannon’s paper, it was the

…

amplifier,” Shockley told Shannon.♦ At that point it still needed a name. One day in the summer of 1949, before the book version of The Mathematical Theory of Communication appeared, Shannon took a pencil and a piece of notebook paper, drew a line from top to bottom, and wrote the powers of ten from

…

little tension between these men. It could be felt weighing down the long footnote that anchored the opening page of Weaver’s portion of The Mathematical Theory of Communication: Dr. Shannon has himself emphasized that communication theory owes a great debt to Professor Norbert Wiener for most of its basic philosophy. Professor Wiener, on

…

represent the information source as a statistical process, generating messages with varying probabilities. He showed them the sample text strings he had used in The Mathematical Theory of Communication—which few of them had read—and described his “prediction experiment,” in which the subject guesses text letter by letter. He told them that English

…

. When Chaitin came upon Turing’s proof of uncomputability, he thought this must be the key. He also found Shannon and Weaver’s book, The Mathematical Theory of Communication, and was struck by its upside-down seeming reformulation of entropy: an entropy of bits, measuring information on the one hand and disorder on the

…

turbulence and random processes. He was a Hero of Socialist Labor and seven times received the Order of Lenin. He first saw Claude Shannon’s Mathematical Theory of Communication rendered into Russian in 1953, purged of its most interesting features by a translator working in Stalin’s heavy shadow. The title became Statistical Theory

…

. The birth of information theory came with its ruthless sacrifice of meaning—the very quality that gives information its value and its purpose. Introducing The Mathematical Theory of Communication, Shannon had to be blunt. He simply declared meaning to be “irrelevant to the engineering problem.” Forget human psychology; abandon subjectivity. He knew there would

…

of Information Theory,” IEEE Transactions on Information Theory 19, no. 1 (1973): 4. ♦ “THE FUNDAMENTAL PROBLEM OF COMMUNICATION”: Claude Elwood Shannon and Warren Weaver, The Mathematical Theory of Communication (Urbana: University of Illinois Press, 1949), 31. ♦ “THIS IS ALREADY DONE TO A LIMITED EXTENT”: Ibid., 11. ♦ LANDMARK 1943 PAPER: “Stochastic Problems in Physics and

…

: The Story of Codes and Ciphers (Garden City, N.Y.: Blue Ribbon, 1939). ♦ “HOW MUCH ‘CHOICE’ IS INVOLVED”: Claude Elwood Shannon and Warren Weaver, The Mathematical Theory of Communication, 18. ♦ “BINARY DIGITS, OR MORE BRIEFLY, BITS”: “A word suggested by J. W. Tukey,” he added. John Tukey, the statistician, had been a roommate of

…

, Juan G. Information and Its Role in Nature. Berlin: Springer, 2005. Rogers, Everett M. “Claude Shannon’s Cryptography Research during World War II and the Mathematical Theory of Communication.” In Proceedings, IEEE 28th International Carnaham Conference on Security Technology, October 1994: 1–5. Romans, James. ABC of the Telephone. New York: Audel & Co., 1901

…

. Sloane and Aaron D. Wyner. Murray Hill, N.J.: Mathematical Sciences Research Center, AT&T Bell Laboratories, 1993. Shannon, Claude Elwood, and Warren Weaver. The Mathematical Theory of Communication. Urbana: University of Illinois Press, 1949. Shenk, David. Data Smog: Surviving the Information Glut. New York: HarperCollins, 1997. Shieber, Stuart M., ed. The Turing Test

…

, 12.1-3.1 noise source in, 7.1, 7.2 origins of, prl.1, prl.2, prl.3, prl.4, 7.1; see also Mathematical Theory of Communication, The (Shannon, Weaver) physics and, prl.1, prl.2, prl.3 place of meaning in, 7.1, 7.2, 8.1, 8.2, 8.3

…

, prl.3, 6.1, 6.2, 6.3, 6.4, 8.1, 8.2, 12.1, 14.1, epl.1 Mathematical Analysis of Logic (Boole) Mathematical Theory of Communication, The (Shannon, Weaver), prl.1, 7.1, 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10

My Life as a Quant: Reflections on Physics and Finance

by Emanuel Derman  · 1 Jan 2004  · 313pp  · 101,403 words

of the subsequent advances in communications. Bardeen, Brattain, and Shockley had invented the transistor there in 1947, and Claude Shannon published his landmark paper "A Mathematical Theory of Communication" in the Bell System Technical Journal in 1948. There were fundamental discoveries made, too-Penzias and Wilson won the Nobel Prize for discovering the cosmic

Crypto: How the Code Rebels Beat the Government Saving Privacy in the Digital Age

by Steven Levy  · 15 Jan 2002  · 468pp  · 137,055 words

decade the two key papers in Shannon’s wartime work found their way into the public domain. In 1948, Shannon’s seminal article on information, “Mathematical Theory of Communication,” ran in the Bell System Technical Journal, and subtly set the stage for the digital epoch. A year later, “Communication Theory of Secrecy Systems” appeared

Einstein's Fridge: How the Difference Between Hot and Cold Explains the Universe

by Paul Sen  · 16 Mar 2021  · 444pp  · 111,837 words

—and come up with one of the greatest scientific insights of the modern age. In 1948, he revealed his thoughts in a paper entitled “A Mathematical Theory of Communication,” published in the Bell Labs technical journal. Less than thirty pages long, Shannon’s paper enabled humans, for the first time, to measure information in

…

to him several times”: From “Shannon: An Interview by Price.” “They never told me”: As quoted in Mind at Play by Soni and Goodman. “A Mathematical Theory of Communication”: From Bell System Technical Journal 27 (1948). “reproducing at one point”: From the above paper. Shannon pointed the similarity out to John von Neumann: This

…

, 224, 225 Maxwell’s description of electromagnetism using, 134 for phyllotaxis, in Turing’s research, 212 Turing’s pattern formation in morphogenesis and, 208–9 “Mathematical Theory of Communication, A” (Shannon), 174–76, 179 mathematics British teaching of, 5 computer calculations in, 171 Gibbs’s education in, 95, 105 Maxwell’s education in, 83

From eternity to here: the quest for the ultimate theory of time

by Sean M. Carroll  · 15 Jan 2010  · 634pp  · 185,116 words

, Order Parameters, and Complexity. Oxford: Oxford University Press, 2006. Shalizi, C. R. Notebooks (2009). http://www.cscs.umich.edu/~crshalizi/notebooks/. Shannon, C. E. “A Mathematical Theory of Communication.” Bell System Technical Journal 27 (1948): 379- 423 and 623-56. Singh, S. Big Bang: The Origin of the Universe. New York: Fourth Estate, 2004

The Alignment Problem: Machine Learning and Human Values

by Brian Christian  · 5 Oct 2020  · 625pp  · 167,349 words

Cognitive Gadgets: The Cultural Evolution of Thinking

by Cecilia Heyes  · 15 Apr 2018

Complexity: A Guided Tour

by Melanie Mitchell  · 31 Mar 2009  · 524pp  · 120,182 words

Shape: The Hidden Geometry of Information, Biology, Strategy, Democracy, and Everything Else

by Jordan Ellenberg  · 14 May 2021  · 665pp  · 159,350 words

The Chip: How Two Americans Invented the Microchip and Launched a Revolution

by T. R. Reid  · 18 Dec 2007  · 293pp  · 91,110 words

A Mind at Play: How Claude Shannon Invented the Information Age

by Jimmy Soni and Rob Goodman  · 17 Jul 2017  · 415pp  · 114,840 words

The Scientist as Rebel

by Freeman Dyson  · 1 Jan 2006  · 332pp  · 109,213 words

Applied Cryptography: Protocols, Algorithms, and Source Code in C

by Bruce Schneier  · 10 Nov 1993

Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy

by George Gilder  · 16 Jul 2018  · 332pp  · 93,672 words

The Computer Boys Take Over: Computers, Programmers, and the Politics of Technical Expertise

by Nathan L. Ensmenger  · 31 Jul 2010  · 429pp  · 114,726 words

The Codebreakers: The Comprehensive History of Secret Communication From Ancient Times to the Internet

by David Kahn  · 1 Feb 1963  · 1,799pp  · 532,462 words

The Idea Factory: Bell Labs and the Great Age of American Innovation

by Jon Gertner  · 15 Mar 2012  · 550pp  · 154,725 words

Turing's Cathedral

by George Dyson  · 6 Mar 2012

Chaos: Making a New Science

by James Gleick  · 18 Oct 2011  · 396pp  · 112,748 words

On Language: Chomsky's Classic Works Language and Responsibility and Reflections on Language in One Volume

by Noam Chomsky and Mitsou Ronat  · 26 Jul 2011

Noam Chomsky: A Life of Dissent

by Robert F. Barsky  · 2 Feb 1997

Whiplash: How to Survive Our Faster Future

by Joi Ito and Jeff Howe  · 6 Dec 2016  · 254pp  · 76,064 words

How Not to Network a Nation: The Uneasy History of the Soviet Internet (Information Policy)

by Benjamin Peters  · 2 Jun 2016  · 518pp  · 107,836 words

Advances in Financial Machine Learning

by Marcos Lopez de Prado  · 2 Feb 2018  · 571pp  · 105,054 words

Deep Utopia: Life and Meaning in a Solved World

by Nick Bostrom  · 26 Mar 2024  · 547pp  · 173,909 words

Nine Algorithms That Changed the Future: The Ingenious Ideas That Drive Today's Computers

by John MacCormick and Chris Bishop  · 27 Dec 2011  · 250pp  · 73,574 words

How to Create a Mind: The Secret of Human Thought Revealed

by Ray Kurzweil  · 13 Nov 2012  · 372pp  · 101,174 words

The Logician and the Engineer: How George Boole and Claude Shannon Created the Information Age

by Paul J. Nahin  · 27 Oct 2012  · 229pp  · 67,599 words

The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal

by M. Mitchell Waldrop  · 14 Apr 2001

The Fractalist

by Benoit Mandelbrot  · 30 Oct 2012

From Counterculture to Cyberculture: Stewart Brand, the Whole Earth Network, and the Rise of Digital Utopianism

by Fred Turner  · 31 Aug 2006  · 339pp  · 57,031 words

The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy

by Sharon Bertsch McGrayne  · 16 May 2011  · 561pp  · 120,899 words

The Physics of Wall Street: A Brief History of Predicting the Unpredictable

by James Owen Weatherall  · 2 Jan 2013  · 338pp  · 106,936 words

The Organized Mind: Thinking Straight in the Age of Information Overload

by Daniel J. Levitin  · 18 Aug 2014  · 685pp  · 203,949 words

Understanding Sponsored Search: Core Elements of Keyword Advertising

by Jim Jansen  · 25 Jul 2011  · 298pp  · 43,745 words

Life's Greatest Secret: The Race to Crack the Genetic Code

by Matthew Cobb  · 6 Jul 2015  · 608pp  · 150,324 words

Fortune's Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street

by William Poundstone  · 18 Sep 2006  · 389pp  · 109,207 words

The Most Human Human: What Talking With Computers Teaches Us About What It Means to Be Alive

by Brian Christian  · 1 Mar 2011  · 370pp  · 94,968 words

Language and Mind

by Noam Chomsky  · 1 Jan 1968

Emergence

by Steven Johnson  · 329pp  · 88,954 words

Physics in Mind: A Quantum View of the Brain

by Werner Loewenstein  · 29 Jan 2013  · 362pp  · 97,862 words

Darwin Among the Machines

by George Dyson  · 28 Mar 2012  · 463pp  · 118,936 words

The Cultural Logic of Computation

by David Golumbia  · 31 Mar 2009  · 268pp  · 109,447 words

From Bacteria to Bach and Back: The Evolution of Minds

by Daniel C. Dennett  · 7 Feb 2017  · 573pp  · 157,767 words

Possible Minds: Twenty-Five Ways of Looking at AI

by John Brockman  · 19 Feb 2019  · 339pp  · 94,769 words

The Deep Learning Revolution (The MIT Press)

by Terrence J. Sejnowski  · 27 Sep 2018

Literary Theory for Robots: How Computers Learned to Write

by Dennis Yi Tenen  · 6 Feb 2024  · 169pp  · 41,887 words

The Demon in the Machine: How Hidden Webs of Information Are Finally Solving the Mystery of Life

by Paul Davies  · 31 Jan 2019  · 253pp  · 83,473 words

Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking

by Foster Provost and Tom Fawcett  · 30 Jun 2013  · 660pp  · 141,595 words

Data Mining: Concepts and Techniques: Concepts and Techniques

by Jiawei Han, Micheline Kamber and Jian Pei  · 21 Jun 2011

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann  · 17 Apr 2017

Why Information Grows: The Evolution of Order, From Atoms to Economies

by Cesar Hidalgo  · 1 Jun 2015  · 242pp  · 68,019 words

Code: The Hidden Language of Computer Hardware and Software

by Charles Petzold  · 28 Sep 1999  · 566pp  · 122,184 words

Information: A Very Short Introduction

by Luciano Floridi  · 25 Feb 2010  · 137pp  · 36,231 words

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity

by Amy Webb  · 5 Mar 2019  · 340pp  · 97,723 words

Making Sense of Chaos: A Better Economics for a Better World

by J. Doyne Farmer  · 24 Apr 2024  · 406pp  · 114,438 words

Lifespan: Why We Age—and Why We Don't Have To

by David A. Sinclair and Matthew D. Laplante  · 9 Sep 2019

A Devil's Chaplain: Selected Writings

by Richard Dawkins  · 1 Jan 2004  · 460pp  · 107,712 words

Turing's Vision: The Birth of Computer Science

by Chris Bernhardt  · 12 May 2016  · 210pp  · 62,771 words

How to Predict the Unpredictable

by William Poundstone  · 267pp  · 71,941 words

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann  · 16 Mar 2017  · 1,237pp  · 227,370 words

The Woman Who Smashed Codes: A True Story of Love, Spies, and the Unlikely Heroine Who Outwitted America's Enemies

by Jason Fagone  · 25 Sep 2017  · 592pp  · 152,445 words

Social Life of Information

by John Seely Brown and Paul Duguid  · 2 Feb 2000  · 791pp  · 85,159 words

The Atlantic and Its Enemies: A History of the Cold War

by Norman Stone  · 15 Feb 2010  · 851pp  · 247,711 words

The Chomsky Reader

by Noam Chomsky  · 11 Sep 1987

The Internet of Us: Knowing More and Understanding Less in the Age of Big Data

by Michael P. Lynch  · 21 Mar 2016  · 230pp  · 61,702 words

Superbloom: How Technologies of Connection Tear Us Apart

by Nicholas Carr  · 28 Jan 2025  · 231pp  · 85,135 words

Novacene: The Coming Age of Hyperintelligence

by James Lovelock  · 27 Aug 2019  · 94pp  · 33,179 words

A World Without Email: Reimagining Work in an Age of Communication Overload

by Cal Newport  · 2 Mar 2021  · 350pp  · 90,898 words

Kitten Clone: Inside Alcatel-Lucent

by Douglas Coupland  · 29 Sep 2014  · 124pp  · 36,360 words

Paper Machines: About Cards & Catalogs, 1548-1929

by Markus Krajewski and Peter Krapp  · 18 Aug 2011  · 222pp  · 74,587 words

Simple Rules: How to Thrive in a Complex World

by Donald Sull and Kathleen M. Eisenhardt  · 20 Apr 2015  · 294pp  · 82,438 words

Energy and Civilization: A History

by Vaclav Smil  · 11 May 2017

The Price of Tomorrow: Why Deflation Is the Key to an Abundant Future

by Jeff Booth  · 14 Jan 2020  · 180pp  · 55,805 words

Commodore: A Company on the Edge

by Brian Bagnall  · 13 Sep 2005  · 781pp  · 226,928 words

The End of Accounting and the Path Forward for Investors and Managers (Wiley Finance)

by Feng Gu  · 26 Jun 2016

The Future of Ideas: The Fate of the Commons in a Connected World

by Lawrence Lessig  · 14 Jul 2001  · 494pp  · 142,285 words