The Signal and the Noise by Nate Silver Concepts

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't

by Nate Silver · 31 Aug 2012 · 829pp · 186,976 words

Ltd, Registered Offices: 80 Strand, London WC2R 0RL, England First published in 2012 by The Penguin Press, a member of Penguin Group (USA) Inc. Copyright © Nate Silver, 2012 All rights reserved Illustration credits Figure 4-2: Courtesy of Dr. Tim Parker, University of Oxford Figure 7-1: From “1918 Influenza: The Mother

…

: Courtesy of Dr. J. Scott Armstrong, The Wharton School, University of Pennsylvania LIBRARY OF CONGRESS CATALOGING IN PUBLICATION DATA Silver, Nate. The signal and the noise : why most predictions fail but some don’t / Nate Silver. p. cm. Includes bibliographical references and index. ISBN 978-1-101-59595-4 1. Forecasting. 2. Forecasting—Methodology. 3. Forecasting—History

…

information when we really want knowledge. The signal is the truth. The noise is what distracts us from the truth. This is a book about the signal and the noise. 1 A CATASTROPHIC FAILURE OF PREDICTION It was October 23, 2008. The stock market was in free fall, having plummeted almost 30 percent over the

…

those people.2 I lived the poker dream for a while, and then it died. I learned that poker sits at the muddy confluence of the signal and the noise. My years in the game taught me a great deal about the role that chance plays in our lives and the delusions it can produce

…

not rise from 2001 through 2011, they were still much warmer than in any prior decade. Nevertheless, this book encourages readers to think carefully about the signal and the noise and to seek out forecasts that couch their predictions in percentage or probabilistic terms. They are a more honest representation of the limits of our

…

will start by buying the first beer for anybody on this list, and the first three for anybody who should have been, but isn’t. —Nate Silver Brooklyn, NY NOTES INTRODUCTION 1. The Industrial Revolution is variously described as starting anywhere from the mid-eighteenth to the early nineteenth centuries. I choose

…

: The Data Deluge Makes the Scientific Method Obsolete,” Wired magazine, June 23, 2008. http://www.wired.com/science/discoveries/magazine/16-07/pb_theory. 38. Nate Silver, “Models Based on ‘Fundamentals’ Have Failed at Predicting Presidential Elections,” FiveThirtyEight, New York Times, March 26, 2012. http://fivethirtyeight.blogs.nytimes.com/2012/03/26

…

passed, later statistical analyses suggested that members of Congress who had voted for the bailout were more likely to lose their seats. See for example: Nate Silver, “Health Care and Bailout Votes May Have Hurt Democrats,” FiveThirtyEight, New York Times, November 16, 2011. http://fivethirtyeight.blogs.nytimes.com/2010/11/16/health

…

/pricehistory/PriceHistory_GetData.cfm. 3. The McLaughlin Group transcript, Federal News Service; taped November 7, 2008. http://www.mclaughlin.com/transcript.htm?id=688. 4. Nate Silver, “Debunking the Bradley Effect,” Newsweek, October 20, 2008. http://www.thedailybeast.com/newsweek/2008/10/20/debunking-the-bradley-effect.html. 5. Academic studies of

…

of their votes. 25. “Election Results: House Big Board,” New York Times, November 2, 2010. http://elections.nytimes.com/2010/results/house/big-board. 26. Nate Silver, “A Warning on the Accuracy of Primary Polls,” FiveThirtyEight, New York Times, March 1, 2012. http://fivethirtyeight.blogs.nytimes.com/2012/03/01/a-warning

…

-on-the-accuracy-of-primary-polls/. 27. Nate Silver, “Bill Buckner Strikes Again,” FiveThirtyEight, New York Times; September 29, 2011. http://fivethirtyeight.blogs.nytimes.com/2011/09/29/bill-buckner-strikes-again/. 28. Otherwise

…

, you should have assigned the congressman a 100 percent chance of victory instead. 29. Matthew Dickinson, “Nate Silver Is Not a Political Scientist,” in Presidential Power: A Nonpartisan Analysis of Presidential Power, Blogs Dot Middlebury, November 1, 2010. http://blogs.middlebury.edu/presidentialpower

…

/2010/11/01/nate-silver-is-not-a-political-scientist/. 30. Sam Wang, “A Weakness in FiveThirtyEight.com,” Princeton Election Consortium, August 8, 2008. http://election.princeton.edu/2008/08

…

Polls So Predictable?” British Journal of Political Science 23, no. 4 (October 1993). http://www.rochester.edu/College/faculty/mperess/ada2007/Gelman_King.pdf. 35. Nate Silver, “Models Based on ‘Fundamentals’ Have Failed at Predicting Presidential Elections,” FiveThirtyEight, New York Times, March 26, 2012. 36. Between 1998 and 2008, the average poll

…

Harper, Online Etymology Dictionary. http://www.etymonline.com/index.php?term=objective. CHAPTER 3: ALL I CARE ABOUT IS W’S AND L’S 1. Nate Silver in Jonah Keri, et al., Baseball Between the Numbers: Why Everything You Know About the Game Is Wrong (New York: Basic Books, 2006). 2. Danny

…

Knobler, “The Opposite of a ‘Tools Guy,’ Pedroia’s Simply a Winner,” CBSSports.com, November 18, 2008. http://www.cbssports.com/mlb/story/11116048. 3. Nate Silver, “Lies, Damned Lies: PECOTA Takes on Prospects, Wrap-up,” BaseballProspectus.com, March 8, 2006. http://www.baseballprospectus.com/article.php?articleid=4841. 4. Law is

…

. Alan Schwarz, “The Great Debate,” Baseball America, January. 7, 2005. http://www.baseballamerica.com/today/features/050107debate.html. 20. Per interview with Billy Beane. 21. Nate Silver, “What Tim Geithner Can Learn from Baseball,” Esquire, March 11, 2009. http://www.esquire.com/features/data/mlb-player-salaries-0409. 22. As a result

…

methodology. The methods I describe herein apply to the 2003–2009 version of PECOTA specifically. 23. Nate Silver, “PECOTA Takes on the Field,” Baseball Prospectus, January 16, 2004. http://www.baseballprospectus.com/article.php?articleid=2515. 24. Nate Silver, “Lies, Damned Lies: Projection Reflection,” Baseball Prospectus, October 11, 2006. http://www.baseballprospectus.com/article

…

. 48. 2008 New Hampshire Democratic Primary polls via RealClearPolitics.com. http://www.realclearpolitics.com/epolls/2008/president/nh/new_hampshire_democratic_primary-194.html. 49. Nate Silver, “Rasmussen Polls Were Biased and Inaccurate; Quinnipiac, SurveyUSA Performed Strongly,” FiveThirtyEight, New York Times, November 4, 2010. http://fivethirtyeight.blogs.nytimes.com/2010/11/04

…

on the river, he would essentially be doing so as a bluff since he wouldn’t expect us to call with a weaker hand. 6. Nate Silver, “Sanity Check: 88 Hand” twoplustwo.com; May 14, 2012. http://forumserver.twoplustwo.com/56/medium-stakes-pl-nl/sanity-check-88-hand-1199549/. 7. G4mblers

…

-uigea-487/. 26. Branon Adams, “The Poker Economy,” Bluff Magazine, November, 2006. http://www.bluffmagazine.com/magazine/The-Poker-Economy-Brandon-Adams-584.htm. 27. Nate Silver, “After ‘Black Friday,’ American Poker Faces Cloudy Future,” FiveThirtyEight, New York Times, April 20, 2011. http://fivethirtyeight.blogs.nytimes.com/2011/04/20/after-black

…

. 18. “Super Tuesday 2012 on Intrade;” Intrade.com, March 8, 2012. http://www.intrade.com/v4/reports/historic/2012-03-07-super-tuesday-2012/. 19. Nate Silver, “Intrade Betting Is Suspicious,” FiveThirtyEight, September 24, 2008. http://www.fivethirtyeight.com/2008/09/intrade-betting-is-suspcious.html. 20

…

. Nate Silver, “Evidence of Irrationality at Intrade,” “Live Coverage: Alabama and Mississippi Primaries;” FiveThirtyEight, New York Times, March 13, 2012. http://fivethirtyeight.blogs.nytimes.com/2012/03/

…

. 30. “FAQ: Copenhagen Conference 2009;” CBCNews.ca, December 8, 2009. http://www.cbc.ca/news/world/story/2009/12/01/f-copenhagen-summit.html. 31. Nate Silver, “Despite Protests, Some Reason for Optimism in Copenhagen,” FiveThirtyEight.com, December 9, 2009. http://www.fivethirtyeight.com/2009/12/despite-protests-some-reasons-for.html

…

Security Theater,” Schneier on Security, November 13, 2009. http://www.schneier.com/blog/archives/2009/11/beyond_security.html. 76. Ibid., Kindle location 1035. 77. Nate Silver, “Crunching the Risk Numbers,” Wall Street Journal, January 8, 2010. http://Online.wsj.com/article/SB10001424052748703481004574646963713065116.html. 78. Russian Authorities: Terrorist Bombing at Moscow Airport

On the Edge: The Art of Risking Everything

by Nate Silver · 12 Aug 2024 · 848pp · 227,015 words

Also by Nate Silver The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t PENGUIN PRESS An imprint of Penguin Random House LLC penguinrandomhouse.com Copyright © 2024 by Nate Silver Penguin Random House supports copyright. Copyright fuels creativity, encourages diverse voices, promotes free speech, and creates a vibrant culture. Thank you for buying an authorized

…

page: courtesy of xkcd library of congress cataloging-in-publication data Names: Silver, Nate, 1978– author. Title: On the edge: the art of risking everything / Nate Silver. Description: New York: Penguin Press, 2024. | Includes bibliographical references and index. Identifiers: LCCN 2024011711 (print) | LCCN 2024011712 (ebook) | ISBN 9781594204128 (hardcover) | ISBN 9780593491638 (ebook) Subjects

…

percent). Trump won, of course, by sweeping several Rust Belt swing states. The reaction of many people in the political world to this forecast was: “Nate Silver is a fucking idiot.” But from my standpoint—and from the standpoint of people in the River, the landscape of skilled gamblers and like-minded

…

people who don’t understand EV. As a first step, I’d like to get you thinking probabilistically. The vital point of my first book, The Signal and the Noise, is that probabilistic forecasts are a sign of humility, not hubris. The world is a complicated place. Small perturbations can have outsized effects, from the

…

for this. It is not as easy as it sounds, and there are many ways it can go wrong (essentially, this is the subject of The Signal and the Noise). But nearly all professions in the River, including the more philosophical ones you’ll find Upriver, involve some attempt at model building. The final term

…

and bridge. The IBM supercomputer Deep Blue had famously beat the chess world champion Garry Kasparov in 1997—this story is recounted in detail in The Signal and the Noise—and by the early 2000s, chess engines that played at grandmaster levels were widely available on home computers. Poker had lagged behind. Sure, there was

…

guy who only plays aces and kings—but I’m gradually aging into that demographic. And if my opponents know me as “statistician and author Nate Silver,” but don’t know my poker-playing background, they’ll usually assume my play is tight and conservative. People tend to read “statistician” as “calculating

…

. That reputation may be changing, though, because one of the poker hands I’m best known for is a bluff. Entitled “America’s Top Statistician Nate Silver Runs Epic Bluff in $10,000 Poker Tournament” on the PokerGO YouTube channel, the clip of that bluff has gotten tens of thousands of views

…

Modeler). So let’s complete the set. Bob Voulgaris: The Edge Finder Haralabos “Bob” Voulgaris has come a long way since I profiled him for The Signal and the Noise. He’d already worked his way up from being an airport skycap who made a pair of daring bets on the Los Angeles Lakers to

…

Gambler”: You’ve got to know when to Hold’em, know when to fold ’em Know when to walk away and know when to raise —Nate Silver, On the Edge As Doyle Brunson figured out fifty years ago, no-limit Texas Hold’em is a game of aggression. Modern poker solvers confirm

…

yacht in the Mediterranean; and now owning a Spanish soccer team. I call this personality type a fox, and that was also featured prominently in The Signal and the Noise. Fox is one of the two archetypes articulated in the Greek poet Archilochus’s saying: “The fox knows many things, but the hedgehog knows one

…

types. This corresponds to what I wrote before about needing both reasonable and unreasonable people, but I want to be more precise. If you read The Signal and the Noise, you may remember our furry friends, foxes and hedgehogs. The terminology comes from the Greek poet Archilochus—“The fox knows many things, but the hedgehog

…

cluster of personality traits that Tetlock regarded as foxlike—knowing many little things—were comparatively more accurate. Those clever little foxes were the heroes of The Signal and the Noise. But do foxes make for good founders? I can think of exceptions to the rule (the polymathic Collison is quite fox-like, for instance). But

…

’s demon—the conjecture that if we knew the location and momentum of every particle in the universe, we could perfectly predict the future—in The Signal and the Noise. *6 “I don’t like the word ‘contrarian’—you just put a minus sign in front of some sort of wisdom-of-the-crowds-type

…

the Village, where ostracization (or if you prefer, “cancellation”) is considered the ultimate punishment. *12 In case it seems familiar, this table is reproduced from The Signal and the Noise with some revisions and additions to make it more focused on attitudes toward risk-taking. *13 Moritz did found a tech-focused newsletter, a business

…

act of viewing the world probabilistically and regularly updating your views as you encounter new evidence—is sort of the lodestar of my first book, The Signal and the Noise.) In fact, even if I had never applied to join Team Rationalist, Alexander—whose soft features, dry wit, and male pattern baldness reminded me uncannily

…

hard to do, in part because there are two sorts of errors one can make. This is slightly technical—there’s a longer discussion in The Signal and the Noise if you want to go deeper—but one problem is called “overfitting.” Basically, that means trying to accommodate every nook and cranny in the dataset

…

being epistemically rational. (Although in fact, he has low cholesterol and is otherwise in good health, or so his wife claims.) As I argue in The Signal and the Noise, making testable predictions is one of the only ways to know whether you’re epistemically rational. So in theory, prediction markets play an important role

…

pseudonymous AI personality who was later outed[*6] by Forbes magazine. But roon told me that he first created his Twitter account to be a Nate Silver reply guy, “with the express intention of trolling your replies” about my election forecasts. (The internet works in mysterious ways.) A person like roon, with

…

do any human feature engineering, the features are already there,” said Ryder. This surprised a lot of machine learning researchers—and it surprised me. In The Signal and the Noise, I expressed skepticism toward “big data’’ approaches because my experience was that you needed to give models a helping hand, imparting some domain knowledge and

…

read on my intentions. There’s one last comparison between language models and poker—or really between language and poker. The critique I made in The Signal and the Noise was that, sure, AIs might work well when they’re playing games like chess that have well-defined rules, but their worth had yet to

…

Northeastern winter and the COVID-19 pandemic, I decided that I wanted to write another book. It had been eight and a half years since The Signal and the Noise had been published, and I’d aged into middle-adulthood. With Disney facing economic headwinds, and my passion for being an “election nerd” having waned

…

worth your while. But I’m still going to claw out every inch of EV that I can if we meet at the poker table. —Nate Silver, Las Vegas, Nevada, April 10, 2024 Glossary: How to Speak Riverian This is a reasonably complete list of technical terms used in On the Edge

…

. GO TO NOTE REFERENCE IN TEXT estimate of Trump’s chances: Matthew Yglesias, “Why I Think Nate Silver’s Model Underrates Clinton’s Odds,” Vox, November 7, 2016, vox.com/policy-and-politics/2016/11/7/13550068/nate-silver-forecast-wrong. GO TO NOTE REFERENCE IN TEXT less than 1 percent: Josh Katz, “Who

…

: What We Know,” Time, December 14, 2022, time.com/6241262/sam-bankman-fried-political-donations. GO TO NOTE REFERENCE IN TEXT the Village’s claims: Nate Silver, “Twitter, Elon and the Indigo Blob,” Silver Bulletin (blog), October 1, 2023, natesilver.net/p/twitter-elon-and-the-indigo-blob. GO TO NOTE REFERENCE

…

TEXT aren’t any insurmountable barriers: See for instance the responses in this Twitter thread, which drew replies from various people with expertise in robotics. Nate Silver (@NateSilver538), “Weird question for my book. Given current tech, could a robot physically play in a poker game? It would need to e.g.:—Handle

…

1, 2020, quoteinvestigator.com/2020/03/01/underestimate. GO TO NOTE REFERENCE IN TEXT poker hands I’m best known for: “America’s Top Statistician Nate Silver Runs Epic Bluff in $10,000 Poker Tournament,” PokerGO, 2022, youtube.com/watch?v=9cVrlVzoh48. GO TO NOTE REFERENCE IN TEXT 1 million in chips

…

National?,” Analytics Blog (blog), Data Golf, April 8, 2019, datagolf.ca/does-experience-matter-at-augusta. GO TO NOTE REFERENCE IN TEXT the NBA playoffs: Nate Silver, “Why the Warriors and Cavs Are Still Big Favorites,” FiveThirtyEight, October 13, 2017, fivethirtyeight.com/features/why-the-warriors-and-cavs-are-still-big-favorites

…

REFERENCE IN TEXT goalie Ken Dryden: Coates, The Hour Between Dog and Wolf, 78. GO TO NOTE REFERENCE IN TEXT national news coverage: Ryan Glasspiegel, “Nate Silver Made Brutal All-in Call at WSOP: ‘F—King Poker,’ ” New York Post, July 13, 2023, nypost.com/2023/07/13

…

/nate-silver-made-brutal-all-in-call-at-wsop-f-king-poker. GO TO NOTE REFERENCE IN TEXT playing to survive: This was not the exact phrasing,

…

CCAdj),” FRED, Federal Reserve Bank of St. Louis, January 1, 1946), fred.stlouisfed.org/series/CP. GO TO NOTE REFERENCE IN TEXT fast food delivery: Nate Silver, “The McDonald’s Theory of Why Everyone Thinks the Economy Sucks,” Silver Bulletin (blog), October 1, 2023, natesilver.net/p/the-mcdonalds-theory-of-why

…

the Epic First Mission to Pluto (New York: Picador, 2018). GO TO NOTE REFERENCE IN TEXT I reintroduced you: Voulgaris was also featured prominently in The Signal and The Noise. GO TO NOTE REFERENCE IN TEXT on a yacht: Haralabos Voulgaris (@haralabob), “Jellyfish no longer a problem,” Twitter, June 21, 2022, twitter.com/haralabob/status

…

TO NOTE REFERENCE IN TEXT sworn to secrecy: Chamath Palihapitiya et al., “#AIS: FiveThirtyEight’s Nate Silver on How Gamblers Think,” All-In with Chamath, Jason, Sacks & Friedberg, 2022, podcasts.apple.com/ie/podcast/ais-fivethirtyeights-nate-silver-on-how-gamblers-think/id1502871393?i=1000564483582. GO TO NOTE REFERENCE IN TEXT aggressive compensation deal

…

3:1 in Air War,” Wesleyan Media Project, November 3, 2016, mediaproject.wesleyan.edu/nov-2016. GO TO NOTE REFERENCE IN TEXT Clinton email scandal: Nate Silver, “The Real Story of 2016,” FiveThirtyEight, January 19, 2017, fivethirtyeight.com/features/the-real-story-of-2016. GO TO NOTE REFERENCE IN TEXT amid sometimes

…

, 2022, sec. News, theguardian.com/news/2022/jul/10/uber-files-leak-reveals-global-lobbying-campaign. GO TO NOTE REFERENCE IN TEXT a political cudgel: Nate Silver, “Twitter, Elon and the Indigo Blob,” Silver Bulletin (blog), October 1, 2023, natesilver.net/p/twitter-elon-and-the-indigo-blob. GO TO NOTE REFERENCE

…

York Times, April 8, 2023, nytimes.com/2022/04/08/business/hiring-without-college-degree.html. GO TO NOTE REFERENCE IN TEXT congressional hearing featuring: Nate Silver, “Why Liberalism and Leftism Are Increasingly at Odds,” Silver Bulletin (blog), December 12, 2023, natesilver.net/p/why-liberalism-and-leftism-are-increasingly. GO TO

…

, June 23, 2023, cointelegraph.com/news/sequoia-partner-says-investing-ftx-was-right-move. GO TO NOTE REFERENCE IN TEXT told me that: Email to Nate Silver, January 8, 2024. GO TO NOTE REFERENCE IN TEXT by SBF nemesis: David Marsanic, “CZ and SBF Twitter Fight Reveals How They Became Rivals,” DailyCoin

…

an xkcd cartoon: Randall Munroe, “Duty Calls,” xkcd, xkcd.com/386. GO TO NOTE REFERENCE IN TEXT most popular stories: As of December 21, 2023; Nate Silver, “Fine, I’ll Run a Regression Analysis. But It Won’t Make You Happy,” Silver Bulletin (blog), October 1, 2023, natesilver.net/p/fine-ill

…

. Technology, nytimes.com/2023/03/31/technology/sam-altman-open-ai-chatgpt.html. GO TO NOTE REFERENCE IN TEXT that Altman knew: Per email to Nate Silver, January 19, 2024. GO TO NOTE REFERENCE IN TEXT taking a shot: Sam Altman (@sama), “i’ve been stopping myself from sending my EA tweetstorm

…

-model-works-and-how-it-fueled-the-tumult-around-ceo-sam-altmans-short-lived-ouster-218340. GO TO NOTE REFERENCE IN TEXT like fast food: Nate Silver, “The McDonald’s Theory of Why Everyone Thinks the Economy Sucks,” Silver Bulletin (blog), October 1, 2023, natesilver.net/p/the-mcdonalds-theory-of-why

…

S. Wood, The Radicalism of the American Revolution, Kindle ed. (New York: Vintage Books, 1993), 5. GO TO NOTE REFERENCE IN TEXT rule of law: Nate Silver, “Why Liberalism and Leftism Are Increasingly at Odds,” Silver Bulletin (blog), December 12, 2023, natesilver.net/p/why-liberalism-and-leftism-are-increasingly. GO TO

…

-highs.aspx. GO TO NOTE REFERENCE IN TEXT large language models: ChatGPT, Claude, and Google Bard. GO TO NOTE REFERENCE IN TEXT my Twitter followers: Nate Silver (@NateSilver588), “The most important inventions of the decade of the 1900s vs the decade of the 2000s. Pretty good evidence for secular stagnation,” Twitter, x

…

K L M N O P Q R S T U V W X Y Z About the Author Nate Silver is the founder of FiveThirtyEight and the New York Times bestselling author of The Signal and the Noise. He writes the Substack "Silver Bulletin." What’s next on your reading list? Discover your next great read

Fluke: Chance, Chaos, and Why Everything We Do Matters

by Brian Klaas · 23 Jan 2024 · 250pp · 96,870 words

, chaos, and arbitrary accidents play an outsize role in why things happen. In an intertwined world, flukes matter. There can be no true split between “the signal” and “the noise.” There is no noise. The noise of one person’s life is the signal for another, even when we can’t detect it. That same

…

on it. The decision to forage, fight, or take flight are all based on attempts to calculate the unknown. Even without numbers, sophisticated logic, or Nate Silver, animals make informed guesses about the future that are shaped by their experiences. So do we. Every experience in our lives becomes a neurological data

…

even more difficult when the probability moves from weather patterns to a unique, non-repeatable event, such as an election. What does it mean when Nate Silver forecasted that Hillary Clinton had a 71.4 (not 71.3 or 71.5) percent chance of winning the 2016 presidential election? Does it mean

…

the next. Past patterns are a reliable predictor of the future, so probabilities are a safe bet. This is the Land of Stationary Probabilities, where Nate Silver feels most at home. Now, let’s move to thornier problems of uncertainty that arise from our complex, dynamic, contingent, intertwined world, prone to tipping

…

, a one-in-a-billion bizarro reality. With just one Earth to observe, there are some things we may never know. Let’s return to Nate Silver’s forecast in the 2016 presidential election, which predicted that Hillary Clinton had a 71.4 percent chance of victory. The models used by his

…

would happen nearly a third of the time! If you say we were wrong, you don’t understand math! This raises the obvious question: Could Nate Silver’s model ever be “wrong” in that election? When the model predicts something with a low probability and it happens, then it’s just the

…

/mervyn-king-on-radical-uncertainty/. deliberately bias the results: This is sometimes called wet bias, and it’s discussed in Nate Silver, The Signal and the Noise: The Art and Science of Prediction (New York: Penguin, 2013). Nate Silver forecasted: “Who Will Win the Presidency?,” FiveThirtyEight, 8 November 2016, https://projects.fivethirtyeight.com/2016-election-forecast/. Ian Hacking explains

…

October 2016. Silver pointed to his model: For Silver’s evaluation of his 2016 model, see Nate Silver, “The Real Story of 2016,” FiveThirtyEight, 19 January 2017. See also Isaac Faber, “Why You Should Care about the Nate Silver vs. Nassim Taleb Twitter War,” Towards Data Science, 17 December 2018. strong-link problem: Chris

Simple Rules: How to Thrive in a Complex World

by Donald Sull and Kathleen M. Eisenhardt · 20 Apr 2015 · 294pp · 82,438 words

Collin R. Payne, “Mindless Eating and Healthy Heuristics for the Irrational,” American Economic Review: Papers and Proceedings 99, no. 2 (2009): 165–69. [>] Meteorologists make: Nate Silver, The Signal and the Noise (New York: Penguin, 2012), 126–27. [>] Japanese honeybees: Atsushi Ugajin et al., “Detection of Neural Activity in the Brains of Japanese Honeybee Workers During the

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die

by Eric Siegel · 19 Feb 2013 · 502pp · 107,657 words

The “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t Half of what we will teach you in medical school will, by the time you are

…

predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. —Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t The trouble with the world is that the stupid are cocksure and the intelligent are full

…

was about something else. Rather than learning about campaign targeting, when it came to the math behind the election, we heard a great deal about Nate Silver. Silver emerged as the media darling of poll analyzers, soaring past the ranks of guru quant or sexy scientist to become the very face of

…

polls and taking an average . . . and counting to 270, right?” You want power? True power comes in influencing the future rather than speculating on it. Nate Silver publicly competed to win election forecasting, while Obama’s analytics team quietly competed to win the election itself. This reflects the very difference between forecasting

…

/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/. Colbert Nation, www.colbertnation.com. Stephen Colbert interviews Nate Silver, New York Times blogger about his book, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. http://www.colbertnation.com/the-colbert-report-videos/420765/november-05-2012

…

/nate-silver. Peggy Noonan, “They’ve Lost That Lovin’ Feeling.” Wall Street Journal, July 30, 2011. http://online.wsj.com/article/SB10001424053111904800304576474620336602248.html. Jack Gillum

How to Read Numbers: A Guide to Statistics in the News (And Knowing When to Trust Them)

by Tom Chivers and David Chivers · 18 Mar 2021 · 172pp · 51,837 words

-bright 2. ‘How our forecasts measure up’, Met Office blog, 2016 https://blog.metoffice.gov.uk/2016/07/10/how-our-forecasts-measure-up/ 3. Nate Silver, The Signal and the Noise: The Art and Science of Prediction, Penguin 2012. Chapter 18: Assumptions in Models 1. Peter Hitchens, ‘There’s powerful evidence this Great Panic is foolish

The Joys of Compounding: The Passionate Pursuit of Lifelong Learning, Revised and Updated

by Gautam Baid · 1 Jun 2020 · 1,239pp · 163,625 words

other people (which is usually more of a persuasion technique than anything else). 4. Distinguish between the signal and the noise. The signal is the truth. The noise is what distracts us from seeing it. In his book The Signal and the Noise, Nate Silver writes, “There isn’t any more truth in the world than there was before the Internet or

…

aware of the inherent conflicts of interest in the investment industry. Robert Cialdini made me aware of the various psychological tactics used by compliance practitioners. Nate Silver and Philip Tetlock educated me on the follies of forecasting and how we can improve our skills at making estimates through probabilistic thinking, Bayesian belief

…

.com/letters/1987.html. 21. Tetlock and Gardner, Superforecasting. 22. Harvey S. Firestone, Men and Rubber: The Story of Business (Whitefish, MT: Kessinger, 2003). 23. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t (London: Penguin, 2015). 24. Goodreads.com, accessed December 10, 2019, https://www.goodreads.com/quotes/393102

…

Nifty Fifty.” American Association of Individual Investors Journal, October 1998. https://www.aaii.com/journal/article/valuing-growth-stocks-revisiting-the-nifty-fifty. Silver, Nate. The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t. London: Penguin, 2015. Simon, Herbert A. “Designing Organizations for an Information Rich World.” In Computers, Communications

The Behavioral Investor

by Daniel Crosby · 15 Feb 2018 · 249pp · 77,342 words

those operating from a more limited decisional universe. Another consequence of financial information overload is that it leads to drawing spurious correlations between variables. As Nate Silver reports, the government produces data on 45,000 economic variables each year!44 Pair this reality with the fact that there are relatively few dramatic

…

and Your Brain, p. 22. 43 Greg B. Davies, Behavioral Investment Management: An Efficient Alternative to Modern Portfolio Theory (McGraw-Hill, 2012), p. 53. 44 Nate Silver, The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t (Penguin, 2015), p. 185. Chapter 7. Emotion “The world is a tragedy to those who feel

…

access to the model.109 Models have also been shown to outperform human intuition in predicting the outcomes of Supreme Court decisions,110 Presidential elections (Nate Silver), movie preferences, prison recidivism, wine quality, marital satisfaction and military success, to name just a few of the over 45 domains in which they have

The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do

by Erik J. Larson · 5 Apr 2021

like Markram and others. The takeaway here is that the myth really does have practical consequences for our human futures—in actual science. OVERFITTING Statistician Nate Silver has also pointed out the inherent danger of overfitting theories (models) to data, where “overfitting” here means spuriously matching a set of data points to

…

-huge-engineering-efforts. 12. Ibid. 13. Gary Marcus and Ernest Davis, “Eight (No Nine!) Problems with Big Data,” New York Times, April 6, 2014. 14. Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: Penguin Books, 2015). Chapter 17: Neocortical Theories of Human Intelligence 1. Jeff Hawkins, On

Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

by Seth Stephens-Davidowitz · 8 May 2017 · 337pp · 86,320 words

in which candidates are searched also seems to contain information that the polls can miss. In the 2012 election between Obama and Republican Mitt Romney, Nate Silver, the virtuoso statistician and journalist, accurately predicted the result in all fifty states. However, we found that in states that listed Romney before Obama in

…

support from a former leader of the Ku Klux Klan. The same hidden racism that hurt Barack Obama helped Donald Trump. Early in the primaries, Nate Silver famously claimed that there was virtually no chance that Trump would win. As the primaries progressed and it became increasingly clear that Trump had widespread

…

racist searches, in other words, underpay black people. And then there is the phenomenon of Donald Trump’s candidacy. As noted in the introduction, when Nate Silver, the polling guru, looked for the geographic variable that correlated most strongly with support in the 2016 Republican primary for Trump, he found it in

…

thirty. If Boston considered his recent past, his age, and his size, they should, without a doubt, have cut David Ortiz. Then, in 2003, statistician Nate Silver introduced a new model, which he called PECOTA, to predict player performance. It proved to be the best—and, also, the coolest. Silver searched for

…

as Boston defeated St. Louis, 4 games to 2, in the World Series. Ortiz was voted World Series MVP.* As soon as I finished reading Nate Silver’s approach to predicting the trajectory of ballplayers, I immediately began thinking about whether I might have a doppelganger, too. Doppelganger searches are promising in

…

was easy to measure offense and pitching but not fielding, so some organizations ended up underestimating the importance of defense. In fact, in his book The Signal and the Noise, Nate Silver estimates that the Oakland A’s, a data-driven organization profiled in Moneyball, were giving up eight to ten wins per year in the mid

…

feeling a strong enough regret that they momentarily forget that Google cannot help them here. 113 highest support for gay marriage: These estimates are from Nate Silver, “How Opinion on Same-Sex Marriage Is Changing, and What It Means,” FiveThirtyEight, March 26, 2013, http://fivethirtyeight.blogs.nytimes.com/2013/03/26/how

…

://www.espn.com/espnmag/story?id=4223584. 198 how can we predict how a baseball player will perform in the future: This is discussed in Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York: Penguin, 2012). 199 “beefy sluggers” indeed do, on average, peak early: Ryan Campbell, “How

…

Shakespeare, William, 89–90 Shapiro, Jesse, 74–76, 93–97, 141–44, 235, 273 “Shattered” (Rolling Stones song), 278 shopping habits, predictions about, 71–74 The Signal and the Noise (Silver), 254 Silver, Nate, 10, 12–13, 133, 199, 200, 254, 255 Simmons, Bill, 197–98 Singapore, pregnancy in, 190 Siroker, Dan, 211–12 sleep