replication crisis Concepts

Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

by Stuart Ritchie · 20 Jul 2020

more to confuse than enlighten. In the next chapter, we’ll see just how untrustworthy, unreliable and unreplicable the scientific literature has become. 2 The Replication Crisis Vaulting ambition, which o’erleaps itself … William Shakespeare, Macbeth, 1.7.27 Published and true are not synonyms. Brian Nosek, Jeffrey Spies and Matt Motyl

…

, it was a priming study – or, rather, the attempted replication of one – that was among the initial spurs of what has become known as ‘the replication crisis’.7 In that original priming study, the researchers asked participants to find the odd one out from a jumbled list of words, the rest of

…

, the study ‘failed to confirm an effect of power posing on testosterone, cortisol, and financial risk’.19 The critical spotlight that was activated in the replication crisis has also been aimed at older pieces of psychology research, with similarly worrying results. Perhaps the most famous psychology study of all time is the

…

per cent.28 Almost all of the replications, even where successful, found that the original studies had exaggerated the size of their effects. Overall, the replication crisis seems, with a snap of its fingers, to have wiped about half of all psychology research off the map.29 Maybe it’s not quite

…

’ seems to be an apt description. * * * We might try to tell ourselves that there’s something unique about psychology as a discipline that caused its replication crisis. Psychologists have the unenviable job of trying to understand highly variable and highly complicated human beings, with all their different personalities and backgrounds and experiences

…

yet considered, where the immediate consequences of a lack of replicability are indisputable. That field is, of course, medical research. Around the time that the replication crisis was brewing in psychology, scientists at the biotechnology company Amgen attempted to replicate fifty-three landmark ‘preclinical’ cancer studies that had been published in top

…

make the necessary changes to improve the quality of research, its Cassandra-like warnings fell on deaf ears.71 Only since the revelations of the replication crisis – beginning with the publication of Bem’s parapsychological claims and the exposure of Stapel’s fraud in 2011, along with the psychological priming and cancer

…

with a bigger sample and a more rigorous experimental setup.39 But in 2017, years after that failed replication effort and the unfolding of the replication crisis in psychology more generally, Bargh published a bestseller called Before You Know It: The Unconscious Reasons We Do What We Do.40 The book makes

…

findings, the public now lacks confidence and is sceptical of the field’s research.91 Nutritional science, like psychology, has been going through its own replication crisis. Some of this may have to do with fraud: for instance, the cardiologist Dipak Das, who published dozens of much-cited papers on the heart

…

< 0.05 to p < 0.005, creating a much higher hurdle for results to clear before we consider them interesting.32 Given the flaws the replication crisis has revealed, the logic goes, we should be substantially more conservative about what we accept as evidence for our hypotheses. However, the downside of raising

…

really all about is transparency: not hiding away your plans or your analysis from the world. The related concept of ‘Open Science’ far predates the replication crisis but might be one of the most powerful antidotes to it.51 Open Science is the idea that as far as possible, every part of

…

to see their impact factors growing each year. There are, however, two reasons that now is the time for change. The first is the ongoing replication crisis itself. For decades, even centuries, many scientists have harboured serious suspicions about the reliability of results and about the system in which those results are

…

, but if the scandals and shortcomings become sufficiently well known, those same concerns might also help us out of it. In fact, worries about the replication crisis among scientists themselves have already produced a groundswell of support for reform in science. Scientists have shown themselves willing to break en masse with the

…

loops – virtuous cycles that create beneficial self-sustaining norms, rather than perverse ones. For instance, the UK Reproducibility Network, formed by academics concerned about the replication crisis and represented by grassroots groups at universities, is now talking with those universities about ways to change their hiring practices to better reward openness and

…

quiver.122 It’s this: the reforms we’ve discussed in this chapter would all be beneficial for science even if there weren’t a replication crisis. This brings to mind the following classic cartoon about climate change, from the Lexington Herald-Leader’s Joel Pett: With apologies to Pett, let me

…

statistics. Pre-registration. Automated error-checking. Clever ways to catch fraudsters. Preprints. Better hiring practices. A new culture of humility. Etc. Etc. What if the replication crisis is a big hoax and we create a better science for nothing? Epilogue O, while you live, tell truth, and shame the Devil! William Shakespeare

…

fly-by-night findings from fields like nutritional epidemiology, serve to do more damage to trust in science than any amount of discussion about the replication crisis ever could. More importantly, though, a sophisticated view on science isn’t one of unquestioning trust. It’s one that’s pithily summed up by

…

[ing]’ in an open letter signed by twenty-eight scientific societies.24 What all this tells us is that, regardless of our discussion of the replication crisis and its associated failings, politicians will still trample all over science if they think it’ll lead them towards votes. The worry that the arguments

…

in this book might be misappropriated to make selective, insincere attacks on research shouldn’t stop us from publicly discussing the replication crisis and its associated problems. We mustn’t make science suck in its stomach whenever a member of the public or a politician is watching. In

…

Kahneman’s imperative from Chapter 2 that ‘you have no choice’ but to believe in behavioural ‘priming’ effects – effects that were subsequently discredited in the replication crisis? The message of this book is that when faced with a scientific finding, you do have a choice: you can choose to suspend your judgement

…

Shapin & Simon Schaffer, Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life (Princeton: Princeton University Press, 1985). 30. Shapin & Schaffer, Leviathan. 2: The Replication Crisis Epigraph: Brian A. Nosek et al., ‘Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability’, Perspectives on Psychological Science 7, no. 6

…

. 55, 57. 7. As far as I can tell, the term originates in a paper by Pashler & Wagenmakers, who didn’t use the exact phrase ‘replication crisis’ directly but talked of ‘a crisis of confidence’ in psychological research after a string of failed replications. Nelson, Simmons and Simonsohn discuss the triggers of

…

at the time of writing in February 2020. The talk was originally titled ‘Your Body Language Shapes Who You Are’ but at some point, post-replication crisis, it has been renamed ‘Your Body Language May Shape Who You Are’. Amy Cuddy, ‘Your Body Language May Shape Who You Are’, presented at TEDGlobal

…

’ve been hoist by my own petard. I’ve been stressing the importance of robust results, but in making the case that there’s a replication crisis, I’m relying on multi-study replication attempts that weren’t representative samples of all the scientific literature. The conclusion of ‘only about half of

…

, cause for enough concern. For responses to other criticisms of the idea that there’s a crisis, see Harold Pashler & Christine R. Harris, ‘Is the Replicability Crisis Overblown? Three Arguments Examined’, Perspectives on Psychological Science 7, no. 6 (Nov. 2012): pp. 531–36; https://doi.org/10.1177/1745691612463401 30. Alexander Bird

…

, ‘Understanding the Replication Crisis as a Base Rate Fallacy’, British Journal for the Philosophy of Science, 13 Aug. 2018; https://doi.org/10.1093/bjps/axy051 31. Of course

…

://doi.org/10.1038/541269a; Daniel Engber, ‘Cancer Research Is Broken’, Slate, 19 April 2016; https://slate.com/technology/2016/04/biomedicine-facing-a-worse-replication-crisis-than-the-one-plaguing-psychology.html 51. Errington, ‘Reproducibility Project’, slide 11. 52. J. Kaiser, ‘The Cancer Test’, Science 348, no. 6242 (26 June 2015

…

two. Either way, next time I write a scientific paper that has anything to do with the replication crisis, should I have to note something to the effect of ‘I wrote a book about the replication crisis and it would be a bit awkward for me if it turned out that science was actually

…

Rogers, ‘Darpa Wants to Solve Science’s Reproducibility Crisis With AI’, Wired, 15 Feb. 2019; https://www.wired.com/story/darpa-wants-to-solve-sciences-replication-crisis-with-robots/ 10. One such program is RMarkdown: https://rmarkdown.rstudio.com/ 11. An even more radical proposal along these lines is the psychologist Jeff

…

very qualified) defences of the beleaguered p-value, see Victoria Savalei & Elizabeth Dunn, ‘Is the Call to Abandon P-Values the Red Herring of the Replicability Crisis?’, Frontiers in Psychology 6:245 (6 March 2015); https://doi.org/10.3389/fpsyg.2015.00245; Paul A. Murtaugh, ‘In Defense of P Values’, Ecology

…

; https://blogs.lse.ac.uk/impactofsocialsciences/2018/06/04/six-principles-for-assessing- scientists-for-hiring-promotion-and-tenure/ 97. Scott O. Lilienfeld, ‘Psychology’s Replication Crisis and the Grant Culture: Righting the Ship’, Perspectives on Psychological Science 12, no. 4 (July 2017): pp. 661–64; https://doi.org/10.1177/1745691616687745

…

/ct/news/documents/2018-11/veracity_index_2018_v1_161118_public.pdf 16. For a psychology-specific example, see Farid Anvari and Daniël Lakens, ‘The Replicability Crisis and Public Trust in Psychological Science’, Comprehensive Results in Social Psychology 3, no. 3 (2 Sept. 2018): pp. 266–86; https://doi.org/10.1080

…

Hwang affair (2005–6) Macchiarini affair (2015–16) meta-scientific research microbiome studies Morton’s skull studies Obokata affair (2014) outcome switching preprints publication bias replication crisis Reuben affair (2009) spin and statistical power and Summerlin affair (1974) Wakefield affair (1998–2010) biomedical papers bird flu bispectral index monitor black holes Black

…

, George Walker business studies BuzzFeed News California Walnut Commission California wildfires (2017) Canada cancer cell lines collaborative projects faecal transplants food and publication bias and replication crisis and sleep and spin and candidate genes carbon-based transistors Cardiff University cardiovascular disease Carlisle, John Carlsmith, James Carney, Dana cash-for-publication schemes cataracts

…

review randomisation Freedom of Information Acts French, Chris Fryer, Roland Fujii, Yoshitaka funding bias and fraud and hype and long-term funding perverse incentive and replication crisis and statistical power and taxpayer money funnel plots Future of Science, The (Nielsen) gay marriage Gelman, Andrew genetically modified crops genetics autocorrect errors candidate genes

…

affair (2012) Hwang affair (2005–6) Macchiarini affair (2015–16) meta-scientific research Obokata affair (2014) outcome switching pharmaceutical companies preprints pre-registration publication bias replication crisis Reuben affair (2009) spin and statistical power and Summerlin affair (1974) Wakefield affair (1998–2010) medical reversal Medical Science Monitor Mediterranean Diet Merton, Robert Mertonian

…

statistical power typos Netflix Netherlands replication studies in Stapel’s racism studies statcheck research neuroscience amyloid cascade hypothesis collaborative projects Macleod’s animal research studies replication crisis sexism and statistical significance and Walker’s sleep studies neutrinos New England Journal of Medicine New York Times New Zealand news media Newton, Isaac Nielsen

…

Schekman Reagan, Ronald recommendation algorithms red grapes Redfield, Rosemary Reflections on the Decline of Science in England (Babbage) Reinhart, Carmen Rennie, Drummond rent-seeking replication; replication crisis Bargh’s priming study Bem’s precognition studies biology and Carney and Cuddy’s power posing studies chemistry and economics and engineering and geoscience and

…

updates on the author, click here. Contents Title Page Copyright Notice Dedication Epigraph Preface PART I: OUGHT AND IS 1. How Science Works 2. The Replication Crisis PART II: FAULTS AND FLAWS 3. Fraud 4. Bias 5. Negligence 6. Hype PART III: CAUSES AND CURES 7. Perverse Incentives 8. Fixing Science Epilogue

The Hidden Half: How the World Conceals Its Secrets

by Michael Blastland · 3 Apr 2019 · 290pp · 82,871 words

fall over at an alarming rate, failing to replicate as scientists seek to repeat each other’s work. You might have heard talk of a replication crisis, even a crisis of expertise, or research-credibility crisis. Take a moment to absorb that phrase: ‘research’ faces a ‘credibility crisis’. We’re not sure

…

’t say whether people had also had enough of politicians, and I suspect at the time he had little idea of the research credibility or replication crisis, which is more serious and remains under-reported. This book is written against that more serious background – of anxiety about failures of knowledge in the

…

incentives are the same, he expects behaviour to follow. And the incentives are similar just about everywhere. Do cows pay? When I talk about the replication crisis to people who’ve not come across it, they express two main doubts: 1. How is it possible for so many honest researchers to think

…

–1 pretence of 105–6 probabilistic 160, 161, 163–4, 172–3 and probability 180 problem of scale 177–80 provenance 116 relevant 82–5 replication crisis 111–7 subverting 76–110 and time variations 87–100, 91, 93, 94, 95 transfer 37, 76–8, 83, 101–2 unknowns 85–7 validity

…

, 254n14 reciprocity 155 reflection 65–6 regularity 73, 160 assumption of 212–4 expectations of 47, 202–4 search for 212, 230 statistical 240–1 replication crisis 18, 111–7, 117– 22, 129, 136, 138 Replication Project 129 research 111–39 balance of evidence 114 breadth 130 claims inflation 130 confidence in

…

, 122–9 limitations of 117–22 micro-particulars 128 multiple analyses 125–6 multiple conclusions 117–22 overclaiming 134–5 priming 126–8 redemption 20 replication crisis 111–7, 117– 22, 129, 136, 138 rigour 19 scepticism 115–6 standards 129–36 statistical significance 122 triangulation 138 validity 101–2 research-credibility

Super Thinking: The Big Book of Mental Models

by Gabriel Weinberg and Lauren McCann · 17 Jun 2019

have found that fewer than 50 percent of positive results can be replicated. That rate is low, and this problem is aptly positive results the replication crisis. This final section offers some models to explain how this happens, and how you can nevertheless gain more confidence in a research area. Replication efforts

…

, 2 would replicate (5 percent of 40) for a total of 50. The replication rate would then be 50 per 100 studies, or 50 percent. Replication Crisis Re-test 100 Studies So, under this scenario, about a fourth of the failed replications (12 of 50) are explained by a lack of power

…

set of one hundred studies, the base rate of false positives is likely much larger than 5 percent, and so another large part of the replication crisis can likely be explained as a base rate fallacy. Unfortunately, studies are much, much more likely to be published if they show statistically significant results

…

eventually published because a secondary hypothesis was found that did show a significant result. The publication of false positives like this directly contributes to the replication crisis and can delay scientific progress by influencing future research toward these false hypotheses. And the fact that negative results aren’t always reported can also

…

able to detect a smaller effect size Specifying statistical tests to run ahead of time to avoid p-hacking Nevertheless, as a result of the replication crisis and the reasons that underlie it, you should be skeptical of any isolated study, especially when you don’t know how the data was gathered

…

, 305–7 reinventing the wheel, 92 relationships, 53, 55, 63, 91, 111, 124, 159, 271, 296, 298 being locked into, 305 dating, 8–10, 95 replication crisis, 168–72 Republican Party, 104 reputation, 215 research: meta-analysis of, 172–73 publication bias and, 170, 173 systematic reviews of, 172, 173 see also

Rationality: What It Is, Why It Seems Scarce, Why It Matters

by Steven Pinker · 14 Oct 2021 · 533pp · 125,495 words

, and years of birth and death) are ignored. Scientists are not immune to the Texas sharpshooter fallacy. It’s one of the explanations for the replicability crisis that rocked epidemiology, social psychology, human genetics, and other fields in the 2010s.59 Think of all the foods that are good for you which

…

the hypothesis is false. The evidence, in other words, must be extraordinary. A failure of Bayesian reasoning among scientists themselves is a contributor to the replicability crisis that we met in chapter 4. The issue hit the fan in 2011 when the eminent social psychologist Daryl Bem published the results of nine

…

lower credence in claims that are surprising. But the problem is not just with journalists. The physician John Ioannidis scandalized his colleagues and anticipated the replicability crisis with his 2005 article “Why Most Published Research Findings Are False.” A big problem is that many of the phenomena that biomedical researchers hunt for

…

belief is their politics: the farther to the right, the more denial.30 Another cause for gloom is that for all the talk of a replicability crisis, the myside bias is only too replicable. In The Bias That Divides Us, the psychologist Keith Stanovich finds it in every race, gender, cognitive style

…

–63 posterior probability, 151, 153 random sampling of examples, 168 reference class, 167–68 reframing of single-event probabilities into frequencies, 168–70, 349–50n27 replicability crisis and, 159–61 San people and, 4, 166 Signal Detection Theory and, 202, 205, 213, 214, 351n6 subjectivist probability and, 115, 151 updating, 157–58

…

of, 127 editing and fact-checking in, 41, 300–301, 314, 316 innumeracy of, 125–27, 314 recommendations for, 127, 314, 316, 317 and the replicability crisis, 161–62 “yellow journalism,” 125 See also media; pundits judicial system overview of classic illusions of, 321 accountability for lying and, 313 adversarial system of

…

counterfactuals, 64–65 monotheism, 40 the mythology mindset and, 301–2, 307 persecution of, progress against, 330–31 See also God Rendezvous game, 233–34 replicability crisis in science Bayesian reasoning failures and, 159–61 preregistration as remedy, 145–46 questionable research practices and, 145–46, 160, 353n13 science journalism and, 161

…

details, 145–46 randomized controlled trials, 264–68 rational ignorance and, 58 testability and, 299, 316 Trump and rejection of norms of, 284 See also replicability crisis —education not undermining pseudoscience, 305–6 vs. sacred mythological beliefs, 305 shallow understanding in educated people, 295, 305 Scientific Revolution, 94–95 Scissors-Paper-Rock

The Data Detective: Ten Easy Rules to Make Sense of Statistics

by Tim Harford · 2 Feb 2021 · 428pp · 103,544 words

(as Ioannidis had said all along) high-profile psychological studies were more likely not to replicate than to stand up—was sometimes described as a “replication crisis” or a “reproducibility crisis.” In the light of Kickended, perhaps none of this should have been a surprise—but it is shocking nonetheless. The famous

…

has collaborated with Daniel Kahneman and many other psychologists. He struck me as well placed to evaluate psychology as a sympathetic outsider. “I think the replication crisis has been great for psychology,” he told me. “There’s just better hygiene.”36 Brian Nosek, meanwhile, told the BBC: “I think if we do

…

Nosek has given useful interviews to several podcasts, including You Are Not So Smart (episode 100), https://youarenotsosmart.com/2017/07/19/yanss-100-the-replication-crisis/; Planet Money (episode 677); EconTalk, November 16, 2015, http://www.econtalk.org/brian-nosek-on-the-reproducibility-project/; The Hidden Brain (episode 32), https://www

…

.npr.org/templates/transcript/transcript.php?storyId=477921050; as well as BBC Analysis, “The Replication Crisis,” November 12, 2018, https://www.bbc.co.uk/programmes/m00013p9. 11. This figure of thirty-nine is based on the subjective opinion of the replicating

…

,” Twitter, March 20, 2019, 10:40 a.m., https://twitter.com/BrendanNyhan/status/1108377656414879744. 36. Author interview with Richard Thaler, July 17, 2019. 37. “The Replication Crisis,” BBC Analysis, November 12, 2018, https://www.bbc.co.uk/programmes/m00013p9. 38. Antonio Granado, “Slaves to Journals, Serfs to the Web: The Use of

Everything Is Predictable: How Bayesian Statistics Explain Our World

by Tom Chivers · 6 May 2024 · 283pp · 102,484 words

’t think [Jaynes] succeeded,” Kevin McConway of the Open University told me, “but he did have a good try.” CHAPTER TWO Bayes in Science THE REPLICATION CRISIS IN SCIENCE, AND SOME WAYS TO FIX IT In 2011, a series of unwelcome things happened, and science was shaken to the core. Not everyone

…

noticed. The “replication crisis,” as it was known, probably didn’t affect your daily life (it didn’t affect mine for some years, and I was writing about science

…

false—had been shown to be prophetic. You might be wondering what this all has to do with Bayes’ theorem. Well: the cause of the replication crisis has been greatly discussed. It is a story of bad incentives—publish or perish, the demand for novelty—and scientists have come up with many

…

the findings, in order to avoid the novelty filter, is a third. But you could go deeper and say that the underlying cause of the replication crisis is even more basic: it’s that science, like Jakob Bernoulli three hundred years ago, is doing sampling probabilities, not inferential probabilities. A p-value

…

probability that their hypotheses are correct, not just whether or not they’ve been falsified. Instinctively, at least, scientists think like Bayesians. BAYES AND THE REPLICATION CRISIS Here’s the basic advantage that Bayesians have over frequentists: they don’t have to leave data on the table. Jens Koed Madsen, the LSE

…

is still winning the race.’ ” And Bayes does have advantages. For one thing, it definitely does solve, or ameliorate, some of the issues of the replication crisis. Going back to the Lindley paradox: under frequentist analysis, a statistically significant result can actually be evidence against your hypothesis. Because Bayes forces you to

…

of this research does stand up to scrutiny, even after 2011 and Daryl Bem and all those things, knowing what we know now about the replication crisis and the statistical problems in psychology. When presented with questions framed like this, people really do seem to give incoherent, irrational answers. Dan Ariely’s

…

a lot of the work on “social priming” that Kahneman’s book cited has since been undermined, as we discussed in the section on the replication crisis in chapter 2. But it’s definitely true that framing affects how people view risk, and that people misjudge risk on the basis of how

…

bias, 240 Redwood, John, 12 reference class, 257 Registered Reports, 169–170 regression to mediocrity, 87 regression to the mean, 87 relative belief ratio, 143 replication crisis, 117–129, 172, 230 Reproducibility Project, 128 research beliefs in, 134–135 bias in, 93 into intelligence, 88–94 pharmaceutical, 165–166 posterior probability distributions

…

, 117–173 as Bayesian, 329–331 determining statistical significance in, 144–155 finding your priors, 159–167 hypothesis testing, 137–144 Lindley paradox, 155–159 replication crisis in, 117–129 use of priors in research, 129–137 Science (journal), 124 “scientific racism,” 80 SD (standard deviation), 57–59, 90 second derivative, 34

Restarting the Future: How to Fix the Intangible Economy

by Jonathan Haskel and Stian Westlake · 4 Apr 2022 · 338pp · 85,566 words

some important type of investment. Let’s consider two examples: the growing importance of software tools and data in research, and what is called the replication crisis. It is widely agreed that the explosion of computing power in recent decades has increased the returns to data-intensive research.12 Much research involves

…

attitudes, but it is a slow process, and it is constrained by the bureaucratic nature of funders. Another widely recognised issue in science is the replication crisis,14 in which a whole range of research findings that were thought to be reliable turn out to be uncertain: when researchers try to replicate

…

people will cause them, subconsciously, to act in an “elderly” way—was shown to be either nonexistent or much weaker than psychologists had thought. The replication crisis has given rise to systematic attempts to see if time-honoured findings are actually replicable.15 Such replication attempts are often funded by philanthropists, such

…

, Lev, 152 reciprocity, 92–93 regulation, sectoral, 228–30 remote work. See work from home (WFH) rent seeking, 117, 138, 141, 216, 244, 254–55 replication crisis, 129–30 reputation, 92–93 research and development (R&D), 48, 53, 55–58, 124–26, 160, 178, 193, 203 retooling hypothesis, 40, 45 reversion

Nobody's Fool: Why We Get Taken in and What We Can Do About It

by Daniel Simons and Christopher Chabris · 10 Jul 2023 · 338pp · 104,815 words

-fraud.html]. Psychologist Yoel Inbar described his firsthand knowledge of the Stapel case in an episode of the podcast Two Psychologists, Four Beers titled “The Replication Crisis Gets Personal” [https://www.fourbeers.com/4]. Stapel’s memoir, Faking Science: A True Story of Academic Fraud, was published in Dutch in 2012 and

The Art of Statistics: Learning From Data

by David Spiegelhalter · 14 Oct 2019 · 442pp · 94,734 words

.8 The inappropriate use of standard statistical methods has received a fair share of the blame for what has become known as the reproducibility or replication crisis in science. With the growing availability of massive data sets and user-friendly analysis software, it might be thought that there is less need for

…

138–40 regression models 171–4 regression to the mean 125, 129–32, 403 regularization 170 relative risk 31, 403 reliability of data 77–9 replication crisis in science 11–12 representative sampling 82 reproducibility crisis 11–12, 297, 342–7, 403 researcher degrees of freedom 350–1 residual errors 129, 403

On the Edge: The Art of Risking Everything

by Nate Silver · 12 Aug 2024 · 848pp · 227,015 words

in academic journals—the majority of results in some fields—can’t be verified when other researchers try to duplicate them. (This is called the replication crisis.) Occasionally, the reason is something like fraud, but more often the issue is just that statistical inference is hard and the pressure to publish is

…

probability, usually 95 percent. The term is falling out of favor in the River as a result of the adaptation of Bayesian statistics and the replication crisis, the failure of many published academic findings using classical statistics to be verified by other researchers. Steam chasing: In sports betting, the practice of following

…

Silicon Valley, 269–70, 272 regulatory capture, 31, 269, 270, 495 reinforcement learning from human feedback (RLHF), 440–41, 442, 495 Reinkemeier, Tobias, 102–3 replication crisis, 179, 497 Repugnant Conclusion, 364–65, 403, 495 resilience, 116–17 results-oriented thinking, 495 retail bookmakers, 186–90, 187, 489, 518n return on investment