confounding variable

How to Read Numbers: A Guide to Statistics in the News (And Knowing When to Trust Them)
by Tom Chivers and David Chivers
Published 18 Mar 2021

We’ll talk in the next chapter about the difficulties of establishing causality, but it definitely sounds like something worth worrying about here. But there’s something else you need to be aware of when you see that two things – in this case, vaping and marijuana use – are strongly correlated: is there something else that correlates with both? This something else is called a ‘confounding variable’. In case you’re not quite following, here’s an example. The proportion of deaths linked to obesity worldwide by year correlates with the amount of carbon dioxide emitted by year.3 So does carbon dioxide make people fat? Probably not. Instead, what’s likely been going on is that the world is getting richer, and as people get richer, they have more money to spend on both high-calorie foods and on carbon-emitting goods like cars and electricity.

…

If you think that A is causing B, but it in fact turns out that B is (or is also) causing A, that’s called ‘reverse causality’. Of course, it can be more complicated: A can be causing B, which is in turn causing A, in a feedback loop. In the case of violence and economic growth, it’s easy to imagine how that might be the case – and if it is, then it will interfere with your measurement just like a confounding variable. So how can you tell the direction that the causal arrow is pointing? A → B, or B → A, or a loop? One way is to use an instrumental variable – something you can measure which is correlated to one of the things you’re interested in, but not the other. In the case of wars and economic growth, one such instrumental variable is rainfall.

…

There’s a statistical anomaly that crops up from time to time, known as collider bias. It throws up some strange outcomes, making real relationships apparently disappear, or creating imaginary relationships out of nothing: it can even make things look like the opposite of reality. We spoke in Chapter 7 about controlling for confounding variables. Let’s imagine that you’re conducting a study looking at how fast people can run. You notice something: on average, the more grey hairs a person has, the slower their time for the mile. It could be that grey hairs slow you down. Or, perhaps more likely, it could be that both factors are linked to some third factor – maybe age.

Statistics in a Nutshell
by Sarah Boslaugh
Published 10 Nov 2012

This distinction would be missed if only crude mortality rates were considered but becomes clear when a stratified analysis removes the influence of the confounding variable (age) from the outcome (mortality). There is no absolute test for confounding, but there are ways to examine the effects of potential confounders on the relationship of interest and make a reasoned decision about whether confounding is present. The general steps to follow in assessing confounding are as follows: Calculate the crude measure of association, ignoring the confounding variable. Stratify the study population by the confounding variable, that is, divide the population into smaller subgroups based on values of the confounding variable. Calculate an adjusted measure of association.

…

Confounding is sometimes described as the “third variable” problem; the relationship between two variables, say exposure and disease, is mixed up or confounded with the influence of a third variable related to both of them. More than one variable can be involved in confounding, but for the sake of simplicity, we demonstrate methods to deal with a single confounding variable. Researchers in epidemiology need to be alert to the potential for confounding in their data, particularly in observational studies when group membership is not under the control of the investigator. For instance, studies of the effects of smoking on health have to take into account the fact that smoking is a voluntary behavior (people choose to smoke or not to smoke) and people who smoke can differ in many other ways (such as alcohol consumption, diet, or level of education) from those who do not.

…

Matching includes all levels of the confounders but controls enrollment in the study or assignment to groups so that the confounders will be equally distributed across the groups. Matching is commonly used in case-control studies in which controls are selected to match the cases already enrolled in the study. There are different systems for matching, but the basic concept is that categories are constructed for the confounding variables, and assignment to groups is controlled so that the distribution of the confounders is the same in each group. There are two ways to implement matching. In direct matching, individuals are matched on a one-to-one basis. In frequency matching, assignment to the groups is directed or monitored so that equal numbers of the confounders are present in each group.

Bad Science
by Ben Goldacre
Published 1 Jan 2008

To me this is not a great surprise, and it illustrates a very simple issue in epidemiological research called ‘confounding variables’: these are things which are related both to the outcome you’re measuring (wrinkles) and to the exposure you are measuring (food), but which you haven’t thought of yet. They can confuse an apparently causal relationship, and you have to think of ways to exclude or minimise confounding variables to get to the right answer, or at least be very wary that they are there. In the case of this study, there are almost too many confounding variables to describe. I eat well—with lots of olive oil, as it happens—and I don’t have many wrinkles.

…

Now, to be fair to nutritionists, they are not alone in failing to understand the importance of confounding variables, in their eagerness for a clear story. Every time you read in a newspaper that ‘moderate alcohol intake’ is associated with some improved health outcome—less heart disease, less obesity, anything—to gales of delight from the alcohol industry, and of course from your friends, who say, ‘Ooh well, you see, it’s better for me to drink a little…’ as they drink a lot—you are almost certainly witnessing a journalist of limited intellect, overinterpreting a study with huge confounding variables. This is because, let’s be honest here: teetotallers are abnormal.

…

There is a practical problem with this kind of research, of course, which I would hope you might spot: most people do get the MMR vaccine, so the individuals you’re measuring who didn’t get the vaccine might be unusual in other ways—perhaps their parents have refused the vaccine for ideological or cultural reasons, or the child has a pre-existing physical health problem—and those factors might themselves be related to autism. There’s little you can do in terms of study design about this potential ‘confounding variable’, because as we said, you’re not likely to do a randomised controlled trial in which you randomly don’t give children vaccines: you just throw the result into the pot with the rest of the information, in order to reach your verdict. As it happens, Smeeth et al. went to great lengths to make sure their controls were representative.

I Think You'll Find It's a Bit More Complicated Than That
by Ben Goldacre
Published 22 Oct 2014

Here we see how clever things called funnel plots can help to show whether one area’s healthcare really is any worse than another’s, whether an increase in antidepressant prescriptions really does mean more people are depressed (or even whether more people are taking antidepressants), and the core skill of all epidemiology: how to correct for ‘confounding variables’, or rather: how to make sure that apparent correlations in your data are real. In an overview of bicycle helmet research, we review every epidemiological error in the textbooks, and a grand claim about the benefits of screening for diseases helps show that doing something – even something small – can often be worse than doing nothing at all.

…

These studies, however, are vulnerable to many methodological shortcomings. If the controls are cyclists presenting with other injuries in the emergency department, then analyses are conditional on having an accident and therefore assume that wearing a helmet does not change the overall accident risk. There are also confounding variables that are generally unmeasured and perhaps even unmeasurable. People who choose to wear bicycle helmets will probably be different from those who ride without a helmet: they may be more cautious, for example, and so less likely to have a serious head injury, regardless of their helmets. People who are forced by legislation to wear a bicycle helmet, meanwhile, may be different again.

…

q=coffee+hallucinations+location%3Auk exactly what the researchers did: http://dx.doi.org/10.1016/j.paid.2008.10.032 survey is still online: http://psychology.dur.ac.uk:82/srj/caffeine2.html ‘Launay-Slade Hallucination Scale’: http://www.google.co.uk/search?q=Launay-Slade+Hallucination+Scale alternative explanations: http://en.wikipedia.org/wiki/Confounding_variable the academic paper: http://dx.doi.org/10.1016/j.paid.2008.10.032 the press release: http://www.alphagalileo.org/index.cfm?fuseaction=readrelease&releaseid=535120&ez_search=1 ‘multiple comparisons’: http://en.wikipedia.org/wiki/Multiple_comparisons no one was there: http://www.thinkgeek.com/caffeine/accessories/5a65/ draw a target around them: http://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy Voices of the Ancients Voices of the Ancients: http://www.badscience.net/2010/01/voices-of-the-ancients/ Daily Mail: http://www.dailymail.co.uk/sciencetech/article-1240746/Prehistoric-sat-nav-set-ancestors-Britain.html the Metro: http://www.metro.co.uk/news/807855-did-prehistoric-satnav-help-britons-find-their-way Matt Parker: http://www.standupmaths.com/ applied the same techniques: http://bengoldacre.posterous.com/did-aliens-play-a-role-in-woolworths BIG DATA There’s Something Magical About Watching Patterns Emerge from Data There’s Something Magical: http://www.badscience.net/2011/06/theres-something-magical-about-watching-patterns-emerge-from-data/ British Medical Journal: http://www.bmj.com/content/342/bmj.d2983.full first NHS reforms: http://www.guardian.co.uk/society/nhs Give Us the Data A consultation is under way: http://c561635.r35.cf2.rackcdn.com/A-Consultation-on-Data-Policy-for-a-Public-Data-Corporation.pdf foolishly restrictive: http://pdcconsult.ernestmarples.com/ everyday government data: http://www.theguardian.com/politics/government-data forbidden to repurpose it: http://hadleybeeman.net/2011/01/26/uses-for-open-data/ TheyWorkForYou.com: http://www.theyworkforyou.com/ all our postcode information: http://en.wikipedia.org/wiki/Postcodes_in_the_United_Kingdom the house-number boundaries: http://ernestmarples.com/blog/2010/01/postcode-petition-response-our-reply/ make the government: http://pdcconsult.ernestmarples.com/ Care.data Can Save Lives: But Not If We Bungle It greatest need in the NHS http://www.theguardian.com/society/nhs at risk by the bungled: http://www.nature.com/news/power-to-the-people-1.14505?

The Book of Why: The New Science of Cause and Effect
by Judea Pearl and Dana Mackenzie
Published 1 Mar 2018

For example, if we are testing a drug and give it to patients who are younger on average than the people in the control group, then age becomes a confounder—a lurking third variable. If we don’t have any data on the ages, we will not be able to disentangle the true effect from the spurious effect. However, the converse is also true. If we do have measurements of the third variable, then it is very easy to deconfound the true and spurious effects. For instance, if the confounding variable Z is age, we compare the treatment and control groups in every age group separately. We can then take an average of the effects, weighting each age group according to its percentage in the target population. This method of compensation is familiar to all statisticians; it is called “adjusting for Z” or “controlling for Z.”

…

Note that Game 5 is embedded in this model in the sense that the variables A, B, C, X, and Y have exactly the same relationships. So we can transfer our conclusions over and conclude that we have to control for A and B or for C; but C is an unobservable and therefore uncontrollable variable. In addition we have four new confounding variables: D = parental asthma, E = chronic bronchitis, F = sex, and G = socioeconomic status. The reader might enjoy figuring out that we must control for E, F, and G, but there is no need to control for D. So a sufficient set of variables for deconfounding is A, B, E, F, and G. FIGURE 4.7. Andrew Forbes’s model of smoking (X) and asthma (Y).

…

Hypothetical causal diagram for smoking and cancer, suitable for front-door adjustment. Suppose we are doing an observational study and have collected data on Smoking, Tar, and Cancer for each of the participants. Unfortunately, we cannot collect data on the Smoking Gene because we do not know whether such a gene exists. Lacking data on the confounding variable, we cannot block the back-door path Smoking Smoking Gene Cancer. Thus we cannot use back-door adjustment to control for the effect of the confounder. So we must look for another way. Instead of going in the back door, we can go in the front door! In this case, the front door is the direct causal path Smoking Tar Cancer, for which we do have data on all three variables.

Mindware: Tools for Smart Thinking
by Richard E. Nisbett
Published 17 Aug 2015

Occasional acid indigestion but don’t know exactly why? Keep a log of food and drink every day with special attention to likely culprits such as alcohol, coffee, soda, and chocolate. Then conduct an actual randomized experiment—flip a coin to decide whether to have a cocktail. And vary one thing at a time to avoid confounding variables. If you stop eating chocolate and drinking soda and your reflux improves, you won’t know whether it’s the food or the beverage that’s the guilty party. Chapter 12 on verbal report, after considering some more scientific methodology, offers lots more suggestions about experimenting on yourself.

…

We’re almost never able to make that claim. We can only measure what we assume might be important and leave out the infinity of variables that we don’t assume are important. But ATTBW: Assumptions Tend to Be Wrong. So the battle is usually lost right there. Second, how well do we measure each of the possible confounding variables? If we measure a variable poorly, we haven’t controlled for it enough. If we measure a variable so poorly that it has no validity, we haven’t controlled for anything. Sometimes MRA is the only research tool available for examining interesting and important questions. An example is the question of whether religious belief and practice are associated with greater or lesser rates of procreation.

…

Does the Buddhist practice of “loving-kindness”—visualizing smiling at others, reflecting on their positive qualities and their acts of generosity, and repeating the words “loving-kindness”—bring you peace and relieve you of anger toward others? A problem with experiments on the self is that you’re dealing with an N of 1. An advantage, however, is that experiments on the self automatically have a within, before/after design, which can improve accuracy because of the reduction in error variance. You can also keep confounding variables to a minimum. If you’re looking to discover the effect of some factor on you, try to keep everything else constant across the study period when you’re comparing presence of the factor versus absence of the factor. That way you can have a fairly good experiment. Don’t take up yoga at the same time as you move from one house to another or break up with your boyfriend.

How Medicine Works and When It Doesn't: Learning Who to Trust to Get and Stay Healthy
by F. Perry Wilson
Published 24 Jan 2023

Ice cream sales are correlated with the murder rate because both things happen more in the summer (ice cream sales because the temperature is hot and ice cream is cold and delicious, murders because more people are outside, leading to more confrontations). We say that outdoor temperature confounds the observed association between ice cream sales and murder rates. A confounding variable is one that is linked to the exposure of interest and the outcome of interest, and thus induces the illusion of causation between the exposure and the outcome. Another example: In 1999, an article appeared in Nature (one of the most prestigious scientific publications) which found that children who slept with a light on were much more likely to be nearsighted when they grew up.

…

Another example: In 1999, an article appeared in Nature (one of the most prestigious scientific publications) which found that children who slept with a light on were much more likely to be nearsighted when they grew up. The press latched on to this with a causal explanation that perhaps the light exposure at night was limiting eye growth in a way that would promote poor vision. In fact, there was no causal relationship between leaving the light on and poor vision. Future studies found that there was a confounding variable—parental nearsightedness—that wasn’t accounted for. It turns out that parents who are nearsighted are more likely to have kids who are nearsighted. Parents who are nearsighted are also more likely to leave the lights on at night—the easier to check on the kids. In other words, there was a missing variable in the Nature analysis.

…

Second, we have a cheap, widely available, effective, and well-tolerated medicine to increase vitamin D levels; we can take supplemental vitamin D—you don’t even need a prescription. But why are those correlations so strong in the first place? In the case of vitamin D levels, we are (once again, and as usual) in the realm of confounding variables. I have often referred to vitamin D levels as the “lifestyle” biomarker. Let’s think of some behaviors that raise your vitamin D level: getting outside, eating a diet rich in fish (and some mushrooms), and exercise (particularly if you do the exercise out in the sun). In other words, your vitamin D level may merely be a marker for a bunch of healthful behaviors you engage in.

How to Read a Paper: The Basics of Evidence-Based Medicine
by Trisha Greenhalgh
Published 18 Nov 2010

Table 3.1 Terms used to describe design features of clinical research studies Term Meaning Parallel group comparison Each group receives a different treatment, with both groups being entered at the same time. In this case, results are analysed by comparing groups. Paired (or matched) comparison Participants receiving different treatments are matched to balance potential confounding variables such as age and sex. Results are analysed in terms of differences between participant pairs. Within-participant comparison Participants are assessed before and after an intervention and results analysed in terms of within-participant changes. Single-blind Participants did not know which treatment they were receiving.

…

The details of how these different features of teetotalism were controlled for by the epidemiologists are discussed elsewhere [8] [9]. Interestingly, when I was writing the third edition of this book in 2005, the conclusion at that time was that even when due allowance was made in the analysis for potential confounding variables in people who described themselves as non-drinkers, these individuals' increased risk of premature mortality remained (i.e. the J curve was a genuine phenomenon) [8]. But by the time I wrote the fourth edition in 2010, a more sophisticated analysis of the various cohort studies (i.e. which controlled more carefully for ‘sick quitters’) had been published [9].

…

A change in a numerical value may be clinically significant without being statistically significant or vice versa (see section ‘Probability and confidence’), and may also be vulnerable to various biases. For example, in a before and after study, time will have moved on between the ‘baseline’ and ‘post intervention’ measures, and a host of confounding variables including the economic climate, public attitudes, availability of particular drugs or procedures, relevant case law, and the identity of the chief executive, may have changed. Qualitative outcomes may be particularly vulnerable to the Hawthorne effect (staff tend to feel valued and work harder when any change in working conditions aimed at improving performance is introduced, whether it has any intrinsic merits or not) [16].

Back to School: Why Everyone Deserves a Second Chance at Education
by Mike Rose
Published 17 Sep 2012

In writing, fundamentals would be the rules of grammar and punctuation as represented in those workbook exercises. In addition to breaking down, you want to keep a tight focus on the task—writing—and remove potentially confounding variables, like reading skill. So if readings are used, they are usually kept simple and at a minimum. This parsing out of reading from writing is structurally reinforced in many institutions with reading and writing each having its own department. Another potentially 121 BAC K TO S C HO OL confounding variable you want to control for is complexity of topic: what students write about if writing beyond the sentence is involved. The standard remedial playbook for decades and decades includes topics involving one’s personal experience (“Write about an event that changed your life”) or a broad social issue (“Why should we vote?”).

Our Kids: The American Dream in Crisis
by Robert D. Putnam
Published 10 Mar 2015

They had time and money that the poorer kids’ families lacked, and they invested those resources in helping their children acquire valuable soft skills through extracurricular activities. Consistent involvement in extracurricular activities is strongly associated with a variety of positive outcomes during the school years and beyond—even after controlling for family background, cognitive skills, and many other potentially confounding variables. These positive outcomes include higher grade-point averages, lower dropout rates, lower truancy, better work habits, higher educational aspirations, lower delinquency rates, greater self-esteem, more psychological resilience, less risky behavior, more civic engagement (like voting and volunteering), and higher future wages and occupational attainment.50 One carefully controlled study, for example, showed that kids consistently involved in extracurricular activities were 70 percent more likely to go to college than kids who were only episodically involved—and roughly 400 percent more likely than kids who were not at all involved.51 Another study, which has a special relevance to the students we met in Orange County, found that involvement in extracurricular activities among low-income Latino students (all too rare, as the experiences of Lola and Sofia illustrate) predicts school achievement.52 Leadership in extracurricular activities appears to have even more intense effects: one study found that club and team leaders are more likely to command higher salaries in managerial positions later in life.53 And an intriguing study of students who attended high school in Cleveland, Ohio, in the 1940s even found neurological effects a half century later: students who had participated in extracurricular activities were substantially less likely than those who hadn’t to suffer from dementia at the turn of the century, even after adjusting for differences in IQ and educational attainment.54 The only negative finding that emerges from the dozens of studies that have been done on the correlates of extracurricular activities is not startling: among young men, participation in sports is often correlated with excessive drinking (but not drug use).

…

Careful, independent evaluations have shown that formal mentoring can help at-risk kids to develop healthy relations with adults (including parents), and in turn to achieve significant gains in academic and psychosocial outcomes—school attendance, school performance, self-worth, and reduced substance abuse, for example—even with careful controls for potentially confounding variables. These measurable effects are strongest when the mentoring relationship is long-term, and strongest for at-risk kids. (Upper-class kids already have informal mentors in their lives, so adding a formal mentor does not add so much to their achievement.) Measurably, mentoring matters.23 Formal mentoring is much less common and less enduring than informal mentoring.

…

Based on unpublished analyses by Carl Frederick of 2011–2012 data on school quality measures for 85 percent of all public K–8 and high schools in the country, as compiled and published in 2014 by the Office of Civil Rights of the U.S. Department of Education, available at http://ocrdata.ed.gov/. With controls for other potentially confounding variables, including the racial composition of the high school, the fraction of the student body eligible for free or reduced-price lunches, a widely used proxy for student poverty, is uncorrelated with the ratio of counselors to students and is positively correlated with more teachers per 100 students.

Data Science from Scratch: First Principles with Python
by Joel Grus
Published 13 Apr 2015

Correlation after removing the outlier You investigate further and discover that the outlier was actually an internal test account that no one ever bothered to remove. So you feel pretty justified in excluding it. Simpson’s Paradox One not uncommon surprise when analyzing data is Simpson’s Paradox, in which correlations can be misleading when confounding variables are ignored. For example, imagine that you can identify all of your members as either East Coast data scientists or West Coast data scientists. You decide to examine which coast’s data scientists are friendlier: coast # of members avg. # of friends West Coast 101 8.2 East Coast 103 6.5 It certainly looks like the West Coast data scientists are friendlier than the East Coast data scientists.

…

closeness centrality, Betweenness Centrality clustering, Clustering-For Further Explorationbottom-up hierarchical clustering, Bottom-up Hierarchical Clustering-Bottom-up Hierarchical Clustering choosing k, Choosing k example, clustering colors, Example: Clustering Colors example, meetups, Example: Meetups-Example: Meetups k-means clustering, The Model clusters, Rescaling, The Ideadistance between, Bottom-up Hierarchical Clustering code examples from this book, Using Code Examples coefficient of determination, The Model combiners (in MapReduce), An Aside: Combiners comma-separated values files, Delimited Filescleaning comma-delimited stock prices, Cleaning and Munging command line, running Python scripts at, stdin and stdout conditional probability, Conditional Probabilityrandom variables and, Random Variables confidence intervals, Confidence Intervals confounding variables, Simpson’s Paradox confusion matrix, Correctness continue statement (Python), Control Flow continuity correction, Example: Flipping a Coin continuous distributions, Continuous Distributions control flow (in Python), Control Flow correctness, Correctness correlation, Correlationand causation, Correlation and Causation in simple linear regression, The Model other caveats, Some Other Correlational Caveats outliers and, Correlation Simpson's Paradox and, Simpson’s Paradox correlation function, Simple Linear Regression cosine similarity, User-Based Collaborative Filtering, Item-Based Collaborative Filtering Counter (Python), Counter covariance, Correlation CREATE TABLE statement (SQL), CREATE TABLE and INSERT cumulative distribution function (cdf), Continuous Distributions currying (Python), Functional Tools curse of dimensionality, The Curse of Dimensionality-The Curse of Dimensionality, User-Based Collaborative Filtering D D3.js library, Visualization datacleaning and munging, Cleaning and Munging exploring, Exploring Your Data-Many Dimensions finding, Find Data getting, Getting Data-For Further Explorationreading files, Reading Files-Delimited Files scraping from web pages, Scraping the Web-Example: O’Reilly Books About Data using APIs, Using APIs-Using Twython using stdin and stdout, stdin and stdout manipulating, Manipulating Data-Manipulating Data rescaling, Rescaling-Rescaling data mining, What Is Machine Learning?

The Great Stagnation
by Tyler Cowen
Published 24 Jan 2011

We don’t know. The scholarly literature on K-12 education suggests there is no obvious “eyeball-ready” correlation between how much money is spent in U.S. public schools and the quality of final outcomes. On the other hand, you can find studies that parse the data more closely and try to adjust for confounding variables, to claim real returns from higher educational spending. One way of reconciling these contrasting results is to believe that money yields better outcomes when well spent. But how often is that the case? If we are asking the fundamental question of how wealthy we are, it is the absolute rather than the statistically adjusted education results that matter, and we are again back to mediocre performance.

Are We Getting Smarter?: Rising IQ in the Twenty-First Century
by James R. Flynn
Published 5 Sep 2012

The laws can be described as follows: (1) a mistaken deinition of intelligence; (2) ignoring that all human behavior requires at least three levels of explanation; (3) ignoring the signiicance of comparative measurements; and (4) ignoring the fact that social science methodology can eliminate the possibility of confounding variables without reference to brain physiology. Jensen deines intelligence in terms of events in the brain and nervous system. This is no more sensible than deining extroversion in terms of physiological events. There may be a physiological description that explains who is extroverted. But the only reason we are aware of that is because it correlates with extroversion deined as being outgoing in your interaction with other people.

…

We have no absolute measure of the war-making behavior of the Allies and Axis powers during World War II, and certainly cannot reduce their behavior to the physiological level, but we know who won. I cannot give an absolute measure of the ability to classify or use logic on the hypothetical, but I can say we are much better at both today than our ancestors were in 1900. The possibility of confounding variables afflicts all levels of explanation, and science can often deal with them level by level. Do larger brains signal that intelligence has increased or that people have just got bigger? The irst step is to calculate a brain-size-to-body-mass ratio and see if that correlates with IQ. Is it possible that more exposure to tests causes IQ gains over time?

The Art of Statistics: How to Learn From Data
by David Spiegelhalter
Published 2 Sep 2019

A regression analysis had the rate of tumours as the dependent, or response, variable, and education as the independent, or explanatory, variable of interest. Other factors entered into the regression included age at diagnosis, calendar year, region of Sweden, marital status and income, all of which were considered to be potential confounding variables. This adjustment for confounders is an attempt to tease out a purer relationship between education and brain tumours, but it can never be wholly adequate. There will always remain the suspicion that some other lurking process might be at work, such as those with higher education seeking better health care and increased diagnoses.

…

signal and the noise: the idea that observed data arises from two components: a deterministic signal which we are really interested in, and random noise that comprises the residual error. The challenge of statistical inference is to appropriately identify the two, and not be misled into thinking that noise is actually a signal. Simpson’s paradox: when an apparent relationship reverses its sign when a confounding variable is taken into account. size of a test: the Type I error rate of a statistical test, generally denoted by α. skewed distribution: when a sample or population distribution is highly asymmetric, and has a long left- or right-hand tail. This might typically occur for variables such as income and sales of books, when there is extreme inequality.

The Art of Statistics: Learning From Data
by David Spiegelhalter
Published 14 Oct 2019

A regression analysis had the rate of tumours as the dependent, or response, variable, and education as the independent, or explanatory, variable of interest. Other factors entered into the regression included age at diagnosis, calendar year, region of Sweden, marital status and income, all of which were considered to be potential confounding variables. This adjustment for confounders is an attempt to tease out a purer relationship between education and brain tumours, but it can never be wholly adequate. There will always remain the suspicion that some other lurking process might be at work, such as those with higher education seeking better health care and increased diagnoses.

…

signal and the noise: the idea that observed data arises from two components: a deterministic signal which we are really interested in, and random noise that comprises the residual error. The challenge of statistical inference is to appropriately identify the two, and not be misled into thinking that noise is actually a signal. Simpson’s paradox: when an apparent relationship reverses its sign when a confounding variable is taken into account. size of a test: the Type I error rate of a statistical test, generally denoted by α. skewed distribution: when a sample or population distribution is highly asymmetric, and has a long left- or right-hand tail. This might typically occur for variables such as income and sales of books, when there is extreme inequality.

Alchemy: The Dark Art and Curious Science of Creating Magic in Brands, Business, and Life
by Rory Sutherland
Published 6 May 2019

.* To use the analogy of the needle in the haystack, more data does increase the number of needles, but it also increases the volume of hay, as well as the frequency of false needles – things we will believe are significant when really they aren’t. The risk of spurious correlations, ephemeral correlations, confounding variables or confirmation bias can lead to more dumb decisions than insightful ones, with the data giving us a confidence in these decisions that is simply not warranted. A large tech company recently developed an AI system to sift applications for jobs, but it rapidly developed extreme gender prejudices – marking people down if their CV mentioned, say, participation in women’s basketball.

…

While I can see that replacing a browser on a laptop may be indicative of certain qualities – conscientiousness, technological competence and the willingness to defer gratification, to name just three – is it acceptable to use this information to discriminate between employees? The company decided that it wasn’t, in part because it would have been unfair to less privileged applicants, who may have had to use a library computer to apply. Illustration by Greg Stevenson The confounding variable here, missing from the data, is the weather, which explains the spurious correlation. To a dumb alogorithm, it might appear from the graph that ice cream consumption drove people to commit crimes. However, the real reason is simple: people consume more ice cream when it is sunnier, and they also commit more crimes on warm evenings.

The World of Caffeine: The Science and Culture of the World's Most Popular Drug
by Bennett Alan Weinberg and Bonnie K. Bealer
Published 5 Dec 2000

Yet another confounding exposure may arise from neglect of non-dietary sources of caffeine, such as over-the-counter and prescription medicines that people may not realize contain caffeinated and so fail to report. Confounding Variables We have already noted ways in which hidden variables can baffle attempts to understand the relationship between the use of caffeine and any given health outcome. Epidemiologists call such variables “confounders” that, according to one researcher, “plague the literature” about the link between caffeine and problems in human reproduction. An example is the confounding variable of maternal age. Coffee consumption tends to increase with age throughout the childbearing years and, at the same time, the risk of many reproductive hazards also increases with age past 25 or 30.7 People who drink coffee differ in significant ways—over and beyond their use of coffee—from those who do not, and those who drink a great deal of coffee differ from those who drink less.

…

Unfortunately, in the area of human health, the complexity of the human body and mind and the inability of investigators to conduct potentially dangerous experiments on human beings as part of their research often make it difficult to make reliable judgments. There are three potential weaknesses of medical studies that pose problems for those seeking to understand caffeine’s health effects: biased sample selection, inaccurately measured exposure, and failure to exclude confounding variables. Because these limitations can be seen most clearly in studies of caffeine’s effects on the outcomes of pregnancy, it is valuable to examine in detail some of the methodological problems that bedevil scientists working on these questions. While reading this section, however, the reader should keep in mind that similar or identical problems attend investigations of every area of caffeine and human health discussed in this book.1 Sample Selection Studies of the effects of caffeine consumption on reproductive hazards, including the risk of delayed conception, spontaneous abortion, prematurity, low birthweight, and major congenital malformations, multiplied like rabbits throughout the 1980s and 1990s.

Free Money for All: A Basic Income Guarantee Solution for the Twenty-First Century
by Mark Walker
Published 29 Nov 2015

This bourgeoning segment of the labor market has been dubbed the “precariat” by Guy Standing.20 The enormous uncertainty in the lives of the precariat worker is no doubt a source of unhappiness. Even a modest BIG of $10,000 could boost happiness more than we would predict on the basis of income alone, if the fact that it is guaranteed makes a difference to how vulnerable workers feel. Universal health insurance is also another confounding variable. We might expect that some of the unhappiness of the lowest income group is due to lack of affordable health care. If BIG and universal health care were implemented at the same time, it would be 130 FREE MONEY FOR ALL hard to predict the level of happiness for the least well-off. Still, there is every reason to predict that the least well-off will gain significantly in terms of happiness.

…

To illustrate, consider a study by Ed Diener and his colleagues.32 The study assessed the cheerfulness of college freshman and their income 19 years later. The group assessed as more cheerful in college earned more than their less cheerful cohorts 19 years later. The evidence here is suggestive that happiness causes increased earnings. As with any attribution of causality, there are significant worries about confounding variables. In this case, an obvious one is parental income. It does not seem out of the realm of possibility that freshmen whose parents earn more may be happier, and higher parental income may explain why their children are able to earn more. To sort out possible confounding influences, researchers “control” for such variables.

Thinking with Data
by Max Shron
Published 15 Aug 2014

Natural Experiments Our best hope in setting up observational designs is to try to find natural experiments. Consider an effort to determine how being convicted of a crime in high school affects earnings from age 25 to 40. It isn’t enough to just compare earning differences between those who were convicted and those who weren’t, because clearly there will be strong confounding variables. A clever strategy would be to compare earnings between those who were arrested but not convicted of a crime, and those who were convicted. Though there are obvious differences in the groups, they are likely to be similar on a number of other variables that serve as a natural control. This is what is known as a natural experiment: we try to argue that we have made a choice about what to compare that accounts for most alternative explanations.

The death and life of the great American school system: how testing and choice are undermining education
by Diane Ravitch
Published 2 Mar 2010

The first school might very well fail to make AYP if it had a student body that was comparable to the one in the second school.”4 State testing systems usually test only once each year, which increases the possibility of random variation. It would help, Linn says, to administer tests at the start of the school year and then again at the end of the school year, to identify the effectiveness of the school. Even then, there would be confounding variables: “For example, students at the school with the higher scores on the state assessment might have received more educational support at home than students at school B. The student bodies attending different schools can differ in many ways that are related to performance on tests, including language background, socioeconomic status, and prior achievement.”5 The professional organizations that set the standards for testing—such as the American Psychological Association and the American Educational Research Association—agree that test results reflect not only what happens in school, but also the characteristics of those tested, including such elusive factors as student motivation and parental engagement.

…

Teaching to the test predictably narrows the curriculum and inflates test scores, so it is not a good idea. Similarly, it may be a bad idea to base teacher terminations solely on test score data; the data must be supplemented by evaluations conducted by experienced educators. There are too many other confounding variables. Some states give their tests in midyear—which teacher should receive credit or blame for the students’ scores? The one who taught them for nearly five months last year, or the one who taught them for nearly five months before the test was administered? Districts such as Denver are giving bonuses not only to teachers who bring up their students’ scores, but also to those who agree to work in “hard-to-serve” urban schools or accept “hard-to-staff” assignments (e.g., teachers of special education, middle-school mathematics, and English as a second language), or who improve their knowledge and skills (for instance, by getting an advanced degree in the subject they teach).

The Diet Myth: Why America's Obsessions With Weight Is Hazardous to Your Health
by Paul Campos
Published 4 May 2005

The correlation between the two was—like the vast majority of such correlations—almost wholly coincidental (“almost wholly” because it turns out that a tiny percentage of people own so many clothes that this fact in itself appears to cause such persons to be sedentary). The “disease” of Leviosity, or whatever it had been called, would now be recognized as a largely imaginary phenomenon. Except for the business about the medical profession owning up to its mistakes, the above analogy describes the current state of obesity research. Now that confounding variables such as activity levels are at last being taken into account, researchers have discovered that, except for the most extreme cases, the apparent relationship between body mass and health—which was weak to begin with—is in most instances random. However, there is still a fifty-billion-dollar-per-year industry dedicated to convincing Americans that if they don’t get rid of their blue jeans they will be severely endangering their health.

…

These researchers would also turn a skeptical eye on the weight-gaining industry, pointing to the plentiful data that indicated attempts to gain weight did more harm than good, and that many of the health problems suffered by thin people could be traced directly to their unsuccessful efforts to become fatter. Yet the real revolution in emaciation studies would not begin until emaciation researchers started to include confounding variables in their previously simplistic data sets, such as smoking and exercise. Once these factors were taken into account, it would begin to become clear that, contrary to the conventional wisdom, there was actually very little relationship between thinness and poor health, except at the most extreme levels.

The Pot Book: A Complete Guide to Cannabis
by Julie Holland
Published 22 Sep 2010

A longitudinal follow-up of this sample (Sherrill et al. 1991) confirmed deficits in this same ratio. A study of New Zealanders who smoked cannabis daily or almost daily showed deficits in this same measure (Taylor et al. 2000). A follow-up of this New Zealand sample found that the results were no longer statistically significant once confounding variables were included (Taylor et al. 2002). Only further work can help resolve these conflicting results, though it is my sincere hope that users will turn to safer methods of administration that will make COPD and lung airway problems less likely. BRONCHOSCOPE EXAMINATIONS AND BIOPSIES Another approach to examining pulmonary consequences in marijuana users involves examination with a bronchoscope.

…

Potential confounders were addressed in these studies, including other drug use and the question of early psychotic symptoms (Zammit et al. 2002; Arseneault et al. 2004). However, as Weiser and others have pointed out, a two- to threefold increase in risk is not so sizable and could be explained by unrecognized confounding variables (Weiser and Noy 2005b). Finally, there is also the issue of potential publication bias; negative studies that find no association between an exposure and an outcome may be less likely to be published. Biological Plausibility Biological plausibility lends support to the hypothesis of a causal association.

…

In one study, the Dunedin cohort (described earlier), an association was found only for the subgroup of older adolescents; there was no correction for baseline depressive symptoms; and the association was only barely statistically significant. Two other studies, which used standard diagnostic criteria for a major episode of depression, found an increase in depression risk of less than 20 percent associated with cannabis use, which might very well be accounted for by unidentified confounding variables. Another study also found just a marginal increase and only in a subgroup. The only study that found a robust increase in risk (fourfold!) evaluated the existence of depressive symptoms (which was not a diagnosis of depression) over only two weeks in individuals with clear abuse of or dependence on cannabis.

Virtual Competition
by Ariel Ezrachi and Maurice E. Stucke
Published 30 Nov 2016

Under the new paradigm, users may be unable to detect the small but statistically significant change in targeted advertisements (or advertised rates). In a Carnegie Mellon study, all conditions were held equal except one (namely gender or specific sites visited). In this controlled experiment, one can measure the direct impact this one change had. But in our day-to-day reality, there are often too many confounding variables and often no handy benchmark.46 A woman searching the web will not necessarily know to what extent the ads she is seeing differ from those her male counterparts see. She will unlikely ask her male colleagues to report the ads they see and compare the differences. Even if users could detect that they were being targeted with rehab ads (or not being offered advertised opportunities for greater salaries), they may not know why they were being discriminated against.

…

Aniko Hannak, Gary Soeller, David Lazer, Alan Mislove, and Christo Wilson, “Measuring Price Discrimination and Steering on E-Commerce Web Sites,” Proceedings of the 2014 Conference on Internet Measurement Conference, New York, 5, http://www.ccs.neu.edu/home/cbw/pdf/imc151 -hannak.pdf [noting that they could not “determine why results are being personalized based on the data from real-world users, since there are too many confounding variables attached to each . . . user (e.g., their location, choice of browser, purchase history, etc.)”]. Li Xi v. Apple Inc., 603 F. Supp. 2d 464 (E.D.N.Y. 2009) (dismissing end-users’ Robinson-Patman Act claims when the plaintiffs never alleged that they were “competitors engaged in the business of reselling iPhones, that they are in actual competition with a favored purchaser, or that they even resold or attempted to resell their iPhones”); Matthew A.

Small Wars, Big Data: The Information Revolution in Modern Conflict
by Eli Berman , Joseph H. Felter , Jacob N. Shapiro and Vestal Mcintyre
Published 12 May 2018

Economists Nathan Nunn and Nancy Qian looked at exports of food aid from the United States from 1971 through 2006 and its effects on violence across recipients.89 As in some of the other studies we’ve discussed, to disentangle the effects of one variable on another (here, food aid on violence) the researchers needed a source of exogenous variation in food aid—that is, variation in food aid related to violence only through its effect on conflict, as opposed to food aid that occurred because of violence or because of some confounding variable, such as poor governance, that resulted in both violence and food aid. (In the Philippine CCT example, randomization ensured that exogeneity.) Nunn and Qian found an exogenous force: variation in U.S. wheat production. Wheat is the largest component of U.S. food aid, constituting 58 percent of total assistance over the study period.90 The share of U.S. food aid donated to different countries is politically determined, but the level of overall food aid in any given year is dictated by excess U.S. production.

…

The report said, “Kinetic action to counter insurgents can create negative informational effects with the wider population, and thereby lead to strategic losses” (3). Those reports do not pass the causal identification test: the findings of opinion polls, for example, did not control for possible confounding variables, such as response bias. However, it is important and reassuring that military thinkers and practitioners drawing on qualitative evidence come to the same conclusion. 35. “Abdullah Abdullah Pulls out of Afghan Presidential Election Run-off,” Telegraph, 1 November 2009, http://www.telegraph.co.uk/news/worldnews/asia/afghanistan/6478225/Abdullah-Abdullah-pulls-out-of-Afghan-presidential-election-run-off.html, accessed 14 July 2016. 36.

The Numbers Game: The Commonsense Guide to Understanding Numbers in the News,in Politics, and inLife
by Michael Blastland and Andrew Dilnot
Published 26 Dec 2008

The results had significant implications for criminal justice and behavior in general, but seem to have been effectively ignored by the Home Office, which refused us an explanation for its unwillingness to support a follow-up trial and finally acquiesced to a new study only in 2008. Yet this comparison had merit. Care was taken to make sure the two groups were as alike as possible so that the risk of there being some lurking difference, what is sometimes known as a confounding variable, was minimized. The selected prisoners were assigned to the two groups at random, without either researchers or subjects knowing who was receiving the real supplement and who was receiving the placebo until afterward, so that any expectations they might have had for the experiment would not be allowed to interfere with the result.

A Mathematician Plays the Stock Market
by John Allen Paulos
Published 1 Jan 2003

It doesn’t matter if the attributes are ethnicity and hip circumference, or (some measure of) anxiety and hair color, or perhaps the amount of sweet corn consumed annually and the number of mathematics courses taken. Despite the correlation’s statistical significance (its unlikelihood of occurring by chance), it is probably not practically significant because of the presence of so many confounding variables. Furthermore, it will not necessarily support the (often ad hoc) story that accompanies it, the one purporting to explain why people who eat a lot of corn take more math. Superficially plausible tales are always available: Corn-eaters are more likely to be from the upper Midwest, where dropout rates are low.

Arbitrary Lines: How Zoning Broke the American City and How to Fix It
by M. Nolan Gray
Published 20 Jun 2022

Nolan Gray) Combined with other planning initiatives, zoning largely succeeded in preserving segregation where it existed and instituting segregation where it didn’t. Indeed, one study finds that cities that adopted zoning between 1900 and 1930 exhibited significantly higher levels of racial and economic segregation in 1970, controlling for a host of potential confounding variables.2 And while racial segregation has slightly moderated since 1970—no thanks to zoning—class segregation has only continued to worsen, with this trend most pronounced in cities with the most restrictive land-use regulations, zoning key among them. Combine this segregation with unequal public service provision and the result is a system of zoning that methodically corrals the most vulnerable Americans into the worst neighborhoods.

The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet
by Nina Teicholz
Published 12 May 2014

It turns out that people who dutifully follow advice are somehow quite different from the sort of people who don’t; maybe they take better care of themselves in general. Maybe they’re richer. But whatever the reason, statisticians generally agree that this compliance effect is quite large. Therefore, any associations found between meat eating and disease, in order to be meaningful, must be big enough to overcome this compliance effect as well as other confounding variables. Yet, like the small association that Harvard researchers found in their 2012 study, the associations seen between red meat consumption and heart disease have generally been minimal, a scientific detail that study leaders tend not to emphasize and that the mainstream media have also, on the whole, overlooked.

…

Elderly women in the study did see slightly lower rates of heart disease, however. VI. This “healthy volunteer bias” was acknowledged by the study leaders, who tried to account for it (Fraser, Sabate, and Beeson 1993, 533). VII. Gary Fraser, the epidemiologist at Loma Linda University who has recently led the study (which is ongoing), wrote that these “possible confounding variables” made it difficult to zero in on what, exactly, might be protecting health. He objected, even, to the way that nutrition experts such as William Castelli, then director of the Framingham Study, were exaggerating his study’s results. Castelli claimed that Seventh-day Adventists experienced only “one seventh” the risk of heart attacks of other Americans, but the difference was really only “modest,” corrected Fraser (Fraser 1988; Fraser, Sabaté, and Beeson 1993, 533).

Potatoes not Prozac
by Kathleen DesMaisons, Ph. D.

A major constraint of this process is that it takes years to evolve and also depends upon funding that may be affected by a changing political climate. Even more problematic is that science books are about data rather than real people who are living complex, messy lives. Scientists work to exclude the complexity and contradiction of messiness by focusing on one variable at a time, attempting to exclude what are called “confounding variables.” Field clinicians work directly with people rather than simply manipulate numbers. Clinicians are in the trenches of everyday life. When I started the process of looking at the science, I was a clinician in the field of addiction. I brought years of experience to the table. I had worked in mental health, nutrition, and public policy.

Longevity: To the Limits and Beyond (Research and Perspectives in Longevity)
by Jean-Marie Robine , James W. Vaupel , Bernard Jeune and Michel Allard
Published 2 Jan 1997

This effect is not yet fully explained but there is growing evidence that it involves systemic upregulation of endogenous stress response mechanisms. Similar effects are observed in some invertebrate models and are currently under investigation in primates. The idea that nutritional supplements, e.g., antioxidants, can influence life span has some support and merits further study. It is important that such studies control for confounding variables such as food intake; for example if a nutritional supplement reduces food palatibility, it may result in reduced calorie intake. Discussion From the biological point of view, there would appear not to be a fIxed limit to human life span, even though for practical purposes the longevity of our species is rather well-defIned and may prove resistant to change.

The Great Leveler: Violence and the History of Inequality From the Stone Age to the Twenty-First Century
by Walter Scheidel
Published 17 Jan 2017

Economic development and urbanization drove up resource inequality as wealth both expanded and became increasingly unevenly distributed. Detailed registers of wealth taxes that were based on periodic assessments of all urban households serve as a fairly accurate proxy of actual assets and their distribution. Several confounding variables need to be taken into account. Even those residents who were recorded as holding no taxable property would have owned some personal belongings whose inclusion might somewhat reduce measured inequality. At the same time, a general exemption for every household’s first 500 gulden in cash applied—an amount that, taxed at 0.5 percent, was equivalent to a tax payment of 2.5 gulden, or more than anyone below the top fifth of the income distribution paid in 1618.

…

Finally, England, which boasted a degree of maldistribution of material resources similar to that found in all these other societies, did not experience any significant domestic upheavals at all. It is tempting to attribute different outcomes to variation in political institutions or performance in war, but the more confounding variables we bring to the table, the more difficult it becomes to apply a coherent endogenizing theory to a wide range of real-life cases. Much work remains to be done.6 ”PEACE FOR OUR TIME”: ALTERNATIVE OUTCOMES This is equally true of my second question. History has its limits. Any historical account of inequality necessarily focuses on what (we think) actually happened and tries to explain why it did.

Testosterone Rex: Myths of Sex, Science, and Society
by Cordelia Fine
Published 13 Jan 2017

Masculinity threats only had an effect on financial decisions made publicly, suggesting that costly displays of masculinity in response to a threat of manhood are only worth it if they serve a face-saving function.48 Although we have to be careful that findings like these are robust and replicable, they have an important implication, as Nelson points out: Differences that may appear at a cursory level to be due to “essential” differences between the sexes may in fact be due (in part or completely) to some additional, confounding variable, such as societal pressures to conform to gender expectations or locations in a social hierarchy of power, or may no longer be seen when the sampling universe is broadened.49 Yet researchers may nonetheless treat results as though they reflect categorical, Mars versus Venus differences, Nelson goes on to point out.

The No Need to Diet Book: Become a Diet Rebel and Make Friends With Food
by Plantbased Pixie
Published 7 Mar 2019

A large accumulation of fat cells in the central trunk is more harmful than body fat in other places. Also known as visceral fat, it gathers around the body’s organs, as opposed to subcutaneous fat, which sits under the skin. Increased BMI may be linked to various diseases, I don’t deny that, but causation often cannot be established, and there are a whole host of confounding variables that often don’t get corrected for in research. These include other lifestyle factors such as exercise, nutrition and stress, as well as socio-economic and genetic factors. We cannot categorically say that excess weight causes a lot of disease. When it comes to mortality, that’s where things get interesting.

Know Thyself
by Stephen M Fleming
Published 27 Apr 2021

Their loss of metacognition, while real, may be a consequence of changes in other cognitive processes. But if we still find differences in metacognition when other aspects of task performance are well matched between groups or individuals, we can be more confident that we have isolated a change in self-awareness that cannot be explained by other factors. To control for this potential confounding variable, Shimamura needed to find patients with impaired metacognition but intact memory. In a second paper published in 1989, he and his colleagues reported exactly this result. In a group of patients who had suffered damage to their PFC, memory was relatively intact but metacognition was impaired.

May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases—And What We Can Do About It
by Alex Edmans
Published 13 May 2024

Available at https://bit.ly/soccercnn 6. Data is Not Evidence: Causation * The ‘p-value’ corresponds to the significance level, which needs to be 0.05 or lower for a result to be deemed significant. † More technical terms for ‘common causes’ are ‘omitted factors’, ‘omitted variables’ or ‘confounding variables’. ‡ Ideally, we’d have read all the required papers and books before the birth. Instead, we’d gone by conventional wisdom, the NCT course and my desktop search, and planned to exclusively breastfeed Caspar. But, as Mike Tyson said, ‘Everyone has a plan until you get punched in the mouth.’ § For the short-term benefits to the baby, there are fewer allergic rashes, gastrointestinal disorders and possibly ear infections, and a lower chance of necrotizing enterocolitis (a serious intestinal infection).

Sleepyhead: Narcolepsy, Neuroscience and the Search for a Good Night
by Henry Nicholls
Published 1 Mar 2018

It’s unsurprising therefore that sleep apnea should increase the risk of cardiovascular disease. The Wisconsin Sleep Study Cohort illustrates this pretty convincingly, a study established in 1988 to try to get a handle on the causes and consequences of having sleep-disordered breathing. When obvious confounding variables like age, sex, body mass index and smoking are controlled for, the greater the apnea-hypopnea score the higher the incidence of coronary heart disease or heart failure. Those in the study with severe sleep apnea were also many times more likely to have died than those with no sleep apnea. Consider this.

The Better Angels of Our Nature: Why Violence Has Declined
by Steven Pinker
Published 24 Sep 2012

Two political scientists, Bruce Russett and John Oneal, have breathed new life into the Democratic Peace theory by firming up the definitions, controlling the confounding variables, and testing a quantitative version of the theory: not that democracies never go to war (in which case every putative counterexample becomes a matter of life or death) but that they go to war less often than nondemocracies, all else being equal.223 Russett and Oneal untangled the knot with a statistical technique that separates the effects of confounded variables: multiple logistic regression. Say you discover that heavy smokers have more heart attacks, and you want to confirm that the greater risk was caused by the smoking rather than by the lack of exercise that tends to go with smoking.

…

Wilson and George Kelling in their famous Broken Windows theory, was that an orderly environment serves as a reminder that police and residents are dedicated to keeping the peace, whereas a vandalized and unruly one is a signal that no one is in charge.166 Did these bigger and smarter police forces actually drive down crime? Research on this question is the usual social science rat’s nest of confounded variables, but the big picture suggests that the answer is “yes, in part,” even if we can’t pinpoint which of the innovations did the trick. Not only do several analyses suggest that something in the new policing reduced crime, but the jurisdiction that spent the most effort in perfecting its police, New York City, showed the greatest reduction of all.

The End of Illness
by David B. Agus
Published 15 Oct 2012

Snowden set out to collect clues about how we can have a better quality of life as we age by following hundreds of nuns who agreed to take mental tests, fill out questionnaires, and donate their brains after they died to be examined for the telltale plaques and tangles that are the definitive diagnosis for Alzheimer’s disease. A common joke among the nuns in the study became a rallying cry: “When we die, our souls will go to heaven, but our brains will go to Kentucky.” The nuns were ideal subjects for the long-term comparative study because they shared such similar life experiences, without confounding variables such as income, pregnancy, or heavy smoking and drinking. In Snowden’s 2001 book, Aging with Grace, which chronicled his experience, he writes that one of the chief messages to be taken from the study is that upbeat attitudes and mentally active lifestyles may offer protection against the onset of implacable, and still mysterious, dementia.

Reset
by Ronald J. Deibert
Published 14 Aug 2020

It is from studies such as these that we are beginning to understand some of the addictive qualities of our digital experiences (which may help explain the panic you feel when you lose your device). But higher-level effects — e.g., the impact of social media on political polarization or authoritarianism — are far more difficult to untangle from other “confounding variables” (to use the language of social science). Societies are complex, and monocausal theories about them are almost always incorrect. Some of the effects people may attribute to social media — e.g., decline of trust in public institutions — are almost certainly the result of multiple, overlapping factors, some of which reach back decades.

California Burning: The Fall of Pacific Gas and Electric--And What It Means for America's Power Grid
by Katherine Blunt
Published 29 Aug 2022

* * * For a two-week stretch in June, the jury heard from a parade of witnesses, first federal pipeline regulators and then PG&E employees. The prosecution flashed evidence of the company’s poor record keeping and haphazard approach to managing changes in pipeline pressure. Bauer scored points on cross-examination, homing in on the many confounding variables that muddied the case. Then, on the morning of July 7, the defense’s hope for victory began to fade. The jury filed into the courtroom to find Bill Manegold, a lifelong engineer who had spent thirty-five years at PG&E. He had retired in 2014 after nearly a decade within the gas division supervising transmission operations.

What’s Your Type?
by Merve Emre
Published 16 Aug 2018

Gone were the nights spent hunched over her Monroe calculator, the hand-crank adding machine she kept in her living room next to the moldering stacks of answer keys she had accumulated over the previous decade. Gone too was the haphazard, if intimate, research program that had defined the indicator’s early days. For Isabel, the typing of friends and family members had proceeded without controlling for any of the confounding variables that would differentiate introversion and extraversion, sensing and intuiting, thinking and feeling, judging and perceiving, from the other factors that determined an individual’s personality: age, gender, economic status, education, political orientation. “Neither of these authors has had formal training in psychology, and consequently little of the very extensive evidence they have developed on the instrument is in a form for immediate assimilation by psychologists generally,” Chauncey warned his staff as he prepared them to start work on the indicator.

Why We Drive: Toward a Philosophy of the Open Road
by Matthew B. Crawford
Published 8 Jun 2020

Crawford didn’t know much about rats but Kelly Lambert, her colleague in the psychology department at the University of Richmond, did, so the two of them teamed up. According to Lambert, we have no idea what rats are capable of because rat research has always been conducted in a highly controlled lab environment, the point of which is to study the effect of one variable and eliminate confounding variables as much as possible. But it is precisely the richness of an animal’s environment that calls forth its evolved capacities, many of which depend on a process of development (during the rat’s own lifetime) in a natural setting no less than the selection pressures that have shaped the species over generations.

Doing Data Science: Straight Talk From the Frontline
by Cathy O'Neil and Rachel Schutt
Published 8 Oct 2013

In fact, you should have very good evidence that stratification helps before you decide to do it. What Do People Do About Confounding Things in Practice? In spite of the raised objections, experts in this field essentially use stratification as a major method to working through studies. They deal with confounding variables, or rather variables they deem potentially confounding, by stratifying with respect to them or make other sorts of model-based adjustments, such as propensity score matching, for example. So if taking aspirin is believed to be a potentially confounding factor, they adjust or stratify with respect to it.

Rich White Men: What It Takes to Uproot the Old Boys' Club and Transform America
by Garrett Neiman
Published 19 Jun 2023

The glass cliff phenomenon may also apply to entire fields. In 2017, the Association of American Medical Colleges found that, for the first time, women represented the majority of medical school matriculants.21 The finding was rightly heralded as a win for gender equality. However, there are also some confounding variables. The medical profession is not as lucrative as it used to be: the pay is still good, but the increase in expenses associated with becoming a doctor—like medical school tuition and malpractice insurance—is far outpacing inflation.22 What finally tipped the scale was not only that female applications grew by 10 percent but also that male applications dropped by 2 percent.

Democracy and Prosperity: Reinventing Capitalism Through a Turbulent Century
by Torben Iversen and David Soskice
Published 5 Feb 2019

Of course, there are many potential confounders, but it is hard to think of any with a more clearly specified micrologic (consistent with our other evidence). The most obvious candidate would be GDP per capita, but while it has a borderline statistically significant negative effect (not shown), it has no effect on the finding for educational opportunity. Other potentially confounding variables such as occupational structure are already controlled for at the individual level, and we can confirm this by including industrial employment shares as a macrovariable: it has no effect. No other argument we are aware of explains the cross-national pattern observed in the data. In concluding this section, we would like to draw attention to the remarkable fact that countries with relatively weak populist sentiments are often noted for having strong populist parties, and vice versa.

Food Allergy: Adverse Reactions to Foods and Food Additives
by Dean D. Metcalfe
Published 15 Dec 2008

Atherton et al. [38] showed that 14 of 20 subjects (70%) with AD between the ages of 2 and 8 years showed significant improvement after completing a 12-week, double-blind, controlled, crossover trial of an egg and cow’s milk (CM) exclusion diet. However, there were notable problems in this study, including 16 of the 36 subjects who did not complete the study (44% dropout) and poor control of confounding variables such as environmental factors and other triggers of AD. Neild et al. [39] studied 53 subjects with AD in another trial using a similar design as Atherton and colleagues. Forty of the 53 subjects (75%) complied adequately with the trial regimen, but only 10 of the 40 subjects benefited from the egg and CM exclusion diet, yielding a response rate to the diet that was not statistically significant.

…

In practice, reporting of results often varies by investigator and may be reported as mean diameter, mean diameter compared to histamine control categorically (e.g. 1⫹, 2⫹, etc.), or as a calculated area. Studies must be evaluated carefully because individual investigators may be reporting data based on a variety of methods that may not be directly comparable (e.g. mean wheal diameter versus largest diameter). Despite the numerous potential confounding variables involved in the PST procedure, the clinical utility is excellent. Technical issues that can impact PST sensitivity are summarized in Table 20.2. Diagnostic value The ability of a test to indicate the presence or absence of disease depends on intrinsic characteristics of the test itself and also features of the population on which it is being applied.

…

Arshad et al. [48] in a study of 120 infants randomized participants to either a prophylactic group (breast-fed with mother on a low allergen diet or given an extensively hydrolyzed formula and house dust mite reduction) or a control group (who followed standard UK DoH advice). Findings demonstrate a reduction in allergic disease (asthma, atopy, rhinitis, and eczema) at least for the first 8 years of life in the prophylactic group. Repeated measurement analysis, adjusted for all relevant confounding variables, confirmed a preventive effect on asthma, AD, rhinitis, and atopy. The protective effects were primarily observed in the subgroup of children with persistent disease (symptoms at all visits) and in those with evidence of allergic sensitization. Study powering did not allow for the assessment of food allergy at 8 years of age, but earlier transient effects were noted.

Safe Food: The Politics of Food Safety
by Marion Nestle
Published 1 Jan 2010

The quality of scientific research depends not only on the question under investigation (some research questions are more interesting and important than others) and the care (“rigor”) with which studies are conducted, but also on the ability of the studies to eliminate (“control for”) all possible causes of the observation other than the one being tested. Scientific methods also extend beyond observations to suggest probable causes, to exclude irrelevant causes (“confounding variables”), and to estimate the probability that a particular cause is the true reason for the observation of interest. The point here is that probability is not the same as proof. Biological experiments in humans are complicated by genetic variation and behavioral differences, and study results nearly always depend on probabilities and statistics.

Wool Omnibus Edition
by Hugh Howey
Published 5 Jun 2012

Someone or some thing was always at fault. She knew to listen, to observe, to ask questions of anyone who could have had anything to do with the faulty equipment or the tools that had served the equipment, following a chain of events all the way down to the bedrock itself. There were always confounding variables—you couldn’t adjust one dial without sending something else a-kilter—but Juliette had a skill, a talent, for knowing what was important and what could be ignored. She assumed it was this talent that Deputy Marnes had originally seen in her, this patience and skepticism she employed to ask one more stupid question and stumble eventually onto the answer.

The 4-Hour Body: An Uncommon Guide to Rapid Fat-Loss, Incredible Sex, and Becoming Superhuman
by Timothy Ferriss
Published 1 Dec 2010

It isn’t quite that easy. I also started cold treatments and supplemental selenium (Brazil nuts), both of which could have contributed, the latter more likely than the former. Do I care about the academic purity? No. I was more concerned with increasing sperm count than isolating variables. Even with two confounding variables, the experiment is directionally valid. Should you wait for a scientific consensus? I don’t think so. This is a case where the current literature is strong enough, and the inconvenience minimal enough, to not wait for doctor’s orders. It can’t hurt you, and it might get your swim team off the bench and back in the game.

Adaptive Markets: Financial Evolution at the Speed of Thought
by Andrew W. Lo
Published 3 Apr 2017

Bell (1927), 171 Bucy, Paul, 78–79 Buffett, Warren, 6, 11, 225, 231, 234–235, 286, 301, 407 Burch, Robert L., 234 Burnham, Terry, 337–338 “butterfly effect,” 361 Caisse d’Epargne, 61 California Public Employees’ Retirement System, 409 Camping, Harold, 342 Canada, 242 cancer, 400–410 Candide (Voltaire), 139 candlestick charting, 17, 23 cap-and-trade system, 416 Capital Asset Pricing Model (CAPM), 27, 212, 249, 251–252, 263, 267, 282 capital requirements, 62, 306–308, 311, 368 capitalism, 7, 89, 412 carbon tax, 416 Cardano, Girolamo, 17, 21, 27 Carlsson, Arvid, 88 Carnegie Mellon University, 33–34, 172, 178, 181 Carroll, Lewis, 322 Carter, Jimmy, 401 Carville, James, 9–10 Case, Karl, 314 Case-Shiller Index, 314 Caspi, Avshalom, 160 cause and effect, 128 Caves, Dick, 127 Cayne, Jimmy, 304, 305 cell phones, 246–248 Center for Adaptive Behavior and Cognition (ABC), 216 Center for Research in Security Prices (CRSP), 254 central banks, 230, 291, 301, 368, 391 Ceradase, 419 Cerezyme, 419 Challenger (space shuttle), 12–16, 24, 38 Chan, Nicholas, 41, 317 chaos theory, 278 Chen, Jiulin, 61 chess, 112, 131, 179 Chicago Board Options Exchange (CBOE), 270, 356–358 Chicago Mercantile Exchange (CME), 360, 368–369, 370 Chicago School, 25 Chicxulub meteor, 241–242 chief risk officers (CROs), 392 chimpanzee, 150, 153, 337 China, 258, 409–410, 411, 412 China Aviation Oil, 61 Citigroup, 318–319 Clark, Luke, 91 climate change, 364, 416 Clinton, Bill, 10, 85 cobweb theorem, 31, 33, 34, 109 Coca Cola, 284–285, 384 cocaine, 89, 90, 91 Coffee, John C., 309–310 Cohn, Alain, 352–353 Cold War, 52 collateralized debt obligations (CDOs), 298, 299, 343 Collier, Paul, 412 Colossal Failure of Common Sense, A (McDonald and Robinson), 317–318 commercial banks, 293, 301, 308, 335, 371 commodities trading, 20, 34 Commodity Futures Trading Commission (CFTC), 359, 360, 377 common law, 372 competition, 3, 153, 168, 214, 217 complexity, 217, 278, 361–364, 372, 374 computational biochemistry, 240 computerized axial tomography (CAT), 78, 102 confirmation bias, 305–306 confounding variables, 139 congenital analgesia, 378 Congo Free State, 412 consilience, 215 Consolidated Supervised Entities, 306 contrarian strategy, 290, 316, 325 controlled experiments, 47, 139 Cook, William, 236 cooperation, 164–165, 168, 214, 336, 340 Coppersmith, Don, 239 core, in networks, 374–376 corn, 28–29, 30 corpus callosum, 113–114 Cortana, 396 cortex, 81, 130; anterior cingulate, 86, 105; prefrontal, see prefrontal cortex cortisol, 81 Cosmides, Leda, 173, 174 cost-benefit analysis, 104, 119, 121–122, 169, 316 Cost Matters Hypothesis, 265, 397 Cotzias, George, 88 Countrywide Financial, 325 coupling, 321–322, 361, 372–374 creative destruction, 219 credit default swaps (CDSs), 298, 300, 379, 407 credit rating agencies, 301 Crick, Francis, 137, 144, 401 Cronqvist, Henrik, 161 crowded trades, 291–292, 293 crowdfunding, 356 cryptography, 238–239, 385 currency trading, 12–16, 24, 38 D.

The 4-Hour Chef: The Simple Path to Cooking Like a Pro, Learning Anything, and Living the Good Life
by Timothy Ferriss
Published 1 Jan 2012

For those of you who want to “go beyond” in your research, I have provided links to Internet resources. My team and I have worked to check that these links are accurate and point to resources available when this book was released for publication. But Internet resources change frequently, and other confounding variables beyond my control intervene. So, for various reasons, the links may not direct you to the resource I had intended. In many cases, you will likely be able to use your favorite search engine to locate the correct link. Where links to a good resource are not working, and avid readers among you let me know, we will work to provide updated and corrected links in posts or pages at fourhourblog.com.

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress
by Steven Pinker
Published 13 Feb 2018

The developing world has thrived in that realm as well, often leapfrogging the West in its adoption of smartphones and of applications for them such as mobile banking, education, and real-time market updates.41 Could the Flynn effect help explain the other rises in well-being we have seen in these chapters? An analysis by the economist R. W. Hafer suggests it could. Holding all the usual confounding variables constant—education, GDP, government spending, even a country’s religious makeup and its history of colonization—he found that a country’s average IQ predicted its subsequent growth in GDP per capita, together with growth in noneconomic measures of well-being like longevity and leisure time.

Theory and Practice of Group Psychotherapy
by Irvin D. Yalom and Molyn Leszcz
Published 1 Jan 1967

To do so thrusts the individual into a state of uncomfortable dissonance. Research on marathon groups is plagued with a multitude of design defects. 16 Some studies failed to employ proper controls (for example, a non–time-extended comparison group). Others failed to sort out the effects of artifact and other confounding variables. For example, in a residential community of drug addicts, an annual marathon group was offered to rape survivors. Because the group was offered only once a year, the participants imbued it with value even before it took place.17 The rigorous controlled studies comparing differences in outcome between time-extended and non–time-extended groups conclude that there is no evidence for the efficacy of the time-extended format.

Ashes to Ashes: America's Hundred-Year Cigarette War, the Public Health, and the Unabashed Triumph of Philip Morris
by Richard Kluger
Published 1 Jan 1996

But the committee’s sense of duty went further and was perhaps best exemplified by the attentive ear that staff director Hamill gave to the industry’s chief scientific representative, Clarence Little, who at seventy-five was performing his final well-paid services for the tobacco manufacturers. He was seeking assurance that the Surgeon General’s panelists would take into account all the possibly confounding variables in the evidence before them. He and Hamill proceeded as wary performers, if not open antagonists, in a high drama, with Hamill recognizing what he called Little’s “blind spot”—an almost dismissive attitude toward the laws of statistics and probability. “You had to see a thing happen in the laboratory to convince him it was true,” Hamill recalled of their exchanges.