The Cult of Statistical Significance: Unmasking the Truth Behind Data Analysis

Imagine a scenario where your life decisions, medical treatments, and business strategies are solely driven by statistical significance. Sounds absurd, doesn't it?

However, in the realm of data analysis, many have succumbed to the seductive trap of blindly following statistical significance alone. This has led to the emergence of what can only be described as "The Cult of Statistical Significance" - a group of individuals who obsess over p-values and fail to recognize the limitations and nuances of this statistical concept.

The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives (Economics, Cognition, And Society)

by Tilman Borgers (Kindle Edition)

4.2 out of 5

Language	:	English
File size	:	1010 KB
Text-to-Speech	:	Enabled
Screen Reader	:	Supported
Enhanced typesetting	:	Enabled
Word Wise	:	Enabled
Print length	:	341 pages

FREE

What is Statistical Significance?

Statistical significance is a concept used to determine the likelihood that an observed result occurred by chance. It is widely utilized in various disciplines to evaluate the validity and reliability of research findings.

Traditionally, researchers claim statistical significance if a p-value (the probability of obtaining the observed result under the null hypothesis) falls below a predetermined threshold, commonly set at 0.05. Alternatively, they may express confidence intervals in which the true value of a parameter is expected to lie.

The Problems with Blindly Following Statistical Significance

While statistical significance can be a valuable tool when used appropriately, the blind devotion to this measure has led to several issues:

1. Overemphasis on p-values: Many researchers solely focus on p-values, disregarding effect sizes and practical significance. This can lead to misleading s, especially when dealing with large sample sizes.
2. P-hacking and Publication Bias: Researchers may manipulate data, conduct multiple tests, or selectively report results to achieve statistical significance and get their studies published. This practice distorts scientific knowledge and can skew the literature.
3. Misinterpretation: Statistical significance does not imply practical significance. A statistically significant result might have minimal or no real-world impact. Ignoring effect sizes and context can lead to misguided decision-making.
4. False Sense of Certainty: Statements like "X is statistically significant" can falsely project certainty and undermine critical thinking. The probabilistic nature of statistics should be acknowledged to avoid misguided s.

Alternatives to Statistical Significance

Several alternatives and supplementary methods exist to complement statistical significance:

• Effect Sizes: Measuring the size of an observed effect provides a more comprehensive understanding of the practical significance.
• Replication: Results that are consistently replicated by different researchers strengthen the evidence for a specific claim.
• Meta-Analysis: Combining findings from multiple studies allows for a more accurate understanding of the overall effect. It reduces the impact of small sample sizes and increases statistical power.

Concluding Thoughts

The Cult of Statistical Significance may have inadvertently misled researchers and practitioners. While statistical significance is an important tool, it should not be worshipped blindly. It is crucial to consider effect sizes, practical significance, and replication before drawing s or making decisions based solely on statistical significance.

Breaking free from this cult-like mentality is necessary to ensure the advancement of accurate knowledge and promote responsible data analysis practices.

The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives (Economics, Cognition, And Society)

by Tilman Borgers (Kindle Edition)

4.2 out of 5

Language	:	English
File size	:	1010 KB
Text-to-Speech	:	Enabled
Screen Reader	:	Supported
Enhanced typesetting	:	Enabled
Word Wise	:	Enabled
Print length	:	341 pages

FREE

“McCloskey and Ziliak have been pushing this very elementary, very correct, very important argument through several articles over several years and for reasons I cannot fathom it is still resisted. If it takes a book to get it across, I hope this book will do it. It ought to.”

—Thomas Schelling, Distinguished University Professor, School of Public Policy, University of Maryland, and 2005 Nobel Prize Laureate in Economics

“With humor, insight, piercing logic and a nod to history, Ziliak and McCloskey show how economists—and other scientists—suffer from a mass delusion about statistical analysis. The quest for statistical significance that pervades science today is a deeply flawed substitute for thoughtful analysis. . . . Yet few participants in the scientific bureaucracy have been willing to admit what Ziliak and McCloskey make clear: the emperor has no clothes.”

—Kenneth Rothman, Professor of Epidemiology, Boston University School of Health

The Cult of Statistical Significance shows, field by field, how “statistical significance,” a technique that dominates many sciences, has been a huge mistake. The authors find that researchers in a broad spectrum of fields, from agronomy to zoology, employ “testing” that doesn’t test and “estimating” that doesn’t estimate. The facts will startle the outside reader: how could a group of brilliant scientists wander so far from scientific magnitudes? This study will encourage scientists who want to know how to get the statistical sciences back on track and fulfill their quantitative promise. The book shows for the first time how wide the disaster is, and how bad for science, and it traces the problem to its historical, sociological, and philosophical roots.

Stephen T. Ziliak is the author or editor of many articles and two books. He currently lives in Chicago, where he is Professor of Economics at Roosevelt University. Deirdre N. McCloskey, Distinguished Professor of Economics, History, English, and Communication at the University of Illinois at Chicago, is the author of twenty books and three hundred scholarly articles. She has held Guggenheim and National Humanities Fellowships. She is best known for How to Be Human* Though an Economist (University of Michigan Press, 2000) and her most recent book, The Bourgeois Virtues: Ethics for an Age of Commerce (2006).