(Essay found in Nesselroade & Grimm, 2019; pgs. 705 – 706) Throughout this book we have periodically stopped to look more closely at some commonly discussed problems in the world of science; in particular, we have tried to understand better the current reproducibility crisis that is afflicting the social sciences. Let us finish this series by stepping back to look a bit more philosophically at the scientific endeavor as a whole. What can we hope to accomplish with the help of science, and what, if anything, lies on the outside? This has sometimes been referred to as science’s demarcation problem;

(Essay found in Nesselroade & Grimm, 2019; pg. 355) Using the 1960 volume of the Journal of Abnormal and Social Psychology, Cohen (1962) conducted an interesting study. Although the authors of the articles in that volume did not use power analyses, Cohen computed the power of the statistical tests used in each of the studies. According to Cohen’s early guidelines, a small effect size is about 0.20; a medium effect size is around 0.50; and a large effect size is around 0.80. Assuming that the researchers would want to detect a medium treatment-effect size, the average power of the tests

(Essay found in Nesselroade & Grimm, 2019; pg. 324) This is another box in the series exploring the various reasons for the current reproducibility crisis in the social, behavioral, and medical sciences. Fellow researchers sometimes wonder if the use of one-tailed tests in the literature occurs because it is the only way to reject the null hypothesis. The following study may be a case in point. Buttery and White (1978) were interested in the relationship between affective states (feelings) and biorhythms. According to biorhythm theory, people experience a 28-day emotional cycle. At the peak of the cycle, people are expected

(Essay found in Nesselroade & Grimm, 2019; pgs. 236 – 237) If there is a “ground zero” for the current reproducibility crisis in the social, behavioral, and medical sciences, it may be found in the personhood of John Ioannidis, Professor of Medicine and of Health Research and Policy at the Stanford University School of Medicine. In 2005, he published an article in PLoS Medicine entitled, “Why Most Published Research Findings are False.” As one might imagine, this article created a firestorm of controversy as well as an avalanche of articles reacting to this claim; some supporting (e.g., Freedman, 2010), some

(Essay found in Nesselroade & Grimm, 2019; pgs. 200 – 201) This is another box in the series looking at the reproducibility crisis in the social, behavioral, and medical sciences. When researchers conclude that the null cannot be rejected (also known as “failing to reject the null hypothesis”) the study’s findings are deemed “non-significant.” This term is a way of expressing the idea that any differences between the sample means of the various conditions in a study are not substantial enough to warrant rejecting the null hypothesis of no difference. (The degree of differences needed to be found between sample

(Essay found in Nesselroade & Grimm, 2019; pgs. 166 – 167) An aspect of probabilistic thinking that seems to have been lost in most modern discussions of probability, and which may be partly responsible for the reproducibility problem in the social and medical sciences, is the realization that uncertainty is not merely the quantification of likelihood, but is also influenced by a clear understanding of the situation; let us use the term “clarity” for a lack of a better one. Now “likelihood” (or “risk” as it is sometimes called) is usually understood as something that can be quantified numerically; like

(Essay found in Nesselroade & Grimm, 2019; pg. 115) Throughout the text a series of several “Boxes” are asking whether the scientific method is broken in light of the non-reproducibility problem currently plaguing the social, behavioral, and medical sciences. In Box 1.1 we looked at the “Wallpaper Effect” and the difficulty in identifying and controlling all extraneous variables. In Box 2.3 we looked at, among other things, different ways the collection of data may be biased through wording effects and order effects. In this box, let us explore some of the problems that occur in the data gathering process. Sometimes

(Essay found in Nesselroade & Grimm, 2019; pgs. 57 – 58) In Box 1.1 we started a series asking whether the scientific method is broken. Public polling suggests most Americans do not possess a ‘great deal of confidence’ in the scientific community (Confidence in Institutions: Trends in Americans’ Attitudes toward Government, Media, and Business, 2016). Part of the problem might be the misrepresentation of scientific data and findings. Data misrepresentation can occur in a number of different ways. One way concerns how science writers interpret scientific findings for the general public. Since most people get their scientific information from the

(Essay found in Nesselroade & Grimm, 2019; pg. 17) The public has recently been informed about a troubling discovery in the world of social and medical science investigation; the non-reproducibility of many scientific findings. Titles like, ‘Scientific Regress,” (Wilson, 2016) “Does Social Science Have a Replication Problem?” (Tucker, 2016) and “Over Half of Psychology Studies Fail Reproducibility Test” (Baker, 2015) seem to be popping-up all over the place. The titles are unnerving and the issues that are raised are both real and serious. Briefly stated, an alarming amount of published research does not produce, when attempts at replication of the