(Essay found in Nesselroade & Grimm, 2019; pg. 17)

The public has recently been informed about a troubling discovery in the world of social and medical science investigation; the non-reproducibility of many scientific findings. Titles like, ‘Scientific Regress,” (Wilson, 2016) “Does Social Science Have a Replication Problem?” (Tucker, 2016) and “Over Half of Psychology Studies Fail Reproducibility Test” (Baker, 2015) seem to be popping-up all over the place. The titles are unnerving and the issues that are raised are both real and serious. Briefly stated, an alarming amount of published research does not produce, when attempts at replication of the study are performed, the same findings that were stated in the original publication. Upon learning this, it is quite natural for people to wonder if there is something fundamentally wrong with the scientific method or the conduction of scientific research.

There are many aspects to this problem. This Box attempts to analyze one such aspect. (Several others will be featured in Boxes found throughout the rest of the textbook.) Many suspect that a leading reason for replication failure is the inability of the replicating researchers to reproduce accurately, fully, and completely the precise conditions under which the original study was performed. Even though the replicators faithfully follow every aspect of the methodological situation as recorded by the original researchers, it is proposed that there are other features of the original study which perhaps went unnoticed or were simply not deemed adequately important to note in the write-up of the initial experiment, but which, nonetheless may have partially determined the recorded outcomes.  Perhaps an extraneous variable thought to be negligible or to have been controlled was, in actuality, not controlled and influential. These differences are colloquially referred to as ‘wallpaper effects;’ a tongue-in-cheek way of suggesting that the original experimental situation must have been influenced by the color of the wallpaper in the room. If this were the case, then the original study findings and the replicated study findings may not be in genuine conflict with each other; both may be accurate reflections of reality. The different outcomes, then, would be explained by the effect of some, as of yet, unidentified other variable responsible for causing the difference. Unfortunately, because this subtle influential variable has not been identified, neither finding can be interpreted with any confidence. The interpretations of these studies, then, are best left as incomplete; to be explained at some later time when further light can be shed on the topic.

Find this and other essays regarding “Is the Scientific Method Broken?” in the Nesselroade & Grimm textbook.

Baker. M. (2015, August 27). Over half of psychology studies fail reproducibility test. Nature. Retrieved from www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.182

Tucker, J. (2016, March 9). Does social science have a replication crisis? Washington Post. Retrieved from https://www.washingtonpost.com/news/monkey-cage/wp/2016/03/09/does-social-science-have-a-replication-crisis/?utm_term=.1c5d71247aa0

Wilson, W. A. (2016, May). Scientific regress. First Things. Retrieved from https://www.firstthings.com/ article/2016/05/scientific-regress