There is a distinction (not always observed by various authors) between Replication and reproducibility.
- Replication is re-running studies to confirm results. This means, collect own data, and get the same effect for your study.
- Reproducibility is the the ability to repeat an analyses on data; follow all the steps, settings and procedures set up by the original study, and see if you get the same results.
Replication can tell us about how robust and generalizable the findings are. There is a long tradition in replicating studies in differnet country / culture, to see if the same effect holds, in the new context.
Reproducibilty can tell us about the quality of validity of the original research. If one is unable to reproduce, there is a flaw in the original study, methods may not have been applied correctly, the data may have been “cleaned”, or statistical assumptions overlooked.
To learn more, see this video:
More on the replication crisis:
Estimating the reproducibility of psychological science
One of the central goals in any scientific endeavor is to understand causality. Experiments that seek to demonstrate a cause/effect relation most often manipulate the postulated causal factor. Aarts et al. describe the replication of 100 experiments reported in papers published in 2008 in three high-ranking psychology journals.
Response and critique to the study
Comment on “Estimating the reproducibility of psychological science”
A paper from the Open Science Collaboration (Research Articles, 28 August 2015, aac4716) attempting to replicate 100 published studies suggests that the reproducibility of psychological science is surprisingly low. We show that this article contains three statistical errors and provides no support for such a conclusion.
Response to the response