In an attempt to ascertain the legitimacy of psychological studies, researchers set out to replicate 100 of them — and found they could reproduce the published results in less than half the time.
As part of The Reproducibility Project: Psychology led by Brian Nosek of the University of Virginia, 270 scientists from around the world attempted to reproduce the results of studies published in 2008 in the world's three top psychology journals.
They were successful in less than half those efforts, as 60 of the chosen studies did not hold up, the study published in the journal Science reported.
The researchers emphasized they found no evidence of fraud, nor could they confirm any of the original study results were patently wrong or false; rather, the researchers said, it indicated the evidence presented in most of the published studies was not as strong as the study authors believed.
"This project is not evidence that anything is broken. Rather, it's an example of science doing what science does," says study co-author Cody Christopherson of Southern Oregon University.
"It's impossible to be wrong in a final sense in science," he explains. "You have to be temporarily wrong, perhaps many times, before you are ever right."
Research is considered reproducible if an independent research team conducting an already-published experiment and closely adhering to the original methods comes up with the same results.
Reproducibility is considered a vital part of the process for gathering evidence to support a proposed theory.
The new study comes amid a sharp rise in the number of papers in highly-regarded journals being retracted when findings could not be reproduced or when fault was found in their design or analysis.
Part of the problem, Nosek says, is that studies that produce unusual or unanticipated findings are more likely to get published than less interesting research examining reproducibility of someone else's findings.
"Scientists aim to contribute reliable knowledge but also need to produce results that help them keep their job as a researcher," he says. "To thrive in science, researchers need to earn publications, and some kinds of results are easier to publish than others, particularly ones that are novel and show unexpected or exciting new directions."
The researchers were quick to emphasize the issue was not limited to the field of psychology, but was also a problem on other fields such as clinical medicine, cell biology, neuroscience, economics and animal research.
