18. Science & Society
text Malini Witlox illustration Bas van der Schot
Tilburg University researchers urge on
sharing data
Scientists of Tilburg University have launched a scientific psychology magazine in which researchers can publish datasets and data papers. The Journal of Open Psychology Data (JOPD) aims to motivate psychologists to share their data, so that other scientists can work with it. Initiator of the Open Access magazine is Jelte Wicherts, a senior lecturer at Tilburg University’s Department of Methodology and Statistics.
S
everal renowned scientists, among whom the American behavioral economist Uri Simonsohn, are on the advisory board (peer reviewers). Simonsohn is known for developing a statistical method to track down suspicious data patterns in scientific articles. Although sharing data became important after former TiU Professor Diederik Stapel’s fraud case, Wicherts has been unhappy with the fact that psychologists weren’t sharing their data for much longer. It caused him a lot of problems during his PhD research at the University of Amsterdam. “The Dutch are scoring higher and higher in IQ-tests. Between 2002 and 2007, I studied the question why Dutch IQs had increased and wanted to see if it was caused by the fact that we’re getting better and better at filling out tests. Thousands of IQ tests must have been filled out. However, I could hardly find anyone who had kept them. It was a total mess, which is a shame because it’s valuable data.”
Shocking results Together with other colleagues, Wicherts set up an investigation in which 141 researchers (Psychology) were asked to provide their data. In accordance with the National Behavioral Code, the scientists had to hand over their data. “The result was shocking. After several reminders, 27 percent of them came up with the data. In 2006, I published an article on this in the leading magazine, American Psychological Association, but people reacted by shrugging their shoulders.” Let’s fast-forward to 2011, the year that Diederik Stapel was revealed as a fraud. A so-called perfect storm gathered within the world of psychology whereby psychologists started to realize that the way they were treating data might not be the best way. Wicherts: “Many arbitrary choices are made when performing research; results are made to look better than that they actually are. And a lot of mistakes are made. But researchers carry on working on the basis of those results which is a complete waste
Univers 30 januari 2014
of money. There are two solutions. First of all, you need to store your data correctly and secondly, you must share your data.”
Breaking with a culture However, in the world of science, researchers often consider their data as their private property. In some cases, it has taken them years to collect all this information and they’re not prepared to share it. With the new magazine, the initiators want to break with this culture. “When an article is published in a paper magazine you’ll often see four pages of text with just a few test results with P-values in a separate text box. This is a shame because you could do so much more. Nowadays, data should make up an integral part of any publication. On paper, you can refer to a database or a website with the data. If the data aren’t made public, it’s impossible to verify that the right analytical choices were made. It’s rather strange that the person who has collected the data, has the final say as to what the data mean. If you request the data because you don’t agree with the analysis, you certainly won’t get it. This is incredibly non-scientific. Just give a clear explanation of the choice you made. NWO and other subsidy providers should take up data-sharing as a precondition to acquiring a subsidy and have the data made public. In my opinion, the NWO isn’t taking a strong stand on this. Making data publicly available makes for better science. As a scientist, I am personally scared stiff of making a mistake in my analysis. I know that by sharing data with other people can see my mistakes so I check everything three times before publishing it.” Wichert’s initiative seems a bit like crying over spilt milk. Since the Stapel case, several psychology departments have straightened things out. Data is no longer destroyed or kept in a box on the loft. The social psychologists at Tilburg University have started to store their data with the use of Dataverse. “However, there is a difference between archiving and opening up to the world. In Dataverse it’s hard to search for anything. And there is no paper in which the scientists state which research methods they have used, which choices were made and which formats were used. We must do this, though. We don’t want to just store data. We give it a kind of quality mark. Are the data interesting, is it properly documented, is it readily available? By the way, the question whether something is interesting is arbitrary. Something might not be interesting in an individual study but could well be as meta-analysis.”