Reproducibility and Generalization: Preliminary Results from A Large Research Project

Topic:

Reproducibility and Generalization: Preliminary Results from A Large Research Project

Time&Date: 

  15:00-16:15 pm, 2019/9/26 (Thursday)

Venue:

  Room 619, Teaching A

Speaker:

Prof. Andrew Delios (National University of Singapore)

Over the past 5 years, there has been increasing concern about the quality of empirical methods and the reliability of empirical results from experimental research. This is the so-called crisis in science. Although the thrust of the work to date has concerned the reproducibility and generalizability of research founded on primary data, we believe it is also important to understand the limitations, constraints and concerns that accompany research conducted using secondary data. Although the supposed empirical authenticity of secondary data is greater because of public access to the source data for studies conducted using such data, we seek to identify and explore limits to reproducibility and generalization through a large research project that examines two decades of research conducted using one archival data base as its main source of data. We identified more than 100 studies that have used the same source data and conduct dual researcher reproducibility and generalization tests on 30 of these studies. In total we complete 120 empirical tests, from which we have summary observations on reproducibility and generalization. In this seminar, I report on these preliminary findings, in a discussion of the current challenges to our existing empirical research standards. We will also have a discussion of emergent new standards in quantitative research, which are fundamental to not only the progression of our field, but also to the ability and likelihood of publishing research in this new era of open source publishing, open data access and responsible science initiatives.