This Article Full Text (PDF) Online Appendix All Versions of this Article: 0049124108318973v1 37/1/3    most recent References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Google Scholar Articles by Gerber, A. S. Articles by Malhotra, N. Social Bookmarking                   What's this?

Publication Bias in Empirical Sociological Research

Do Arbitrary Significance Levels Distort Published Results?

Yale University, New Haven, Connecticut

Neil Malhotra

Stanford University, Stanford, California, neilm{at}stanford.edu

Despite great attention to the quality of research methods in individual studies, if publication decisions of journals are a function of the statistical significance of research findings, the published literature as a whole may not produce accurate measures of true effects. This article examines the two most prominent sociology journals (the American Sociological Review and the American Journal of Sociology) and another important though less influential journal (The Sociological Quarterly) for evidence of publication bias. The effect of the .05 significance level on the pattern of published findings is examined using a ``caliper'' test, and the hypothesis of no publication bias can be rejected at approximately the 1 in 10 million level. Findings suggest that some of the results reported in leading sociology journals may be misleading and inaccurate due to publication bias. Some reasons for publication bias and proposed reforms to reduce its impact on research are also discussed.

Key Words: publication bias • caliper test • meta-analysis • hypothesis testing

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?