Sociological Methods & Research

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Click here for more information

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (2)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Freese, J.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Sociological Methods & Research, Vol. 36, No. 2, 153-172 (2007)
DOI: 10.1177/0049124107306659

Replication Standards for Quantitative Social Science

Why Not Sociology?

Jeremy Freese

Northwestern University, Evanston, Illinois, jfreese{at}northwestern.edu

The credibility of quantitative social science benefits from policies that increase confidence that results reported by one researcher can be verified by others. Concerns about replicability have increased as the scale and sophistication of analyses increase the possible dependence of results on subtle analytic decisions and decrease the extent to which published articles contain full descriptions of methods. The author argues that sociology should adopt standards regarding replication that minimize its conceptualization as an ethical and individualistic matter and advocates for a policy in which authors use independent online archives to deposit the maximum possible information for replicating published results at the time of publication and are explicit about the conditions of availability for any necessary materials that are not provided. The author responds to several objections that might be raised to increasing the transparency of quantitative sociology in this way and offers a candidate replication policy for sociology.

Key Words: replication • data sharing • data archiving • transparency

References

  • Altman, Micah and Gary King. 2006. ``A Proposed Standard for the Scholarly Citation of Quantitative Data'' Institute for Qualitative Social Science, Harvard University, Cambridge, MA. Unpublished manuscript. http://gking.harvard.edu/projects/repl.shtml
  • American Psychological Association. 2001. Publication Manual of the American Psychological Association. 5th ed. Washington, DC: American Psychological Association.
  • American Sociological Association. 1997. American Sociological Association Style Guide. 2nd ed. Washington, DC: American Sociological Association.
  • ———. 1999. Code of Ethics and Policies and Procedures of the ASA Committee on Professional Ethics. Washington, DC: ASA. http://www.asanet.org/galleries/default-file/Code%20of%20Ethics.pdf
  • Banks, James, Michael Marmont, Zoe Oldfield, and James P. Smith. 2006. ``Disease and Disadvantage in the United States and in England.'' Journal of the American Medical Association 295:2037-45.[CrossRef]
  • Bernanke, Ben S. 2004. ``Editorial Statement.'' American Economic Review 94:404.[ISI]
  • Burawoy, Michael. 2003. ``Revisits: An Outline of a Theory of Reflexive Ethnography.'' American Sociological Review 68:645-79.[CrossRef]
  • Dewald, William G., Jerry G. Thursby, and Richard G. Anderson. 1986. ``Replication in Empirical Economics: The Journal of Money, Credit, and Banking Project '' 76:587-603.
  • Freese, Jeremy and Brian Powell. 2001. ``Making Love Out of Nothing at All? Null Findings and the Trivers-Willard Hypothesis'' American Journal of Sociology 106:1776-88.[CrossRef]
  • Gerber, Alan and Neil Malhotra. 2006. ``Can Political Science Literatures Be Believed? A Study of Publication Bias in the APSR and the AJPS.'' Working Paper. Society for Political Methodology, St. Louis, MO.
  • Gleditsch, Nils Petter, Patrick James, James Lee Ray, and Bruce Russett. 2003. ``Editors' Joint Statement: Minimum Replication Standards for International Relations Journals.'' International Studies Perspectives 4:105.
  • Gleditsch, Nils Petter and Claire Metelits. 2003. ``The Replication Debate.'' International Studies Perspectives 4:72-79.[CrossRef]
  • Gleditsch, Nils Petter, Claire Metelits, and Håvard Strand. 2003. ``Posting Your Data: Will You Be Scooped or Will You Be Famous?'' International Studies Perspectives 4:89-95.
  • Harford, Tim. 2005. The Undercover Economist: Exposing Why the Rich Are Rich, the Poor Are Poor, and You Can Never Buy a Decent Used Car. Oxford, UK: Oxford University Press.
  • Hauser, Robert M. 1987. ``Sharing Data: It's Time for ASA Journals to Follow the Folkways of a Scientific Sociology.'' American Sociological Review 52 (December): vi-viii.
  • Ho, Daniel E., Kosuke Imai, Gary King, and Elizabeth Stuart. Forthcoming. ``Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference'' Political Analysis.
  • Jann, Ben. 2005. ``Earning Returns to Education in Urban China: A Note on Testing Differences Among Groups.'' American Sociological Review 70:860-64.[ISI]
  • Johnson, Gary R. 2000. ``Editorial: Science, Sulloway, and Birth Order: An Ordeal and an Assessment.'' Politics and the Life Sciences 19:221-45.
  • Journal of Human Resources. 1989. JHR Policy on Replication and Data Availability. http://www.ssc.wisc.edu/jhr/replication.html
  • Kanazawa, Satoshi. 2001. ``Why We Love Our Children.'' American Journal of Sociology 106:1761-75.[CrossRef]
  • King, Gary. 1995a. ``Replication, Replication.'' PS: Political Science and Politics 28:444-52.[CrossRef]
  • ———. 1995b. ``A Revised Proposal, Proposal.'' PS: Political Science and Politics 28: 494-99.[CrossRef]
  • ———. 2003. ``The Future of Replication.'' International Studies Perspectives 4:100-105.
  • ———. 2006. ``Publication, Publication.'' PS: Political Science and Politics 39:119-25.[CrossRef]
  • ———, James Honaker, Anne Joseph, and Kenneth Scheve. 2001. ``Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation.'' American Political Science Review 95:49-69.[ISI]
  • Leahey, Erin. 2006. ``Overseeing Scientific Research: The Case of Data Editing '' University of Arizona. Unpublished manuscript.
  • Leahey, Erin, Barbara Entwisle, and Peter Einaudi. 2003. ``Diversity in Everyday Research Practice: The Case of Data Editing.'' Sociological Methods & Research 32:63-89.
  • Levitt, Steven and Stephen J. Dubner. 2005. Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. New York: HarperCollins.
  • Long, J. Scott and Jeremy Freese. 2006. Regression Analysis for Categorical Dependent Variables Using Stata. 2nd ed. College Station, TX: Stata Press.
  • McCullough, B.D., Kerry Anne McGeary, and Teresa D. Harrison. 2006. ``Lessons from the JMCB archive '' Journal of Money, Credit, and Banking 38:1093-107.[CrossRef][ISI]
  • McCullough, B.D. and H.D. Vinod. 2003. ``Verifying the Solution From a Nonlinear Solver.'' American Economic Review 93:873-92.[CrossRef][ISI]
  • Miller, Alan S. and Rodney Stark. 2002. ``Gender and Religiousness: Can Socialization Explanations Be Saved?'' American Journal of Sociology 107:1399-423.[CrossRef]
  • National Institutes of Health. 2003. Final Statement on Sharing Research Data. http://grants.nih.gov/grants/guide/notice-files/NOT-OD-03-032.html
  • Peng, Roger D., Francesca Dominici, and Scott L. Zeger. 2006. ``Reproducible Epidemiologic Research.'' American Journal of Epidemiology 163:783-89.[Abstract/Free Full Text]
  • Risman, Barbara J., Shannon N. Davis, and Catherine Zimmer. 2005. ``Biological Constraints on Gender? Feminists Rest With Testosterone.'' Presented at the annual meeting of the American Sociological Association, Philadelphia, PA.
  • Roth, Louise Marie and Jeffrey C. Kroll. 2006. ``Risky Business: Assessing Risk-Preference Explanations for Gender Differences in Religiosity'' University of Arizona. Unpublished manuscript.
  • Stone, Walter J. 1995. ``Reflection, Reflection: A Comment and Modest Proposal.'' PS: Political Science and Politics 28:459-60.[CrossRef]
  • Udry, J. Richard. 2000. ``Biological Limits of Gender Construction.'' American Sociological Review 65:443-57.[CrossRef][ISI]
  • Wicherts, Jelte M., Borsboom, Denny Kats, Judith, Molenaar, Dylanand 2006. ``The Poor Availability of Psychological Research Data for Reanalysis.'' American Psychologist 61:726-28.[CrossRef][Medline] [Order article via Infotrieve]
  • Wu, Xiaogang and Xie Yu. 2003. ``Does the Market Pay Off? Earnings Returns to Education in Urban China.'' American Sociological Review 68:425-42.[CrossRef][ISI]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?


This article has been cited by other articles:


Home page
 Sociological Methods ResearchHome page
G. Firebaugh
Replication Data Sets and Favored-Hypothesis Bias: Comment on Jeremy Freese (2007) and Gary King (2007)
Sociological Methods Research, November 1, 2007; 36(2): 200 - 209.
[Abstract] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (2)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Freese, J.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?