Sociological Methods & Research recent issues

Is Optimal Matching Suboptimal?

Hollister, M. — Fri, 06 Nov 2009 17:01:42 PST

Optimal matching (OM) is a method for measuring the similarity between pairs of sequences (e.g., work histories). This article discusses two problems with optimal matching. First, the author identifies a flaw in OM ‘‘indel costs’’ and proposes a solution to this flaw. Second, the author discusses the need for benchmarks to measure the added value of OM and to test competing versions. To that end, the author conducts an empirical test of traditional OM, the alternative localized OM, and sequence comparison. The test documents the problem with traditional OM and shows that it is solved by localized OM. The test also demonstrates the value of OM and sequence comparison in examining occupational sequences; both methods capture variation beyond traditional human capital and status attainment measures, although the marginal improvements of OM over sequence comparison may not justify its computational complexity. These results point to the need for more systematic approaches to sequence analysis methods.

A Coefficient of Association Between Categorical Variables With Partial or Tentative Ordering of Categories

Siersma, V., Kreiner, S. — Fri, 06 Nov 2009 17:01:42 PST

Goodman and Kruskal’s coefficient measuring monotone association and its partial variants are useful for the analysis of multiway contingency tables containing ordinal variables. When the categories of a variable are only partly ordered and the variable is treated as a nominal variable, information in the ordering of the categories and statistical power is lost. The authors suggest a ^P measure that is the maximum of the ordinary coefficients obtained by permuting the categories of nominal or partially ordered variables, while leaving the partial ordering intact. When the assumption of a monotone underlying association is justified, this measure has higher power than nominal tests for association. Furthermore, the resulting optimal monotone ordering gives insight into the nature of this association, which is not obtained by tests for nominal variables. The properties of the ^P coefficient are investigated in a simulation study and its use illustrated in two data sets.

Question Order and Interviewer Effects in CATI Scale-up Surveys

Snidero, S., Zobec, F., Berchialla, P., Corradetti, R., Gregori, D. — Fri, 06 Nov 2009 17:01:42 PST

The scale-up estimator is a network-based estimator for the size of hidden or hard to count subpopulations. Several issues arise in the public health context when the aim is the estimation of injuries occurring in a certain population, where two common problems are present: (a) Small injuries are usually difficult to observe and rarely reported in the official data and (b) people are not always compliant in giving information about some specific injuries, in particular when children are involved. This study checked the methodological issues arising from using a computer-assisted telephone interview (CATI) survey using the scale-up methodology for detecting the number of injuries due to choking in children ages 0 to 14 in Italy. For this purpose, 1,000 CATI interviews were conducted during a week using a questionnaire based on 33 questions about populations of known size according to census data. Then, each respondent was asked about other questions related to the main target population (e.g., number of children known to suffer from a choking accident). A sensitivity analysis was conducted for estimating the effect of varying subpopulations, order of the questions, and interviewer effects on the resulting estimates. For the interviewer effect, no particular differences were observed in the overall estimates of injuries. The conclusion is the scale-up estimator in association with CATI methodology shows a high potential in the field of injury prevention, being accurate and robust, but particular attention should be given to the training of the interviewers to improve stability of the estimates.

A Conceptual Framework for Ordered Logistic Regression Models

Fullerton, A. S. — Fri, 06 Nov 2009 17:01:42 PST

Ordinal-level measures are very common in social science research. Researchers often analyze ordinal dependent variables using the proportional odds logistic regression model. However, this ‘‘traditional’’ method is one of many different types of logistic regression models available for the analysis of ordered response variables. In this article, the author identifies 12 distinct models that rely on logistic regression and fit within a framework of three major approaches with variations within each approach based on the application of the proportional odds assumption. This typology provides a degree of conceptual clarity that is missing in the extant literature on logistic regression models for ordinal outcomes. The author illustrates the similarities and differences among the different models with examples from the General Social Survey and the American National Election Study.

Book Review: Alfred DeMaris Regression With Social Data: Modeling Continuous and Limited Response Variables, 2004 64 pp. $135.00. New York: John Wiley. ISBN 0471224855

Firebaugh, G. — Fri, 06 Nov 2009 17:01:42 PST

Book Review: Richard A. Berk Regression Analysis: A Constructive Critique. Advanced Quantitative Techniques in the Social Sciences Series 11. Thousand Oaks, CA: Sage, 2009

Siroky, D. — Fri, 06 Nov 2009 17:01:42 PST

Book Review: Edited by Ross Stolzenberg Sociological Methodology. Washington, DC: Blackwell, 2006. 345 pp. $149.95

Sykes, B. L. — Fri, 06 Nov 2009 17:01:42 PST

Measuring High School Graduation Rates at the State Level: What Difference Does Methodology Make?

Warren, J. R., Halpern-Manners, A. — Wed, 02 Sep 2009 14:06:51 PDT

Recent evidence makes clear that states public high school graduation rates are well measured using information from the Common Core of Data (CCD). This article investigates the substantive consequences for the results of empirical analyses of using different CCD-based measures of states’ public high school graduation rates. The authors show that substantive conclusions about the levels, correlates, and predictors of states’ public high school graduation rates are dependent on how those rates are measured using the CCD data. Warren’s (2005) estimated completion rate is the most conceptually and technically sound CCD-based measure, and that measure is improved in this study. The public high school graduation rate for the class of 2004 was about 76 percent, although that rate varied considerably by race/ethnicity and across states.

A Case for Cases: Comparative Narratives in Sociological Explanation

Abell, P. — Wed, 02 Sep 2009 14:06:51 PDT

When case studies are constructed as narratives, then causal explanation can be achieved without either comparison or generalization. Narratives provide paths of causal links on a chronology of actions or events. The links, in turn, can be studied as Bayesian inferences generating Bayesian narratives. The causal paths in a narrative have a Boolean structure.

Inferring Causal Complexity

Baumgartner, M. — Wed, 02 Sep 2009 14:06:52 PDT

In The Comparative Method, Ragin (1987) outlined a procedure of Boolean causal reasoning operating on pure coincidence data that has since become widely known as qualitative comparative analysis (QCA) among social scientists. QCA—including its recent forms as presented in Ragin (2000, 2008)—is designed to analyze causal structures featuring no more than one effect and a possibly complex configuration of mutually independent direct causes of that effect. This article presents a procedure of causal reasoning that operates on the same type of empirical data as QCA and that implements Boolean techniques related to the ones resorted to by QCA. Yet in contrast to QCA, the procedure introduced here successfully identifies structures involving both multiple effects and mutually dependent causes. In this sense, this article generalizes QCA.

Goodness-of-Fit Tests and Descriptive Measures in Fuzzy-Set Analysis

Eliason, S. R., Stryker, R. — Wed, 02 Sep 2009 14:06:52 PDT

In this article the authors develop goodness-of-fit tests for fuzzy-set analyses to formally assess the fit between empirical information and various causal hypotheses while accounting for measurement error in membership scores. These goodness-of-fit tests, and the accompanying logic, provide a sound inferential foundation for fuzzy-set methodology. The authors also develop descriptive measures to complement these tests. Examples from Stryker and Eliason (2003) and Mahoney (2003) show how goodness-of-fit tests and descriptive measures may be used to assess individual causal factors as well as conjunctions of factors. The authors show how these tools provide more information in a fuzzy-set analysis than do tests currently in use. In providing this inferential foundation, the authors also show that fuzzy-set methods (a) are no less amenable to falsificationist methods of the Neyman-Pearson type than are standard statistical techniques and (b) may be usefully applied in either an exploratory/inductive or a confirmatory/deductive research design.

Robustness of Group-Based Models for Longitudinal Count Data

Weakliem, D. L., Entner Wright, B. R. — Wed, 02 Sep 2009 14:06:52 PDT

In recent years, there have been efforts to develop latent class models for trajectories. The semiparametric mixed Poisson regression (SMPR) model has been used in many empirical studies, but there have been few attempts to evaluate the robustness of the estimates from this model. The authors use simulated data to evaluate the performance of the SMPR model under a variety of assumptions. They find that estimates are sensitive to the conditional distribution of counts and misspecification of the shape of the trajectory. When there is only one underlying trajectory and overdispersion is present, the SMPR model frequently finds multiple groups, which often appear to differ in shape as well as level. The tendency can be substantially reduced by use of the zero-inflated Poisson distribution in conjunction with top-coding of large counts. The article concludes with a discussion of other extensions and alternatives to the standard SMPR model that might provide more robust estimates.

Tracing the Effects of Hurricane Katrina on the Population of New Orleans: The Displaced New Orleans Residents Pilot Study

Sastry, N. — Wed, 02 Sep 2009 14:06:52 PDT

The Displaced New Orleans Residents Pilot Study is designed to examine the current location, well-being, and plans of people who lived in the city of New Orleans when Hurricane Katrina struck on August 29, 2005. The study is based on a representative sample of pre-Katrina dwellings in the city. Respondents were administered a short paper-and-pencil interview by mail, by telephone, or in person. The pilot study was fielded in the fall of 2006, approximately 1 year after Hurricane Katrina. This article describes the motivation for the pilot study, outlines its design, and describes the fieldwork results using a set of fieldwork outcome rates and multivariate logistic models. It ends with a discussion of the lessons learned from the pilot study for future studies of the effects of Hurricane Katrina on the population of New Orleans. The results point to the challenges and opportunities of studying this unique population.

How Much Does It Cost?: Optimization of Costs in Sequence Analysis of Social Science Data

Gauthier, J.-A., Widmer, E. D., Bucher, P., Notredame, C. — Wed, 02 Sep 2009 14:06:52 PDT

One major methodological problem in analysis of sequence data is the determination of costs from which distances between sequences are derived. Although this problem is currently not optimally dealt with in the social sciences, it has some similarity with problems that have been solved in bioinformatics for three decades. In this article, the authors propose an optimization of substitution and deletion/insertion costs based on computational methods. The authors provide an empirical way of determining costs for cases, frequent in the social sciences, in which theory does not clearly promote one cost scheme over another. Using three distinct data sets, the authors tested the distances and cluster solutions produced by the new cost scheme in comparison with solutions based on cost schemes associated with other research strategies. The proposed method performs well compared with other cost-setting strategies, while it alleviates the justification problem of cost schemes.

From Schelling to Spatially Explicit Modeling of Urban Ethnic and Economic Residential Dynamics

Benenson, I., Hatna, E., Or, E. — Fri, 22 May 2009 14:18:14 PDT

The robustness of outcomes to the parameterization of behavioral rules is a crucial property of any model aimed at simulating complex human systems. Schelling model of residential segregation satisfies this criterion. Based on the recently available high-resolution census GIS, we apply Schelling model for investigating urban population patterns at the resolution of individual buildings and families. First, we simulate ethnic residential dynamics in Yaffo (an area of Tel Aviv), and demonstrate good quantitative correspondence for a 40-year period. Second, we investigate income-based residential patterns in nine Israeli cities, reveal their high heterogeneity, and explain the latter by the presence of low fraction of wealthier householders who are tolerant of their poorer neighbors and reside in their proximity. We extend Schelling model in this direction and demonstrate qualitative correspondence between the model's outcomes and the observed income-based residential patterns.

Changing Neighborhoods--Neighborhoods Changing: A Framework for Spatially Explicit Agent-Based Models of Social Systems

O'Sullivan, D. — Fri, 22 May 2009 14:18:14 PDT

The nature of urban neighborhoods, neighborhood effects, and the dynamics of residential segregation are important themes in contemporary sociological inquiry. Agent-based models of social systems have been widely applied in this context. However, both applied and model-based research in these areas suffer from weaknesses in underlying conceptualizations and representations of spatial context. Drawing on human geography and the sociospatial perspective in urban sociology, a framework that enables richer and more realistic representation of urban neighborhoods in agent-based simulation models is proposed and outlined. The framework relies on a graph representation of the spatial relations among spatial locations and can accommodate welldefined administrative zones, vague or ill-defined neighborhoods, hierarchically nested spatial zoning systems, overlapping neighborhoods, and changing relations among neighborhoods. Results from a preliminary application of the framework demonstrate its utility and possibilities for research into the effects of neighborhood structure on social processes.

Using Heterogeneous Choice Models to Compare Logit and Probit Coefficients Across Groups

Williams, R. — Fri, 22 May 2009 14:18:14 PDT

Allison (1999) notes that comparisons of logit and probit coefficients across groups can be invalid and misleading, proposes a procedure by which these problems can be corrected, and argues that ``routine use [of this method] seems advisable'' and that ``it is hard to see how [the method] can be improved.'' In this article, the author argues that as originally proposed, Allison's method can have serious problems and should not be applied on a routine basis. However, this study also shows that his model belongs to a larger class of models variously known as heterogeneous choice or location-scale models. Several advantages of this broader and more flexible class of models are illustrated. Dependent variables can be ordinal in addition to binary, sources of heterogeneity can be better modeled and controlled for, and insights can be gained into the effects of group characteristics on outcomes that would be missed by other methods.

Critiquing Models of Emotions

Smith, H., Schneider, A. — Fri, 22 May 2009 14:18:14 PDT

This article provides the first systematic empirical examination of four major genres of theories concerning the nature and rise of the corpus of human emotions with more than 2,000 statistical tests of five hypotheses. The distinction between evolutionary-universal and other ``secondary'' emotions is empirically uninformative for all five cultures. Next, the emotion-wheel theory of Plutchik receives no empirical support. All palette theories fail four empirical tests. More than 90 empirical tests fail to support Kemper and Turner in assuming that many secondary emotions arise through complex combinations of primary emotions due to socialization. The Johnson-Laird and Oatley hypothesis of five universal clusters of emotions is also tested and rejected. Researchers need to rethink the heuristic value of dichotomizing and lumping emotions in categories such as universal, primary, basic, secondary, tertiary, and so forth. There are clear empirical advantages to differentiating between emotions with three dimensions rather than two dimensions.

Name-Based Cluster Sampling

Ferguson, D. A. — Fri, 22 May 2009 14:18:14 PDT

An innovative method is proposed for generating valid national samples of online e-mail addresses that are proportionate to the population. Although multistage cluster sampling is not new, this particular application using first names and/or surnames has not been previously published or presented. This article outlines the method in detail and presents a number of advantages over other forms of samples. Limitations are also discussed.

A Reply to Zax's (2002) Critique of Grofman and Migalski (1988): Double-Equation Approaches to Ecological Inference When the Independent Variable Is Misspecified

Grofman, B., Barreto, M. A. — Fri, 22 May 2009 14:18:14 PDT

The authors reply to Zax's critique of the double-equation method for ecological regression and of the specific extension to it proposed by Grofman and Migalski. Although Zax does correct two minor errors in Grofman and Migalski's statement of the double-equation approach, neither of those errors affected the final calculations reported in their article. Furthermore, nothing Zax reports affects their fundamental conclusion that double-equation methods can be superior to single-equation techniques if there is substantial error in the measurement of the independent variable. In particular, by analyzing an election for which, from exit polls, the ``true'' parameters of Hispanic and non-Hispanic levels of political cohesion are known, the authors show that double-equation ecological regression estimates derived from registration data are highly accurate in reproducing the true individual-level behavioral parameters (group means).

Introduction to the Special Issue on Web Surveys

Witte, J. C. — Tue, 17 Feb 2009 08:44:02 PST

Selection Bias in Web Surveys and the Use of Propensity Scores

Schonlau, M., van Soest, A., Kapteyn, A., Couper, M. — Tue, 17 Feb 2009 08:44:02 PST

Web surveys are a popular survey mode, but the subpopulation with Internet access may not represent the population of interest. The authors investigate whether adjusting using weights or matching on a small set of variables makes the distributions of target variables representative of the population. This application has a rich sampling design; the Internet sample is part of an existing probability sample, the Health and Retirement Study, that is representative of the U.S. population aged 50 and older. For the dichotomous variables investigated, the adjustment helps. On average, the sample means in the Internet access sample differ by 6.5 percent before and 3.7 percent after adjustment. Still, a large number of adjusted estimates remain significantly different from their target estimates based on the complete sample. This casts doubt on the common procedure to use only a few variables to correct for the selectivity of convenience samples.

Estimation for Volunteer Panel Web Surveys Using Propensity Score Adjustment and Calibration Adjustment

Lee, S., Valliant, R. — Tue, 17 Feb 2009 08:44:02 PST

A combination of propensity score and calibration adjustment is shown to reduce bias in volunteer panel Web surveys. In this combination, the design weights are adjusted by propensity scores to correct for selection bias due to nonrandomized sampling. These adjusted weights are then calibrated to control totals for the target population and correct for coverage bias. The final set of weights is comprised of multiple components, and the estimator of a total no longer takes a linear form. Therefore, approximate methods are needed to derive variance estimates. This study compares three variance estimation methods through simulation. The first method resembles what is used in commercial statistical software based on squared residuals. The second approach uses a variance estimator originally derived for the generalized regression estimator. The third method uses jackknife replication. Results indicate bias reduction is crucial for valid variance estimation and favor the replication method over the other approaches.

Web Survey Design: Balancing Measurement, Response, and Topical Interest

Shropshire, K. O., Hawdon, J. E., Witte, J. C. — Tue, 17 Feb 2009 08:44:02 PST

Using data from Survey2001, we analyze how visual images embedded in a web-based survey can (1) reduce nonresponse in the specific case that a respondent prematurely terminates the survey and (2) preserve measurement validity. Page-by-page progression through the survey is modeled as a survival process with early termination seen as failure. While images had no apparent effect on the termination process, respondent interest in the survey topic was linked to early termination. These results highlight the importance of placing interest-related questions early in the survey to better control for the effects of interest-driven attrition. Moreover, these findings suggest that an interactional information system approach, one that not only collects data but also pushes relevant information to respondents, may serve to generate or maintain interest and in the process reduce survey attrition.

Design of Web Questionnaires: An Information-Processing Perspective for the Effect of Response Categories

Toepoel, V., Vis, C., Das, M., van Soest, A. — Tue, 17 Feb 2009 08:44:02 PST

In this article, an information-processing perspective is used to explore the impact of response categories on the answers respondents provide in Web surveys. Response categories have a significant effect on response formulation in questions that are difficult to process, whereas in easier questions (where responses are based on direct recall) the response scales have a smaller effect. In general, people with less cognitive sophistication are more affected by contextual cues. The Need for Cognition and the Need to Evaluate indexes for motivation account for a significant part of the variance in survey responding. Interactions of ability to process information and motivation combine in regulating responses for questions that are more difficult to process. The results hint at a substantial role of satisficing in Web surveys.

Designing Scalar Questions for Web Surveys

Christian, L. M., Parsons, N. L., Dillman, D. A. — Tue, 17 Feb 2009 08:44:02 PST

This paper explores how the visual design of scalar questions influences responses in web surveys. We present the results of five experiments embedded in two web surveys of university students. We find that consistently presenting the positive end of the scale first did not impact responses but increases response times. Displaying the categories in multiple columns influence how respondents process the scale and increase response times. Separating the midpoint, ``don't know'' option, or endpoints spatially does not impact responses when the visual and conceptual midpoint align. Removing the graphical layout of the scale influences responses when lower numbers indicate more positive categories and increases response time. Finally, response times are longer for polar point scales with numeric labels, but there are no differences in responses. Overall, our results suggest that the visual design of response scales impacts measurement, but that some manipulations produce larger and more significant differences than others.

Smartphones: An Emerging Tool for Social Scientists

Raento, M., Oulasvirta, A., Eagle, N. — Tue, 17 Feb 2009 08:44:02 PST

Recent developments in mobile technologies have produced a new kind of device: a programmable mobile phone, the smartphone. In this article, the authors argue that the technological and social characteristics of this device make it a useful tool in social sciences, particularly sociology, social psychology, urban studies, technology assessment, and media studies. The device is willingly carried by a large fraction of people in developed countries, integrates a number of technologies for automatic observation, can be programmed to interact with the user, and can communicate with remote researchers. This allows unobtrusive and cost-effective access to previously inaccessible sources of data on everyday social behavior, such as physical proximity of people, phone calls, and patterns of movement. The authors describe three studies in human behavior that have augmented existing methods with the smartphone, two of which the authors conducted themselves. Based on their experience, the authors critically evaluate the improvements and threats to validity and reliability of smartphone-augmented methods. These approaches are rapidly becoming feasible for the social scientist, since research software for smartphones have been published in open source, which lowers the technical and economic investment needed for their utilization in research.

Book Review: M. Smithson and J. Verkuilen (2006). Fuzzy Set Theory: Applications in the Social Sciences. Thousand Oaks, CA: Sage

Vaisey, S. — Tue, 17 Feb 2009 08:44:02 PST