| 1. |
- Bader, Felix, et al.
(author)
-
On the Transportability of Laboratory Results
- 2021
-
In: Sociological Methods & Research. - : Sage Publications. - 0049-1241 .- 1552-8294. ; 50:3, s. 1452-1481
-
Journal article (peer-reviewed)abstract
- The “transportability” of laboratory findings to other instances than the original implementation entails the robustness of rates of observed behaviors and estimated treatment effects to changes in the specific research setting and in the sample under study. In four studies based on incentivized games of fairness, trust, and reciprocity, we evaluate (1) the sensitivity of laboratory results to locally recruited student-subject pools, (2) the comparability of behavioral data collected online and, under varying anonymity conditions, in the laboratory, (3) the generalizability of student-based results to the broader population, and (4), with a replication at Amazon Mechanical Turk, the stability of laboratory results across research contexts. For the class of laboratory designs using interactive games as measurement instruments of prosocial behavior we find that rates of behavior and the exact behavioral differences between decision situations do not transport beyond specific implementations. Most clearly, data obtained from standard participant pools differ significantly from those from the broader population. This undermines the use of empirically motivated laboratory studies to establish descriptive parameters of human behavior. Directions of the behavioral differences between games, in contrast, are remarkably robust to changes in samples and settings. Moreover, we find no evidence for either anonymity effects nor mode effects potentially biasing laboratory measurement. These results underscore the capacity of laboratory experiments to establish generalizable causal effects in theory-driven designs.
|
|
| 2. |
- Barban, Nicola, et al.
(author)
-
Causal Effects of the Timing of Life-course Events : Age at Retirement and Subsequent Health
- 2020
-
In: Sociological Methods & Research. - : Sage Publications. - 0049-1241 .- 1552-8294. ; 49:1, s. 216-249
-
Journal article (peer-reviewed)abstract
- In this article, we combine the extensive literature on the analysis of life-course trajectories as sequences with the literature on causal inference and propose a new matching approach to investigate the causal effect of the timing of life-course events on subsequent outcomes. Our matching approach takes into account pre-event confounders that are both time-independent and time-dependent as well as life-course trajectories. After matching, treated and control individuals can be compared using standard statistical tests or regression models. We apply our approach to the study of the consequences of the age at retirement on subsequent health outcomes, using a unique data set from Swedish administrative registers. Once selectivity in the timing of retirement is taken into account, effects on hospitalization are small, while early retirement has negative effects on survival. Our approach also allows for heterogeneous treatment effects. We show that the effects of early retirement differ according to preretirement income, with higher income individuals tending to benefit from early retirement, while the opposite is true for individuals with lower income.
|
|
| 3. |
- Do, Salomé, et al.
(author)
-
The Augmented Social Scientist. : Using Sequential Transfer Learning to Annotate Millions of Texts with Human-Level Accuracy
- 2022
-
In: Sociological Methods and Research. - New York : Sage Publications. - 0049-1241 .- 1552-8294.
-
Journal article (peer-reviewed)abstract
- The last decade witnessed a spectacular rise in the volume of available textual data. With this new abundance came the question of how to analyze it. In the social sciences, scholars mostly resorted to two well-established approaches, human annotation on sampled data on the one hand (either performed by the researcher, or outsourced to microworkers), and quantitative methods on the other. Each approach has its own merits – a potentially very fine-grained analysis for the former, a very scalable one for the latter – but the combination of these two properties has not yielded highly accurate results so far. Leveraging recent advances in sequential transfer learning, we demonstrate via an experiment that an expert can train a precise, efficient automatic classifier in a very limited amount of time. We also show that, under certain conditions, expert-trained models produce better annotations than humans themselves. We demonstrate these points using a classic research question in the sociology of journalism, the rise of a “horse race” coverage of politics. We conclude that recent advances in transfer learning help us augment ourselves when analyzing unstructured data.
|
|
| 4. |
- Engzell, Per
(author)
-
What Do Books in the Home Proxy For? A Cautionary Tale
- 2021
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 50:4, s. 1487-1514
-
Journal article (peer-reviewed)abstract
- In studies of educational achievement, students' self-reported number of books in the family home is a frequently used proxy for social, cultural, and economic background. Absent hard evidence about what this variable captures or how well, its use has been motivated by strong associations with student outcomes. I show that these associations rest on two types of endogeneity: Low achievers accrue fewer books and are also prone to underestimate their number. The conclusion is substantiated both by comparing reports by students and their parents and by the fact that girls report on average higher numbers despite being similar to boys on other measures of social background. The endogenous bias is large enough to overturn classical attenuation bias; it distorts cross-country patterns and invalidates many common study designs. These findings serve as a caution against overreliance on standard regression assumptions and contribute to ongoing debates about the empirical robustness of social science.
|
|
| 5. |
|
|
| 6. |
- Manzo, Gianluca, et al.
(author)
-
Heuristics, Interactions, and Status Hierarchies : An Agent-based Model of Deference Exchange
- 2015
-
In: Sociological Methods & Research. - : Sage Publications. - 0049-1241 .- 1552-8294. ; 44:2, s. 329-387
-
Journal article (peer-reviewed)abstract
- Since Merton’s classical analysis of cumulative advantage in science, it has been observed that status hierarchies display a sizable disconnect between actors’ quality and rank and that they become increasingly asymmetric over time, without, however, turning into winner-take-all structures. In recent years, formal models of status hierarchies tried to account for these facts by combining two micro-level, counterbalancing mechanisms: “social influence” (supposedly driving inequality) and the desire for “reciprocation in deferential gestures” (supposedly limiting inequality). In the article, we adopt as empirical benchmark basic features that are common to most distributions of status indicators (e.g., income, academic prestige, wealth, social ties) and argue that previous formal models were only partially able to reproduce such macro-level patterns. We then introduce a novel agent-based computational model of deferential gestures that improves on the realism of previous models by introducing heuristic-based decision making, actors’ heterogeneity, and status homophily in social interactions. We systematically and extensively study the model’s parameter space and consider a few variants to determine under which conditions the macroscopic patterns of interest are more likely to appear. We find that specific forms of status-based heterogeneity in actors’ propensity to interact with status-dissimilar others are needed to generate status hierarchies that best approximate these macroscopic features.
|
|
| 7. |
- Markovsky, Barry, et al.
(author)
-
Comparing Direct and Indirect Measures of Just Rewards
- 2012
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 41:1, s. 199-216
-
Journal article (peer-reviewed)abstract
- We offer the first comparison between "direct" and "indirect" methods for measuring perceptions of distributive justice in reward allocations. The direct method simply asks respondents what they would consider to be a fair salary for a particular person in a given set of circumstances. In contrast, the indirect method infers fair salaries from respondents' judgments about the relative unfairness of hypothetical salaries. The particular indirect method that we will assess is a vignette survey technique pioneered by Jasso and Rossi (1977) and used in a number of more recent publications. The vignettes describe characteristics of a hypothetical employee, with the objective of deriving what respondents believe to be the just reward for that employee. Our experimental test suggests that the two methods yield incompatible results and that neither is immune to bias. The indirect method also suffers from a type of specification error that leads to untenable results. We conclude by suggesting directions for new research to gain a better understanding of these problems and, ultimately, to circumvent them.
|
|
| 8. |
|
|
| 9. |
- Munck, Ingrid, et al.
(author)
-
Measurement Invariance in Comparing Attitudes Toward Immigrants Among Youth Across Europe in 1999 and 2009: The Alignment Method Applied to IEA CIVED and ICCS
- 2018
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 47:4, s. 687-728
-
Journal article (peer-reviewed)abstract
- This study applies the alignment method, a technique for assessing measurement equivalence across many groups, to the analysis of adolescents' support for immigrants' rights in a pooled data set from the 1999 International Association for the Evaluation of Educational Achievement (IEA) Civic Education Study and the 2009 IEA International Civics and Citizenship Education Study. We examined measurement invariance across 92 groups (country by cohort by gender), finding that a five-item scale was statistically well-grounded for unbiased group comparisons despite the presence of significant noninvariance in some groups. Using the resulting group mean scores, we compared European youth's attitudes finding that female students had more positive attitudes than did male students across countries and cohorts. An analysis of countries participating in both studies revealed that students in most countries demonstrated more positive attitudes in 2009 than in 1999. The alignment methodology makes it feasible to comprehensively assess measurement invariance in large data sets and to compute aligned factor scores for the full sample that can update existing databases for more efficient further secondary analysis and with metainformation concerning measurement invariance.
|
|
| 10. |
- Spaiser, Viktoria, et al.
(author)
-
Identifying Complex Dynamics in Social Systems : A New Methodological Approach Applied to Study School Segregation
- 2018
-
In: Sociological Methods & Research. - : SAGE PUBLICATIONS INC. - 0049-1241 .- 1552-8294. ; 47:2, s. 103-135
-
Journal article (peer-reviewed)abstract
- It is widely recognized that segregation processes are often the result of complex nonlinear dynamics. Empirical analyses of complex dynamics are however rare, because there is a lack of appropriate empirical modeling techniques that are capable of capturing complex patterns and nonlinearities. At the same time, we know that many social phenomena display nonlinearities. In this article, we introduce a new modeling tool in order to partly fill this void in the literature. Using data of all secondary schools in Stockholm county during the years 1990 to 2002, we demonstrate how the methodology can be applied to identify complex dynamic patterns like tipping points and multiple phase transitions with respect to segregation. We establish critical thresholds in schools' ethnic compositions, in general, and in relation to various factors such as school quality and parents' income, at which the schools are likely to tip and become increasingly segregated.
|
|
| 11. |
- Stadtfeld, Christoph, et al.
(author)
-
Statistical Power in Longitudinal Network Studies
- 2020
-
In: Sociological Methods & Research. - : Sage Publications. - 0049-1241 .- 1552-8294. ; 49:4, s. 1103-1132
-
Journal article (peer-reviewed)abstract
- Longitudinal social network studies can easily suffer from insufficient statistical power. Studies that simultaneously investigate change of network ties and change of nodal attributes (selection and influence studies) are particularly at risk because the number of nodal observations is typically much lower than the number of observed tie variables. This article presents a simulation-based procedure to evaluate statistical power of longitudinal social network studies in which stochastic actor-oriented models are to be applied. Two detailed case studies illustrate how statistical power is strongly affected by network size, number of data collection waves, effect sizes, missing data, and participant turnover. These issues should thus be explored in the design phase of longitudinal social network studies.
|
|
| 12. |
- Stark, T. H., et al.
(author)
-
Generalization of Classic Question Order Effects Across Cultures
- 2020
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 49:3, s. 567-602
-
Journal article (peer-reviewed)abstract
- Questionnaire design is routinely guided by classic experiments on question form, wording, and context conducted decades ago. This article explores whether two question order effects (one due to the norm of evenhandedness and the other due to subtraction or perceptual contrast) appear in surveys of probability samples in the United States and 11 other countries (Canada, Denmark, Germany, Iceland, Japan, the Netherlands, Norway, Portugal, Sweden, Taiwan, and the United Kingdom;N= 25,640). Advancing theory of question order effects, we propose necessary conditions for each effect to occur, and found that the effects occurred in the nations where these necessary conditions were met. Surprisingly, the abortion question order effect even appeared in some countries in which the necessary condition was not met, suggesting that the question order effect there (and perhaps elsewhere) was not due to subtraction or perceptual contrast. The question order effects were not moderated by education. The strength of the effect due to the norm of evenhandedness was correlated with various cultural characteristics of the nations. Strong support was observed for the form-resistant correlation hypothesis.
|
|
| 13. |
- Voyer, Andrea, et al.
(author)
-
From Strange to Normal : Computational Approaches to Examining Immigrant Incorporation Through Shifts in the Mainstream
- 2022
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 51:4, s. 1540-1579
-
Journal article (peer-reviewed)abstract
- This article presents a computational approach to examining immigrant incorporation through shifts in the social “mainstream.” Analyzing a historical corpus of American etiquette books, texts from 1922–2017 describing social norms, we identify mainstream shifts related to long-standing groups which once were and may currently still be seen as immigrant outsiders in the United States: Catholic, Chinese, Irish, Italian, Jewish, Mexican, and Muslim groups. The analysis takes a computational grounded theory approach, combining qualitative readings and computational text analyses. Using word embeddings, we operationalize the chosen groups as focal group concepts. We extract sections of text that are salient to the focal group concepts to create group-specific text corpora. Two computational approaches make it possible to examine mainstream shifts in these corpora. First, we use sentiment analysis to observe the positive sentiment in each corpus and its change over time. Second, we observe changes in each corpus's position on a semantic dimension represented by the poles of “strange” and “normal.” The results indicate mainstream shifts through increases in positive sentiment and movement from strange to normal over time for most of the group-specific corpora. These research techniques can be adapted to other studies of social sentiment and symbolic inclusion.
|
|
| 14. |
- Yuan, Ke-Hai, et al.
(author)
-
ML Versus MI for Missing Data With Violation of Distribution Conditions
- 2012
-
In: Sociological Methods & Research. - : SAGE Publications. - 0049-1241 .- 1552-8294. ; 41:4, s. 598-629
-
Journal article (peer-reviewed)abstract
- Normal-distribution-based maximum likelihood (ML) and multiple imputation (MI) are the two major procedures for missing data analysis. This article compares the two procedures with respects to bias and efficiency of parameter estimates. It also compares formula-based standard errors (SEs) for each procedure against the corresponding empirical SEs. The results indicate that parameter estimates by MI tend to be less efficient than those by ML; and the estimates of variance -covariance parameters by MI are also more biased. In particular, when the population for the observed variables possesses heavy tails, estimates of variance -covariance parameters by MI may contain severe bias even at relative large sample sizes. Although performing a lot better, ML parameter estimates may also contain substantial bias at smaller sample sizes. The results also indicate that, when the underlying population is close to normally distributed, SEs based on the sandwich-type covariance matrix and those based on the observed information matrix are very comparable to empirical SEs with either ML or MI. When the underlying distribution has heavier tails, SEs based on the sandwich-type covariance matrix for ML estimates are more reliable than those based on the observed information matrix. Both empirical results and analysis show that neither SEs based on the observed information matrix nor those based on the sandwich-type covariance matrix can provide consistent SEs in MI. Thus, ML is preferable to MI in practice, although parameter estimates by MI might still be consistent.
|
|
| 15. |
|
|
