Sökning: id:"swepub:oai:gup.ub.gu.se/273207" >
IRT Models for Expe...
IRT Models for Expert-Coded Panel Data
-
- Marquardt, Kyle L., 1983 (författare)
- Gothenburg University,Göteborgs universitet,Statsvetenskapliga institutionen,Department of Political Science
-
Pemstein, D. (författare)
-
(creator_code:org_t)
- 2018-09-03
- 2018
- Engelska.
-
Ingår i: Political Analysis. - : Cambridge University Press (CUP). - 1047-1987 .- 1476-4989. ; 26:4, s. 431-456
- Relaterad länk:
-
https://gupea.ub.gu....
-
visa fler...
-
https://gup.ub.gu.se...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Data sets quantifying phenomena of social-scientific interest often use multiple experts to code latent concepts. While it remains standard practice to report the average score across experts, experts likely vary in both their expertise and their interpretation of question scales. As a result, the mean may be an inaccurate statistic. Item-response theory (IRT) models provide an intuitive method for taking these forms of expert disagreement into account when aggregating ordinal ratings produced by experts, but they have rarely been applied to cross-national expert-coded panel data. We investigate the utility of IRT models for aggregating expert-coded data by comparing the performance of various IRT models to the standard practice of reporting average expert codes, using both data from the V-Dem data set and ecologically motivated simulated data. We find that IRT approaches outperform simple averages when experts vary in reliability and exhibit differential item functioning (DIF). IRT models are also generally robust even in the absence of simulated DIF or varying expert reliability. Our findings suggest that producers of cross-national data sets should adopt IRT techniques to aggregate expert-coded data measuring latent concepts.
Ämnesord
- SAMHÄLLSVETENSKAP -- Statsvetenskap (hsv//swe)
- SOCIAL SCIENCES -- Political Science (hsv//eng)
Nyckelord
- Bayesian methods
- expert opinion
- latent variables
- IRT models
- cross-national data
- anchoring vignettes
- elections
- reliability
- agreement
- democracy
- quality
- opinion
- Government & Law
Publikations- och innehållstyp
- ref (ämneskategori)
- art (ämneskategori)
Hitta via bibliotek
Till lärosätets databas