| 1. |
|
|
| 2. |
- Grossmann, Igor, et al.
(författare)
-
Insights into the accuracy of social scientists' forecasts of societal change
- 2023
-
Ingår i: Nature Human Behaviour. - : Springer Nature. - 2397-3374. ; 7, s. 484-501
-
Tidskriftsartikel (refereegranskat)abstract
- How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data. How accurate are social scientists in predicting societal change, and what processes underlie their predictions? Grossmann et al. report the findings of two forecasting tournaments. Social scientists' forecasts were on average no more accurate than those of simple statistical models.
|
|
| 3. |
- Lundmark, Sebastian, 1986, et al.
(författare)
-
The need for public opinion and survey methodology research to embrace preregistration and replication, exemplified by a team's failure to replicate their own findings on visual cues in grid-type questions
- 2023
-
Ingår i: International journal of public opinion research. - : Oxford University Press (OUP). - 0954-2892 .- 1471-6909. ; 35:1
-
Tidskriftsartikel (refereegranskat)abstract
- Survey researchers take great care to measure respondents' answers in an unbiased way; but, how successful are we as a field at remedying unintended and intended biases in our research? The validity of inferences drawn from studies has been found to be improved by the implementation of preregistration practices. Despite this, only 3 of the 83 published articles in POQ and IJPOR in 2020 feature explicitly stated preregistered hypotheses or analyses. This manuscript aims to show survey methodologists how preregistration and replication (where possible) are in service to the broader mission of survey methodology. To that end, we present a practical example of how unknown biases in analysis strategies without preregistration or replication inflate type I errors. In an initial data collection, our analysis showed that the visual layout of battery-type questions significantly decreased data quality. But after committing to replicating and preregistering the hypotheses and analysis plans, none of the results replicated successfully, despite keeping the procedure, sample provider, and analyses identical. This manuscript illustrates how preregistration and replication practices might, in the long term, likely help unburden the academic literature from follow-up publications relying on type I errors.
|
|
