Misinterpretations of P-values and statistical tests persist among researchers and professionals working with statistics and epidemiology

• Background: The aim was to investigate inferences of statistically significant test results among persons with more or less statistical education and research experience.Methods: A total of 75 doctoral students and 64 statisticians/epidemiologist responded to a web questionnaire about inferences of statistically significant findings. Participants were asked about their education and research experience, and also whether a 'statistically significant' test result (P = 0.024, alpha-level 0.05) could be inferred as proof or probability statements about the truth or falsehood of the null hypothesis (H-0) and the alternative hypothesis (H-1).Results: Almost all participants reported having a university degree, and among statisticians/epidemiologist, most reported having a university degree in statistics and were working professionally with statistics. Overall, 9.4% of statisticians/epidemiologist and 24.0% of doctoral students responded that the statistically significant finding proved that H-0 is not true, and 73.4% of statisticians/epidemiologists and 53.3% of doctoral students responded that the statistically significant finding indicated that H(0 )is improbable. Corresponding numbers about inferences about the alternative hypothesis (H-1) were 12.0% and 6.2% about proving H-1 being true and 62.7 and 62.5% for the conclusion that H-1 is probable. Correct inferences to both questions, which is that a statistically significant finding cannot be inferred as either proof or a measure of a hypothesis' probability, were given by 10.7% of doctoral students and 12.5% of statisticians/epidemiologists.Conclusions: Misinterpretation of P-values and statistically significant test results persists also among persons who have substantial statistical education and who work professionally with statistics.

Subject headings

MEDICIN OCH HÄLSOVETENSKAP  -- Hälsovetenskap -- Folkhälsovetenskap, global hälsa, socialmedicin och epidemiologi (hsv//swe)

MEDICAL AND HEALTH SCIENCES  -- Health Sciences -- Public Health, Global Health, Social Medicine and Epidemiology (hsv//eng)

NATURVETENSKAP  -- Matematik -- Sannolikhetsteori och statistik (hsv//swe)

NATURAL SCIENCES  -- Mathematics -- Probability Theory and Statistics (hsv//eng)

Keyword

Statistical inference

null hypothesis significance testing

statistics

frequentist

P-value

Publication and Content Type

ref (subject category)

art (subject category)