The GRIM Test : A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology

• We present a simple mathematical technique that we call granularity-related inconsistency of means (GRIM) for verifying the summary statistics of research reports in psychology. This technique evaluates whether the reported means of integer data such as Likert-type scales are consistent with the given sample size and number of items. We tested this technique with a sample of 260 recent empirical articles in leading journals. Of the articles that we could test with the GRIM technique (N = 71), around half (N = 36) appeared to contain at least one inconsistent mean, and more than 20% (N = 16) contained multiple such inconsistencies. We requested the data sets corresponding to 21 of these articles, receiving positive responses in 9 cases. We confirmed the presence of at least one reporting error in all cases, with three articles requiring extensive corrections. The implications for the reliability and replicability of empirical psychology are discussed.

Subject headings

SAMHÄLLSVETENSKAP  -- Psykologi (hsv//swe)

SOCIAL SCIENCES  -- Psychology (hsv//eng)

Keyword

Psychology

Psykologi

Publication and Content Type

ref (subject category)

art (subject category)