Evaluating Evaluations of Innovation Policy: Exploring Reliability, Methods, and Conflicts of Interest

• Expansions of innovation policies have been paralleled with an increasein the evaluations of such policies. Yet, there are few systematic evaluations of how such evaluations are conducted, by whom, and their overall conclusions. We analyze 110 evaluations of innovation policy in Sweden from 2005 to 2019. Our findings show that the majority of these evaluations are positive, about one-third are neutral in their conclusions, and very few are negative. The majority of evaluations were conducted by consulting firms, close to one-third by expert government agencies, and around 10% by university researchers or as self-evaluations by the governmental agencies responsible for the policy themselves. Few evaluations employed causal methods to assess the potential effects of policies. We discuss conflicts of interest and question the reliability of evaluations of innovation policy.

• Expansions of innovation policies have been paralleled with an increase in the evaluations of such policies. Yet, there are few systematic evaluations of how such evaluations are conducted, by whom, and their overall conclusions. We analyze 110 evaluations of innovation policy in Sweden from 2005 to 2019. Our findings show that the majority of these evaluations are positive, about one-third are neutral in their conclusions, and very few are negative. The majority of evaluations were conducted by consulting firms, close to one-third by expert government agencies, and around 10% by university researchers or as self-evaluations by the governmental agencies responsible for the policy themselves. Few evaluations employed causal methods to assess the potential effects of policies. We discuss conflicts of interest and question the reliability of evaluations of innovation policy.

Subject headings

SAMHÄLLSVETENSKAP  -- Ekonomi och näringsliv -- Företagsekonomi (hsv//swe)

SOCIAL SCIENCES  -- Economics and Business -- Business Administration (hsv//eng)

SAMHÄLLSVETENSKAP  -- Ekonomi och näringsliv -- Nationalekonomi (hsv//swe)

SOCIAL SCIENCES  -- Economics and Business -- Economics (hsv//eng)

SAMHÄLLSVETENSKAP  -- Statsvetenskap (hsv//swe)

SOCIAL SCIENCES  -- Political Science (hsv//eng)

Keyword

Evaluation

Consultants

Evaluating practice

Meta-Evaluation

Sweden

Innovation policy

Consultants

Publication and Content Type

kap (subject category)

vet (subject category)