The impact of a proposal for innovation measurement in the software industry

• Background: Measuring an organization's capability to innovate and assessing its innovation output and performance is a challenging task. Previously, a comprehensive model and a suite of measurements to support this task were proposed. Aims: In the current paper, seven years since the publication of the paper titled Towards innovation measurement in the software industry, we have reflected on the impact of thework. Method:We have mainly relied on quantitative and qualitative analysis of the citations of the paper using an established classification schema. Results: We found that the article has had a significant scientific impact (indicated by the number of citations), i.e., (1) cited in literature from both software engineering and other fields, (2) cited in grey literature and peerreviewed literature, and (3) substantial citations in literature not published in the English language. However, we consider a majority of the citations in the peer-reviewed literature (75 out of 116) as neutral, i.e., they have not used the innovation measurement paper in any substantial way. All in all, 38 out of 116 have used, modified or based their work on the definitions, measurements or the model proposed in the article. This analysis revealed a significant weakness of the citing work, i.e., among the citing papers, we found only two explicit comparisons to the innovation measurement proposal, and we found no papers that identify weaknesses of said proposal. Conclusions: This work highlights the need for being cautious of relying solely on the number of citations for understanding impact, and the need for further improving and supporting the peer-review process to identify unwarranted citations in papers. © 2020 IEEE Computer Society. All rights reserved.

Subject headings

SAMHÄLLSVETENSKAP  -- Medie- och kommunikationsvetenskap -- Biblioteks- och informationsvetenskap (hsv//swe)

SOCIAL SCIENCES  -- Media and Communications -- Information Studies (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Produktionsteknik, arbetsvetenskap och ergonomi (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Production Engineering, Human Work Science and Ergonomics (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Annan teknik -- Övrig annan teknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies -- Other Engineering and Technologies not elsewhere specified (hsv//eng)

Keyword

Citation analysis

Impact

Innovation

Measurement

Relevance

Capability to innovate

Comprehensive model

English languages

Grey literature

Innovation measurements

Peer-review process

Quantitative and qualitative analysis

Software industry

Software engineering

innovation

impact

relevance

measurement

citation analysis

Publication and Content Type

ref (subject category)

kon (subject category)