Impact of Gamification on Trace Link Vetting: A Controlled Experiment

• [Context] Automatically generated trace links must be vetted by human analysts before use. The task of vetting trace links is considered boring due to its repetitive nature and tools that are not engaging to the analyst. Therefore, a lack of developer engagement can hamper the successful implementation of a traceability strategy in an organisation. [Objective] In this study, we examine whether two gamification features, levels and badges, have a positive effect on human analysts’ engagement and ultimately on the quality of vetted trace links. [Method] We have conducted a controlled experiment with 24 participants that vetted trace link candidates and recorded their speed, correctness, enjoyment, and perceived usability of the tool. [Results] The results indicate that there was no significant difference between the speed, correctness, and perceived usability of the control and the experiment group. However, gamification features significantly increased the users’ perceived enjoyment. Levels and badges were perceived positively by the majority of the participants while some pitfalls and improvements were pointed out. [Conclusion] Our study indicates the need for further research as the results raise several questions, in particular, w.r.t. what analyst behaviour gamification incentivises, and the impact of gamification on long-term enjoyment.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

Keyword

Software engineering

Gamification

Traceability

Traceability management

Publication and Content Type

ref (subject category)

kon (subject category)