Evaluating the Effects of Different Requirements Representations on Writing Test Cases

• Context and Motivation One must test a system to ensure that the requirements are met, thus, tests are often derived manually from requirements. However, requirements representations are diverse; from traditional IEEE-style text, to models, to agile user stories, the RE community of research and practice has explored various ways to capture requirements. Question/problem But, do these different representations influence the quality or coverage of test suites? The state-of-the-art does not provide insights on whether or not the representation of requirements has an impact on the coverage, quality, or size of the resulting test suite. Results In this paper, we report on a family of three experiment replications conducted with 148 students which examines the effect of different requirements representations on test creation. We find that, in general, the different requirements representations have no statistically significant impact on the number of derived tests, but specific affordances of the representation effect test quality, e.g., traditional textual requirements make it easier to derive less abstract tests, whereas goal models yield less inconsistent test purpose descriptions. Contribution Our findings give insights on the effects of requirements representation on test derivation for novice testers. Our work is limited in the use of students.

Subject headings

TEKNIK OCH TEKNOLOGIER  -- Maskinteknik -- Tillförlitlighets- och kvalitetsteknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Mechanical Engineering -- Reliability and Maintenance (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

Keyword

Requirements representation

Test design

Experiment

Publication and Content Type

kon (subject category)

ref (subject category)