Using NLP Tools to Detect Ambiguities in System Requirements - A Comparison Study

Requirements engineering is a time-consuming process, and it can benefit significantly from automated tool support. Ambiguity detection in natural language requirements is a challenging problem in the requirements engineering community. Several Natural Language Processing tools and techniques have been developed to improve and solve the problem of ambiguity detection in natural language requirements. However, there is a lack of empirical evaluation of these tools. We aim to contribute the understanding of the empirical performance of such solutions by evaluating four tools using the dataset of 180 system requirements from the electric train propulsion system provided to us by our industrial partner Alstom. The tools that were selected for this study are Automated Requirements Measurement (ARM), Quality Analyzer for Requirement Specifications (QuARS), REquirements Template Analyzer (RETA), and Requirements Complexity Measurement (RCM). Our analysis showed that selected tools could achieve high recall. Two of them had the recall of 0.85 and 0.98. But they struggled to achieve high precision. The RCM, an in-house developed tool by our industrial partner Alstom, achieved the highest precision in our study of 0.68.

NATURVETENSKAP Data- och informationsvetenskap Systemvetenskap, informationssystem och informatik hsv//swe
NATURAL SCIENCES Computer and Information Sciences Information Systems hsv//eng
NATURVETENSKAP Data- och informationsvetenskap Datavetenskap hsv//swe
NATURAL SCIENCES Computer and Information Sciences Computer Sciences hsv//eng
Ambiguity
Natural language processing
Natural language requirements
Requirements engineering
Electric tools
Natural language processing systems
Propulsion
Automated tool support
Comparison study
Complexity measurement
High-precision
Industrial partners
NLP tools
Requirement engineering
System requirements

Leon, MiguelMälardalens universitet,Innovation och produktrealisering(Swepub:mdh)mlz02 (author)
Afzal, WasifMälardalens universitet,Inbyggda system(Swepub:mdh)wal01 (author)
Lindberg, P.Alstom Sweden, Västerås, Sweden (author)
Bohlin, Markus,1976-Mälardalens universitet,Innovation och produktrealisering(Swepub:mdh)mbn05 (author)
Mälardalens universitetInnovation och produktrealisering (creator_code:org_t)

By the author/editor: Bajceta, Aleksan ...; Leon, Miguel; Afzal, Wasif; Lindberg, P.; Bohlin, Markus, ...

About the subject

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...; and Information Syst ...

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...; and Computer Science ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.