Using NLP Tools to Detect Ambiguities in System Requirements - A Comparison Study

• Requirements engineering is a time-consuming process, and it can benefit significantly from automated tool support. Ambiguity detection in natural language requirements is a challenging problem in the requirements engineering community. Several Natural Language Processing tools and techniques have been developed to improve and solve the problem of ambiguity detection in natural language requirements. However, there is a lack of empirical evaluation of these tools. We aim to contribute the understanding of the empirical performance of such solutions by evaluating four tools using the dataset of 180 system requirements from the electric train propulsion system provided to us by our industrial partner Alstom. The tools that were selected for this study are Automated Requirements Measurement (ARM), Quality Analyzer for Requirement Specifications (QuARS), REquirements Template Analyzer (RETA), and Requirements Complexity Measurement (RCM). Our analysis showed that selected tools could achieve high recall. Two of them had the recall of 0.85 and 0.98. But they struggled to achieve high precision. The RCM, an in-house developed tool by our industrial partner Alstom, achieved the highest precision in our study of 0.68. 

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap -- Systemvetenskap, informationssystem och informatik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Information Systems (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Sciences (hsv//eng)

Nyckelord

Ambiguity

Natural language processing

Natural language requirements

Requirements engineering

Electric tools

Natural language processing systems

Propulsion

Automated tool support

Comparison study

Complexity measurement

High-precision

Industrial partners

NLP tools

Requirement engineering

System requirements

Publikations- och innehållstyp

ref (ämneskategori)

kon (ämneskategori)