Towards a methodology for participant selection in software engineering experiments a vision of the future

• Background. Software Engineering (SE) researchers extensively perform experiments with human subjects. Well-defined samples are required to ensure external validity. Samples are selected purposely or by convenience, limiting the generalizability of results. Objective.We aim to depict the current status of participants selection in empirical SE, identifying the main threats and how they are mitigated. We draft a robust approach to participants selection. Method. We reviewed existing participants selection guidelines in SE, and performed a preliminary literature review to find out how participants selection is conducted in SE in practice. Results. We outline a new selection methodology, by 1) defining the characteristics of the desired population, 2) locating possible sources of sampling available for researchers, and 3) identifying and reducing the "distance" between the selected sample and its corresponding population. Conclusion. We propose a roadmap to develop and empirically validate the selection methodology. © 2021 IEEE Computer Society. All rights reserved.

Subject headings

NATURVETENSKAP  -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Software Engineering (hsv//eng)

Keyword

Controlled Experiment

Empirical Software Engineering

Generalizability

Participant Selection

Threats to Validity

Current status

External validities

Human subjects

Robust approaches

Software engineering experiments

Threat to validity

Software engineering

Publication and Content Type

ref (subject category)

kon (subject category)