| 1. |
- Minhas, Nasir Mehmood, 1973-
(författare)
-
Regression Testing Challenges and Solutions : An Industry-Academia Perspective
- 2019
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Background: Software quality assurance (QA) is an essential activity in the software development lifecycle. Among the different QA activities, regression testing is a challenging task for large-scale software development. Regression testing is a well-researched area, and a large number of techniques have been proposed to fulfill the needs of industry. Despite the extensive research, the adoption of proposed regression testing techniques in the industry is limited. Studies show that there is a visible gap between research and practice.Objective: This work aims at reducing the gap between industry and academia in regression testing. To fulfill this aim we have the following objectives:1) Understanding the practitioners' goals regarding regression testing.2) Understanding the current state of regression testing practice and challenges in the industry.3) Investigating the testing research applicable in an industrial context.Method: We conducted multiple studies using different methods.To explore the industry perspective on regression testing we used focus group and interview-based studies. To explore solutions from the literature, we used the systematic literature review and systematic mapping study.Results: This thesis presents the practitioners' specific regression testing goals. The identified goals are confidence, controlled fault slippage, effectiveness, efficiency, and customer satisfaction. The challenges identified in the thesis are of two categories, 1) management related challenges and 2) technical challenges. Technical challenges relate to test suite maintenance, test case selection, test case prioritization, evaluation of regression testing.We have mapped 26 empirically evaluated regression testing techniques to the context, effect, and information taxonomies, and provided a guide to the practitioners regarding the adoption of the techniques in an industrial setting. We have also classified 56 model-based test case generation techniques regarding their strengths/limitations, input/intermediate models used, and relevance to the industrial context.Conclusions: The challenges identified in this study are not new for research and practice. There could be two reasons regarding the presence of recurring challenges: 1) regression testing techniques proposed in the literature do not fit the companies’ context, 2) or, companies are not aware of the availability of the techniques that could be suitable for their context. To support the adoption of existing research on regression testing in the industry, we have presented three taxonomies. These taxonomies, allow the characterization of regression testing techniques and enable to determine which of these techniques might be suitable in a given context. Furthermore, the identification of information needs for these techniques would be helpful to learn the implications regarding the cost of adoption. Regarding the support in test case generation, we conclude that current research on interaction model-based test case generation techniques did not illustrate the use of rigorous methodology, and currently, model-based test case generation techniques have low relevance for the industrial problems.
|
|
| 2. |
- Irshad, Mohsin, 1984-
(författare)
-
Realizing the systematic reuse of automated acceptance tests in practice
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Context: Automated acceptance testing has become a relevant practice of agile software development (e.g., Extreme Programming). Automated acceptance tests (AATs) are closely tied to requirements and provide a mechanism for continuous validation of requirements as system-level tests. Studies have shown that these tests are costly to develop, maintain, and reuse.Objectives: This thesis examines and supports the systematic reuse of automated acceptance tests by providing approaches to search, identity, adapt reusable test cases while considering the reuse costs. Additionally, the maintainability and usage of automated acceptance tests in largescale software projects are studied.Method: We used various research methods to investigate development, reuse, and maintaining the automated acceptance tests. Workshops and interviews of practitioners were used to identify the benefits and challenges of automated acceptance tests. Next, method engineering was used to construct a systematic reuse process for automated acceptance tests. Qualitative feedback collected using a survey, and industrial demonstration examined the reuse process’s performance expectancy, effort expectancy, and necessary facilitating conditions. Two systematic literature reviews are used to identify techniques to adapt tests for future reuse opportunities and calculate the reuse costs of automated acceptance tests. Later, we developed and evaluated an approach for refactoring behavior-driven development-based automated acceptance tests using action research.Results: A cost-aware systematic reuse process was constructed to support the reuse of automated acceptance tests containing eleven activities. For each activity, guidelines on expected input, expected output, the actors performing the activity, and techniques (automated using scripts) are suggested and evaluated. The techniques involve approaches to support development for reuse and methods to calculate the costs of reusing automated acceptance tests. The industrial evaluation of the reuse process and the techniques showed its usefulness and relevance for the industry. Furthermore, seven challenges (i.e., the scale of the software projects, ownership, lack of competence, cost benefits, specification of behaviors in large-scale projects, difficulty writing system-level test-cases, and versioning control of behaviors) and five benefits (i.e., understanding of a business aspect of requirements, improved quality of requirements, a guide to system-level use-cases, reuse of artifacts in large-scale projects, and help for test organization.) of using automated acceptance tests in large-scale projects are identified with software practitioners’ help. Later, we proposed a semiautomated four-step approach for pre-processing, measuring, ranking, and identifying refactoring candidates. The approach and the two proposed measures were successfully evaluated using two industrial projects. It was noted that similarity measures could support the maintenance of the specification base using refactoring.Conclusions: The studies show that automated acceptance tests are reusable and can be refactored using our proposed approach. The evaluation shows that the results apply to the software industry in the evaluated context. However, further work is required to evaluate the reuse process in different contexts.
|
|
| 3. |
- Moström, Jan Erik, 1962-
(författare)
-
A study of student problems in learning to program
- 2011
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Programming is a core subject within Computer Science curricula and many also consider it a particularly difficult subject to learn. There have been many studies and suggestions on what causes these difficulties and what can be done to improve the situation. This thesis builds on previous work, trying to understand what difficulties students have when learning to program. The included papers cover several areas encountered when trying to learn programming. In Paper I we study how students use annotations during problem solving. The results show that students who annotate more also tend to be more successful. However, the results also indicate that there might be a cultural bias towards the use of annotations. Not only do students have problems with programming, they also have problems with designing software. Even graduating students fail to a large extent on simple design tasks. Our results in Paper II show that the majority of the students do not go beyond restating the problem when asked to design a system. Getting stuck is something that most learners experience at one time or another. In Paper III we investigate how successful students handle these situations. The results show that the students use a large number of different strategies to get unstuck and continue their learning. Many of the strategies involve social interaction with peers and others. In Papers IV, V, and VI we study what students experience as being key and threshold concepts in Computer Science. The results show that understanding particular concepts indeed affect the students greatly, changing the way they look at Computer Science, their peers, and themselves. The two last papers, Papers VII and VIII, investigate how researchers, teachers and students view concurrency. Most researchers/teachers claim that students have difficulties because of non-determinism, not understanding synchronization, etc. According to our results the students themselves do not seem to think that concurrency is significantly more difficult than any other subject. Actually most of them find concurrency to be both easy to understand and fun.
|
|
| 4. |
- Nordström, Marie, 1959-
(författare)
-
Object oriented quality in introductory programming education
- 2010
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Examples are important when we attempt to learn something new. To learn prob-lem solving and programming is an acknowledged difficulty. Teaching and learning introductory object oriented problem solving and programming has been discussed extensively since the late 1990’ies, when a major shift to object orientation as first programming paradigm took place. Initially, this switch was not considered to cause any major problems, because of the accumulated knowledge for how pro-gramming should be taught. This turned out to be naive. Knowledge gained for the imperative paradigm did not apply well to the object oriented paradigm. Because of its importance for the field of computer science, introductory pro- gramming education has drawn a lot of attention. Most of the research done in connection to object oriented problem solving and programming has been focused on students learning and the difficulty to acquire skills in programming. Less investigated is the foundation of the educational mission, the characteris- tics of object orientation and how this is best supported by the educator. There is no obvious agreement of what the basics of object orientation are, especially not from an educational point of view. In this thesis, two major aspects concerning the teaching of object orientation have been investigated: the definition of object oriented quality, specifically in examples for novices, and educators’ views on aspects of object orientation. Based on research of how object orientation is characterised in literature and in software design principles, a set of concepts and principles are presented as a description of basic characteristics of object orientation. These are applied to the educational context, and a number of heuristics, called Eduristics, for the design of object oriented examples for novices are defined. The Eduristics are then used to discuss the flaws and shortcomings of common textbook examples, but also how the object oriented quality of examples can be improved. To be able to evaluate the quality of examples, we initiated and participated in the development of an evaluation tool. This tool has been used to evaluate a number of examples from popular textbooks. The results show that the object oriented quality of examples is low. To explore the ways educators view a number of aspects of object orientation and the teaching of it, ten interviews have been conducted. The results of this study show that the level of abstraction in the conceptual model of object orientation among educators is low, and that novices are not given any support for object oriented problem solving.
|
|
| 5. |
- Eriksson, Magnus, 1976-
(författare)
-
Engineering Families of Software-Intensive Systems using Features, Goals and Scenarios
- 2007
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Over the last decade, software has become an integral part of many products with which it is not traditionally associated (e.g., automobiles, medical equipment, home appliances, etc.). This has led to problems in many organizations, since it has proved difficult to integrate software engineering processes with other engineering processes. To address the increased complexity and to coordinate their engineering efforts, many organizations working in such domains have therefore introduced systems engineering into their business processes. Systems engineering is an interdisciplinary approach to system development. Teamwork is essential in developing complex systems, and systems engineering orchestrates this process throughout the lifespan of a system. The focus of systems engineering is on defining customer needs and required functionality, documenting requirements, synthesizing a design and validating the system. A problem with the traditional techniques of systems engineering is, however, that they provide inadequate support for achieving high levels of reuse between different projects. In this dissertation, methods and tools with the overall purpose of providing an improved reuse infrastructure for systems engineering artifacts and their resulting detailed design artifacts are proposed. The proposed methods and tools are based on the software product-line approach, a reuse strategy which focuses on families of related systems that address particular market segments. Studies have shown that if an empirical study does not report experiences from that which practitioners consider a “real” situation, results are likely to be ignored. Even though case studies cannot achieve the scientific rigor of formal experiments, case studies can provide enough information to decide if a specific technology will benefit a particular organization. To ensure industry-relevant research results, case studies on real development projects were chosen as the main vehicle for performing research. This dissertation reports experiences from four empirical studies which were performed within the Swedish defense industry. The studies indicate that the proposed methods and tools indeed provide an improved reuse infrastructure, compared to the previously used methods and tools.
|
|
| 6. |
- Loconsole, Annabella, 1965-
(författare)
-
Definition and validation of requirements management measures
- 2007
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The quality of software systems depends on early activities in the software development process, of which the management of requirements is one. When requirements are not managed well, a project can fail or become more costly than intended, and the quality of the software developed can decrease. Among the requirements management practices, it is particularly important to quantify and predict requirements volatility, i.e., how much the requirements are likely to change over time. Software measures can help in quantifying and predicting requirements attributes like volatility. However, few measures have yet been defined, due to the fact that the early phases are hard to formalise. Furthermore, very few requirements measures have been validated, which would be needed in order to demonstrate that they are useful. The approach to requirements management in this thesis is quantitative, i.e. to monitor the requirements management activities and requirements volatility through software measurement. In this thesis, a set of 45 requirements management measures is presented. The measures were defined using the goal question metrics framework for the two predefined goals of the requirements management key process area of the capability maturity model for software. A subset of these measures was validated theoretically and empirically in four case studies. Furthermore, an analysis of validated measures in the literature was performed, showing that there is a lack of validated process, project, and requirements measures in software engineering. The studies presented in this thesis show that size measures are good estimators of requirements volatility. The important result is that size is relevant: increasing the size of a requirements document implies that the number of changes to requirements increases as well. Furthermore, subjective estimations of volatility were found to be inaccurate assessors of requirements volatility. These results suggest that practitioners should complement the subjective estimations for assessing volatility with the objective ones. Requirements engineers and project managers will benefit from the research presented in this thesis because the measures defined, proved to be predictors of volatility, can help in understanding how much requirements will change. By deploying the measures, the practitioners would be prepared for possible changes in the schedule and cost of a project, giving them the possibility of creating alternative plans, new cost estimates, and new software development schedules.
|
|
| 7. |
- Nordström, Marie, 1959-
(författare)
-
He[d]uristics - Heuristics for designing object oriented examples for novices
- 2009
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The use of examples is known to be important in learning, they should be “exemplary” and function as role-models. Teaching and learning problem solving and programming in the object oriented paradigm is recognised as difficult. Object orientation is designed to handle complexity and large systems, and not with education in focus. The fact that object orientation often is used as first paradigm makes the design of examples even more difficult and important. In this thesis, a survey of the literature is made to establish a set of characteristics for object orientation in general. This set of characteristics is then applied to the educational setting of introducing novices to object oriented problem solving and programming, resulting in a number of heuristics for educational purposes, called He[d]uristics. The proposed He[d]uristics are targeted towards educators designing small-scale examples for novices, and is an attempt to provide help in designing suitable examples, not a catalogue of good ones. The He[d]uristics are discussed and exemplified and also evaluated versus the derived set of characteristics and known common problems experienced by novices.
|
|
| 8. |
- Nurdiani, Indira, et al.
(författare)
-
Understanding the order of agile practice introduction : Comparing agile maturity models and practitioners’ experience
- 2019
-
Ingår i: Journal of Systems and Software. - : Elsevier Inc.. - 0164-1212 .- 1873-1228. ; 156, s. 1-20
-
Tidskriftsartikel (refereegranskat)abstract
- Context: Agile maturity models (AMMs) suggest that agile practices are introduced in a certain order. However, whether the order of agile practice introduction as suggested in the AMMs is relevant in industry has not been evaluated in an empirical study. Objectives: In this study, we want to investigate: (1) order of agile practice introduction mentioned in AMMs, (2) order of introducing agile practices in industry, and (3) similarities and differences between (1) and (2). Methods: We conducted a literature survey to identify strategies proposed by the AMMs. We then compared the AMMs’ suggestions to the strategies used by practitioners, which we elicited from a survey and a series of interviews from an earlier study. Results: The literature survey revealed 12 AMMs which provide explicit mappings of agile practices to maturity levels. These mappings showed little agreement on when practices should be introduced. Comparison of the AMMs’ suggestions and the empirical study revealed that the guidance suggested by AMMs are not aligned with industry practice. Conclusion: Currently, AMMs do not provide sufficient information to guide agile adoption in industry. Our results suggest that there might be no universal strategy for agile adoption that works better than others. © 2019 Elsevier Inc.
|
|
| 9. |
- Nurdiani, Indira, et al.
(författare)
-
Understanding the order of agile practice introduction : Comparing agile maturity models and practitioners’ experience
- 2019
-
Konferensbidrag (refereegranskat)abstract
- Agile software development has become increasingly popular over the past years. Agile methods are perceived to address challenges caused by the rapid change in the market while reducing time to market and development. As the popularity of Agile method is growing, the need for Agile adoption guidance also increases. Over the past years, Agile Maturity Models (AMMs) have been proposed to offer guidelines in Agile adoption. Agile maturity models (AMMs) suggest that Agile practices are introduced in a specific order. However, a number of evaluations of AMMs show that they are not properly validated and not suited for use in industry. Current evaluations of AMMs are based on certain criteria and not based on empirical studies. There are no studies that evaluate AMMs' suggestion to introduce Agile in certain order against industry practice. The relevance of the AMMs in the industry is not yet examined.In this study, we conducted a literature review to identify the order of Agile practice introduction mentioned in AMMs. We then compared the AMMs' suggestions to the strategies used by practitioners, which we elicited from a survey and a series of interviews.The literature survey revealed 12 AMMs which provide explicit mappings of Agile practices to maturity levels. These mappings show that the AMMs are not in agreement pertaining to the which practice is to be included in which maturity levels. The AMMs do not provide clear information nor rationale why certain practices need to be introduced in a specific maturity level. Comparison of the AMMs suggestions and the empirical study revealed that the guidance suggested by AMMs are not aligned with industry practice. The AMMs take a simplistic view of introducing Agile practices in a pre-determined order without consideration of the different contexts in industry. Our study indicates that practitioners iteratively add, remove, and modify the set of Agile practices they use to match their needs and contexts. In addition, the AMMs do not consider the potential perceived trade-off of adding Agile practices, such as team member discomfort, conflicts with other teams, and recurring bugs.Currently, AMMs do not provide sufficient information to guide Agile adoption in the industry. Associating Agile maturity with sets of Agile practices is inadequate and too simplified. Our results suggest that there might be no universal strategy for Agile adoption that works better than others.
|
|
| 10. |
- Toll, Daniel, 1978-
(författare)
-
Measuring Programming Assignment Effort
- 2016
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Students often voice that the programming assignments are hard and that they spend a lot of time on solving them. Is this true; are we giving them too hard assignments and how much and what do they spend the time on? This is what we want to gain insight to. We constructed a tool that records programming sessions with finer granularity than the existing solutions. The tool has recorded 2643 programming sessions from students. Using that data we found that students spend only 15% of their time writing code, and that on average 40% of their programming effort is spent reading and navigating. We also estimate the time spent outside of the tool to be almost 20%. The increased detail in the recordings can be used to measure the effect of programming source code comments, and we found that the effect of both helpful and redundant comments increases the reading time but did not help to reduce the students writing effort. Finally, we used the tool to examine the effects of an improved programming assignment and found that the total effort was not reduced.
|
|
