| 1. |
|
|
| 2. |
- Ahmad, Azeem, et al.
(författare)
-
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
- 2021
-
Ingår i: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE COMPUTER SOC. - 1530-1362. ; , s. 338-348
-
Konferensbidrag (refereegranskat)abstract
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
|
|
| 3. |
- Ahmad, Azeem, et al.
(författare)
-
An Industrial Study on the Challenges and Effects of Diversity-Based Testing in Continuous Integration
- 2023
-
Ingår i: IEEE International Conference on Software Quality, Reliability and Security, QRS. - 2693-9177. - 9798350319583
-
Konferensbidrag (refereegranskat)abstract
- Many test prioritisation techniques have been proposed in order to improve test effectiveness of Continuous Integration (CI) pipelines. Particularly, diversity-based testing (DBT) has shown promising and competitive results to improve test effectiveness. However, the technical and practical challenges of introducing test prioritisation in CI pipelines are rarely discussed, thus hindering the applicability and adoption of those proposed techniques. This research builds on our prior work in which we evaluated diversity-based techniques in an industrial setting. This work investigates the factors that influence the adoption of DBT both in connection to improvements in test cost-effectiveness, as well as the process and human related challenges to transfer and use DBT prioritisation in CI pipelines. We report on a case study considering the CI pipeline of Axis Communications in Sweden. We performed a thematic analysis of a focus group interview with senior practitioners at the company to identify the challenges and perceived benefits of using test prioritisation in their test process. Our thematic analysis reveals a list of ten challenges and seven perceived effects of introducing test prioritisation in CI cycles. For instance, our participants emphasized the importance of introducing comprehensible and transparent techniques that instill trust in its users. Moreover, practitioners prefer techniques compatible with their current test infrastructure (e.g., test framework and environments) in order to reduce instrumentation efforts and avoid disrupting their current setup. In conclusion, we have identified tradeoffs between different test prioritisation techniques pertaining to the technical, process and human aspects of regression testing in CI. We summarize those findings in a list of seven advantages that refer to specific stakeholder interests and describe the effects of adopting DBT in CI pipelines.
|
|
| 4. |
- Ahmad, Azeem, 1984-
(författare)
-
Contributions to Improving Feedback and Trust in Automated Testing and Continuous Integration and Delivery
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- An integrated release version (also known as a release candidate in software engineering) is produced by merging, building, and testing code on a regular basis as part of the Continuous Integration and Continuous Delivery (CI/CD) practices. Several benefits, including improved software quality and shorter release cycles, have been claimed for CI/CD. On the other hand, recent research has uncovered a plethora of problems and bad practices related to CI/CD adoption, necessitating some optimization. Some of the problems addressed in this work include the ability to respond to practitioners’ questions and obtain quick and trustworthy feedback in CI/CD. To be more specific, our effort concentrated on: 1) identifying the information needs of software practitioners engaged in CI/CD; 2) adopting test optimization approaches to obtain faster feedback that are realistic for use in CI/CD environments without introducing excessive technical requirements; 3) identifying perceived causes and automated root cause analysis of test flakiness, thereby providing developers with guidance on how to resolve test flakiness; and 4) identifying challenges in addressing information needs, providing faster and more trustworthy feedback. The findings of the research reported in this thesis are based on data from three single-case studies and three multiple-case studies. The research uses quantitative and qualitative data collected via interviews, site visits, and workshops. To perform our analyses, we used data from firms producing embedded software as well as open-source repositories. The following are major research and practical contributions. Information Needs: The initial contribution to research is a list of information needs in CI/CD. This list contains 27 frequently asked questions on continuous integration and continuous delivery by software practitioners. The identified information needs have been classified as related to testing, code & commit, confidence, bug, and artifacts. We investigated how companies deal with information needs, what tools they use to deal with them, and who is interested in them. We concluded that there is a discrepancy between the identified needs and the techniques employed to meet them. Since some information needs cannot be met by current tools, manual inspections are required, which adds time to the process. Information about code & commit, confidence level, and testing is the most frequently sought for and most important information. Evaluation of Diversity Based Techniques/Tool: The contribution is to conduct a detailed examination of diversity-based techniques using industry test cases to determine if there is a difference between diversity functions in selecting integrationlevel automated test. Additionally, how diversity-based testing compares to other optimization techniques used in industry in terms of fault detection rates, feature coverage, and execution time. This enables us to observe how coverage changes when we run fewer test cases. We concluded that some of the techniques can eliminate up to 85% of test cases (provided by the case company) while still covering all distinct features/requirements. The techniques are developed and made available as an open-source tool for further research and application. Test Flakiness Detection, Prediction & Automated Root Cause Analysis: We identified 19 factors that professionals perceive affect test flakiness. These perceived factors are divided into four categories: test code, system under test, CI/test infrastructure, and organizational. We concluded that some of the perceived factors of test flakiness in closed-source development are directly related to non-determinism, whereas other perceived factors concern different aspects e.g., lack of good properties of a test case (i.e., small, simple and robust), deviations from the established processes, etc. To see if the developers’ perceptions were in line with what they had labelled as flaky or not, we examined the test artifacts that were readily available. We verified that two of the identified perceived factors (i.e., test case size and simplicity) are indeed indicative of test flakiness. Furthermore, we proposed a light weight technique named trace-back coverage to detect flaky tests. Trace-back coverage was combined with other factors such as test smells indicating test flakiness, flakiness frequency and test case size to investigate the effect on revealing test flakiness. When all factors are taken into consideration, the precision of flaky test detection is increased from 57% (using single factor) to 86% (combination of different factors).
|
|
| 5. |
- Ahmad, Azeem, et al.
(författare)
-
Data visualisation in continuous integration and delivery : Information needs, challenges, and recommendations
- 2022
-
Ingår i: IET Software. - : WILEY. - 1751-8806 .- 1751-8814. ; 16:3, s. 331-349
-
Tidskriftsartikel (refereegranskat)abstract
- Several operations, ranging from regular code updates to compiling, building, testing, and distribution to customers, are consolidated in continuous integration and delivery. Professionals seek additional information to complete the mission at hand during these tasks. Developers who devote a large amount of time and effort to finding such information may become distracted from their work. We will better understand the processes, procedures, and resources used to deliver a quality product on time by defining the types of information that software professionals seek. A deeper understanding of software practitioners information needs has many advantages, including remaining competitive, growing knowledge of issues that can stymie a timely update, and creating a visualisation tool to assist practitioners in addressing their information needs. This is an extension of a previous work done by the authors. The authors conducted a multiple-case holistic study with six different companies (38 unique participants) to identify information needs in continuous integration and delivery. This study attempts to capture the importance, frequency, required effort (e.g. sequence of actions required to collect information), current approach to handling, and associated stakeholders with respect to identified needs. 27 information needs associated with different stakeholders (i.e. developers, testers, project managers, release team, and compliance authority) were identified. The identified needs were categorised as testing, code & commit, confidence, bug, and artefacts. Apart from identifying information needs, practitioners face several challenges in developing visualisation tools. Thus, 8 challenges that were faced by the practitioners to develop/maintain visualisation tools for the software team were identified. The recommendations from practitioners who are experts in developing, maintaining, and providing visualisation services to the software team were listed.
|
|
| 6. |
- Ahmad, Azeem, et al.
(författare)
-
Empirical analysis of practitioners perceptions of test flakiness factors
- 2021
-
Ingår i: Software testing, verification & reliability. - : Wiley-Blackwell. - 0960-0833 .- 1099-1689. ; 31:8
-
Tidskriftsartikel (refereegranskat)abstract
- Identifying the root causes of test flakiness is one of the challenges faced by practitioners during software testing. In other words, the testing of the software is hampered by test flakiness. Since the research about test flakiness in large-scale software engineering is scarce, the need for an empirical case-study where we can build a common and grounded understanding of the problem as well as relevant remedies that can later be evaluated in a large-scale context is a necessity. This study reports the findings from a multiple-case study. The authors conducted an online survey to investigate and catalogue the root causes of test flakiness and mitigation strategies. We attempted to understand how practitioners perceive test flakiness in closed-source development, such as how they define test flakiness and what practitioners perceive can affect test flakiness. The perceptions of practitioners were compared with the available literature. We investigated whether practitioners perceptions are reflected in the test artefacts such as what is the relationship between the perceived factors and properties of test artefacts. This study reported 19 factors that are perceived by professionals to affect test flakiness. These perceived factors are categorized as test code, system under test, CI/test infrastructure, and organization-related. The authors concluded that some of the perceived factors in test flakiness in closed-source development are directly related to non-determinism, whereas other perceived factors concern different aspects, for example, lack of good properties of a test case, deviations from the established processes, and ad hoc decisions. Given a data set from investigated cases, the authors concluded that two of the perceived factors (i.e., test case size and test case simplicity) have a strong effect on test flakiness.
|
|
| 7. |
- Ahmad, Azeem, et al.
(författare)
-
Identifying Randomness related Flaky Tests through Divergence and Execution Tracing
- 2022
-
Ingår i: 2022 IEEE 15TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW 2022). - : IEEE COMPUTER SOC. - 9781665496285 ; , s. 293-300
-
Konferensbidrag (refereegranskat)abstract
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, known as non-deterministic tests, change their outcomes without any changes in the codebase, thus reducing the trust of developers during a software release as well as in the quality of a product. While rerunning test cases is a common approach, it is resource intensive, unreliable, and does not uncover the actual cause of test flakiness. Our paper evaluates an approach to identify randomness-related flaky. This paper used a divergence algorithm and execution tracing techniques to identify flaky tests, which resulted in the FLAKYPY prototype. In addition, this paper discusses the cases where FLAKYPY successfully identified the flaky test as well as those cases where FLAKYPY failed. The papers discuss how the reporting mechanism of FLAKYPY can help developers in identifying the root cause of randomness-related test flakiness. Thirty-two open-source projects were used in this. We concluded that FLAKYPY can detect most of the randomness-related test flakiness. In addition, the reporting mechanism of FLAKYPY reveals sufficient information about possible root causes of test flakiness.
|
|
| 8. |
- Asplund, Mikael, 1981-, et al.
(författare)
-
Integrering av den sociala dimensionen i datautbildningar
- 2019
-
Ingår i: Bidrag från 7:e Utvecklingskonferensen för Sveriges ingenjörsutbildningar. - : Luleå: Luleå tekniska universitet. - 9789177906056 ; , s. 189-190
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)
|
|
| 9. |
- de Oliveira Neto, Francisco Gomes, et al.
(författare)
-
Improving continuous integration with similarity-based test case selection
- 2018
-
Ingår i: Proceedings of the 13th International Workshop on Automation of Software Test. - New York : ACM Digital Library. - 0270-5257. - 9781450357432 ; , s. 39-45
-
Konferensbidrag (refereegranskat)abstract
- Automated testing is an essential component of Continuous Integration (CI) and Delivery (CD), such as scheduling automated test sessions on overnight builds. That allows stakeholders to execute entire test suites and achieve exhaustive test coverage, since running all tests is often infeasible during work hours, i.e., in parallel to development activities. On the other hand, developers also need test feedback from CI servers when pushing changes, even if not all test cases are executed. In this paper we evaluate similarity-based test case selection (SBTCS) on integration-level tests executed on continuous integration pipelines of two companies. We select test cases that maximise diversity of test coverage and reduce feedback time to developers. Our results confirm existing evidence that SBTCS is a strong candidate for test optimisation, by reducing feedback time (up to 92% faster in our case studies) while achieving full test coverage using only information from test artefacts themselves.
|
|
| 10. |
- Duboc, Leticia, et al.
(författare)
-
Do we really know what we are building? Raising awareness of potential sustainability effects of software systems in requirements engineering
- 2019
-
Ingår i: Proceedings of the IEEE International Conference on Requirements Engineering. - : IEEE COMPUTER SOC. - 2332-6441 .- 1090-705X. ; 2019-September, s. 6-16
-
Konferensbidrag (refereegranskat)abstract
- Integrating novel software systems in our society, economy, and environment can have far-reaching effects. As a result, software systems should be designed in such a way as to maintain or improve the sustainability of the socio-technical system of their destination. However, a paradigm shift is required to raise awareness of software professionals on the potential sustainability effects of software systems. While Requirements Engineering is considered the key to driving this change, requirements engineers lack the knowledge, experience and methodological support for doing so. This paper presents a question-based framework for raising awareness of the potential effects of software systems on sustainability, as the first step towards enabling the required paradigm shift. A feasibility study of the framework was carried out with two groups of computer science students. The results of the study indicate that the framework helps enable discussions about potential effects that software systems could have on sustainability.
|
|
| 11. |
- Duboc, Leticia, et al.
(författare)
-
Requirements engineering for sustainability: an awareness framework for designing software systems for a better tomorrow
- 2020
-
Ingår i: Requirements Engineering. - : Springer Science and Business Media LLC. - 0947-3602 .- 1432-010X. ; 25:4, s. 469-492
-
Tidskriftsartikel (refereegranskat)abstract
- Integrating novel software systems in our society, economy and environment can have far-reaching effects. As a result, software systems should be designed in such a way as to maintain or improve the sustainability of their intended socio-technical systems. However, a paradigm shift is required to raise awareness of software professionals on the potential sustainability effects of software systems. While Requirements Engineering is considered the key for driving this change, requirements engineers lack the knowledge, experience and methodological support for acting as facilitators for a broader discussion on sustainability effects. This paper presents a question-based framework for raising awareness of the potential effects of software systems on sustainability, as the first step towards enabling the required paradigm shift. An evaluation study of the framework was conducted with four groups of computer science students. The results of the study indicate that the framework is applicable to different types of systems and helps to facilitate discussions about the potential effects that software systems could have on sustainability.
|
|
| 12. |
- Eidenskog, Maria, 1981-, et al.
(författare)
-
Changing the world one engineer at a time – unmaking the traditional engineering education when introducing sustainability subjects
- 2023
-
Ingår i: International Journal of Sustainability in Higher Education. - : EMERALD GROUP PUBLISHING LTD. - 1467-6370 .- 1758-6739. ; 24:9, s. 70-84
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose: The information technology (IT) sector has been seen as central to society's transformation to a more just and sustainable society, which underlines teachers’ responsibility to foster engineers who can contribute specifically to such ends. This study aims to report an effort to significantly update an existing engineering programme in IT with this ambition and to analyse the effects and challenges associated with the transformation.Design/methodology/approach: This study is based on a combination of action-oriented research based on implementing key changes to the curriculum; empirical investigations including surveys and interviews with students and teachers, and analysis of these; and a science and technology studies-inspired analysis.Findings: Respondents were generally positive towards adding topics relating to sustainability. However, in the unmaking of traditional engineering subjects, changes created a conflict between core versus soft subjects in which the core subjects tended to gain the upper hand. This conflict can be turned into productive discussions by focusing on what kinds of engineers the authors’ educate and how students can be introduced to societal problems as an integrated part of their education.Practical implications: This study can be helpful for educators in the engineering domain to support them in their efforts to transition from a (narrow) focus on traditional disciplines to one where the bettering of society is at the core.Originality/value: This study provides a novel approach to the transformation of engineering education through a theoretical analysis seldom used in studies of higher education on a novel case study.
|
|
| 13. |
- Enberg, Cecilia, 1976-, et al.
(författare)
-
Rapport: Lärande för hållbar utveckling vid utbildningsprogram vid IEI, LiU
- 2018
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Denna rapport beskriver en studie vars syfte har varit att undersöka vilka utmaningar och möjligheter som finns med att på ett systematiskt sätt arbeta för att främja ett lärande för hållbar utveckling vid LiU. Vår utgångspunkt har varit universitetskanslerämbetets tematiska utvärdering från 2017 av lärosätenas arbete med att främja en hållbar utveckling genom sina utbildningar. Vi har genomfört en studie med programansvariga vid utbildningar som ges vid institutionen för ekonomisk och industriell utveckling, IEI varav en del på filosofisk och andra på teknisk fakultet. Dels har vi gått igenom utbildningsplanerna för programmen, dels har vi genomfört en enkätundersökning och en uppsättning fokusgruppsdiskussioner med programansvariga.Ett mindre antal av programmen och specifikt civilingenjörsutbildningen i Energi, Miljö & Management (EMM) har i utbildningsplaner referenser till specifika mål för en hållbar utveckling som man avser förbereda sina studenter att kunna bidra till. De flesta programs primära fokus ligger dock på anställningsbarhet för sina studenter. Från enkäten och fokusgruppsdiskussioner har vi förstått att det för många programansvariga har varit det första tillfället de fått att fundera över hur deras utbildningar eventuellt bidrar till att förbereda studenter för att hantera utmaningar kopplade till en hållbar utveckling. I våra resultat märks också en tydlig ambivalens och tveksamhet inför både vad man gör just nu, vad man vill göra och vilken sorts drivkrafter och styrning man önskar i det arbetet. Flera programansvariga hävdar å ena sidan att begreppet är svårdefinierat, men att man å andra sidan är säker på att man arbetar i en riktning som gör att utbildningarna bidrar till att främja en hållbar utveckling. Man hävdar att en del lärare vid de egna programmen kan sakna kompetens att hantera problem med flera dimensioner, men anser samtidigt att de egna lärarna har den kompetens som behövs för utbildnings-programmen. Man ser positivt på studenters och externa finansiärers önskemål om förändringar av utbildningarna, men är samtidigt tveksam till många av de sätt som LiU skulle kunna använda för att stödja arbetet med för-ändringar.Baserat på vår analys av programansvarigas respons och universitetskanslersämbetets rekommendationer ger vi avslutningsvis rekommendationer för fortsatt arbete med frågan om lärande för hållbar utveckling på LiU.
|
|
| 14. |
- Eriksson, Henrik, et al.
(författare)
-
A Cloud-Based Simulation Architecture for Pandemic Influenza Simulation
- 2011
-
Ingår i: AMIA Annual Symposium Proceedings 2011. - : Curran. - 9781618396082 ; , s. 364-373
-
Konferensbidrag (refereegranskat)abstract
- High-fidelity simulations of pandemic outbreaks are resource consuming. Cluster-based solutions have been suggested for executing such complex computations. We present a cloud-based simulation architecture that utilizes computing resources both locally available and dynamically rented online. The approach uses the Condor framework for job distribution and management of the Amazon Elastic Computing Cloud (EC2) as well as local resources. The architecture has a web-based user interface that allows users to monitor and control simulation execution. In a benchmark test, the best costadjusted performance was recorded for the EC2 H-CPU Medium instance, while a field trial showed that the job configuration had significant influence on the execution time and that the network capacity of the master node could become a bottleneck. We conclude that it is possible to develop a scalable simulation environment that uses cloud-based solutions, while providing an easy-to-use graphical user interface.
|
|
| 15. |
- Johansson, Björn, Professor, PhD, 1973-, et al.
(författare)
-
Understanding the challenge of the energy crisis : Tackling system complexity with megagaming
- 2023
-
Ingår i: ECCE '23: Proceedings of the European Conference on Cognitive Ergonomics 2023. - New York, NY United States : ACM Digital Library. - 9798400708756
-
Konferensbidrag (refereegranskat)abstract
- Climate change is tightly coupled to the usage of energy in differentforms. Creating and using sustainable energy solutions is increasinglybeing recognized as the greatest challenge of our time. Theenergy crisis concerns everyone who consumes energy or servicesthat rely on energy to be performed, in practice almost everythingwe buy. However, the temporal and psychological distance to climatechange has led to a situation where the consequences of ourenergy usage often is considered a future problem or “someoneelse’s” problem. Current methods to increase awareness and understandingof climate change and the need to create sustainableenergy solutions are usually based on simulation or negotiationgames. Methods similar to crisis management exercises are uncommon.In this paper, we propose to use Megagames, large scale socialgames, as a method for increasing the awareness of the challengesrelated to creating sustainable energy solutions. Examples fromtwo Megagame test runs are provided and discussed.
|
|
| 16. |
- Leifler, Ola, 1978-, et al.
(författare)
-
A Model for Document Processing in Semantic Desktop Systems
- 2008
-
Ingår i: Proceedings of the I-KNOW '08, the International Conference on Knowledge Management. - Berlin, Heidelberg : Springer Verlag.
-
Konferensbidrag (refereegranskat)abstract
- There is a significant gap between the services provided by dedicated information systems and general desktop systems for document communication and preparation. This situation is a serious knowledge-management problem, which often results in information loss, poor communication, and confusion among users. Semantic desktops promise to bring knowledge-based services to common desktop applications and, ultimately, to support knowledge management by adding advanced functionality to familiar computing environments. By custom tailoring these systems to different application domains, it is possible to provide dedicated services that assist users in combining document handling and communication with structured workflow processes and the services provided by dedicated systems. This paper presents a model for developing custom-tailored document processing for semantic-desktop systems. Our approach has been applied to the domain of military command and control, which as based on highly-structured document-driven processes. Key Words: semantic desktop, document-driven processes, semantic documents, planning
|
|
| 17. |
|
|
| 18. |
- Leifler, Ola
(författare)
-
Affordances and Constraints of Intelligent Decision Support for Military Command and Control : Three Case Studies of Support Systems
- 2011
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Researchers in military command and control (C2) have for several decades sought to help commanders by introducing automated, intelligent decision support systems. These systems are still not widely used, however, and some researchers argue that this may be due to those problems that are inherent in the relationship between the affordances of technology and the requirements by the specific contexts of work in military C2. In this thesis, we study some specific properties of three support techniques for analyzing and automating aspects of C2 scenarios that are relevant for the contexts of work in which they can be used.The research questions we address concern (1) which affordances and constraints of these technologies are of most relevance to C2, and (2) how these affordances and limitations can be managed to improve the utility of intelligent decision support systems in C2. The thesis comprises three case studies of C2 scenarios where intelligent support systems have been devised for each scenario.The first study considered two military planning scenarios: planning for medical evacuations and similar tactical operations. In the study, we argue that the plan production capabilities of automated planners may be of less use than their constraint management facilities. ComPlan, which was the main technical system studied in the first case study, consisted of a highly configurable, collaborative, constraint-management framework for planning in which constraints could be used either to enforce relationships or notify users of their validity during planning. As a partial result of the first study, we proposed three tentative design criteria for intelligent decision support: transparency, graceful regulation and event-based feedback.The second study was of information management during planning at the operational level, where we used a C2 training scenario from the Swedish Armed Forces and the documents produced during the scenario as a basis for studying properties of Semantic Desktops as intelligent decision support. In the study, we argue that (1) due to the simultaneous use of both documents and specialized systems, it is imperative that commanders can manage information from heterogeneous sources consistently, and (2) in the context of a structurally rich domain such as C2, documents can contain enough information about domain-specific concepts that occur in several applications to allow them to be automatically extracted from documents and managed in a unified manner. As a result of our second study, we present a model for extending a general semantic desktop ontology with domain-specific concepts and mechanisms for extracting and managing semantic objects from plan documents. Our model adheres to the design criteria from the first case study.The third study investigated machine learning techniques in general and text clustering in particular, to support researchers who study team behavior and performance in C2. In this study, we used material from several C2 scenarios which had been studied previously. We interviewed the participating researchers about their work profiles, evaluated machine learning approaches for the purpose of supporting their work and devised a support system based on the results of our evaluations. In the study, we report on empirical results regarding the precision possible to achieve when automatically classifying messages in C2 workflows and present some ramifications of these results on the design of support tools for communication analysis. Finally, we report how the prototype support system for clustering messages in C2 communications was conceived by the users, the utility of the design criteria from case study 1 when applied to communication analysis, and the possibilities for using text clustering as a concrete support tool in communication analysis.In conclusion, we discuss how the affordances and constraints of intelligent decision support systems for C2 relate to our design criteria, and how the characteristics of each work situation demand new adaptations of the way in which intelligent support systems are used.
|
|
| 19. |
- Leifler, Ola, et al.
(författare)
-
Analysis tools in the study of distributed decision-making : a meta-study of command and control research
- 2012
-
Ingår i: Cognition, Technology & Work. - : Springer London. - 1435-5558 .- 1435-5566. ; 14:2, s. 157-168
-
Tidskriftsartikel (refereegranskat)abstract
- Our understanding of distributed decision making in professional teams and their performance comes in part from studies in which researchers gather and process information about the communications and actions of teams. In many cases, the data sets available for analysis are large, unwieldy and require methods for exploratory and dynamic management of data. In this paper, we report the results of interviewing eight researchers on their work process when conducting such analyses and their use of support tools in this process. Our aim with the study was to gain an understanding of their workflow when studying distributed decision making in teams, and specifically how automated pattern extraction tools could be of use in their work. Based on an analysis of the interviews, we elicited three issues of concern related to the use of support tools in analysis: focusing on a subset of data to study, drawing conclusionsfrom data and understanding tool limitations. Together, these three issues point to two observations regarding tool use that are of specific relevance to the design of intelligent support tools based on pattern extraction: open-endedness and transparency.
|
|
| 20. |
- Leifler, Ola, et al.
(författare)
-
Automated text-based analysis for decision-making research
- 2012
-
Ingår i: Cognition, Technology & Work. - : Springer London. - 1435-5558 .- 1435-5566. ; 14:2, s. 129-142
-
Tidskriftsartikel (refereegranskat)abstract
- We present results from a study on constructing and evaluating a support tool for the extraction of patterns in distributed decision -making processes, based on design criteria elicited from a study on the work process involved in studying such decision-making. Specifically, we devised and evaluated an analysis tool for C2 researchers who study simulated decision-making scenarios for command teams. The analysis tool used text clustering as an underlying pattern extraction technique and was evaluated together with C2 researchers in a workshop to establish whether the design criteria were valid and the approach taken with the analysis tool was sound. Design criteria elicited from an earlier study with researchers (open-endedness and transparency) were highly consistent with the results from the workshop. Specifically, evaluation results indicate that successful deployment of advanced analysis tools requires that tools can treat multiple data sources and offer rich opportunities for manipulation and interaction (open-endedness) and careful design of visual presentations and explanations of the techniques used (transparency). Finally, the results point to the high relevance and promise of using text clustering as a support for analysis of C2 data.
|
|
| 21. |
- Leifler, Ola, 1978-
(författare)
-
Combining Technical and Human-Centered Strategies for Decision Support in Command and Control - The ComPlan Approach
- 2008
-
Ingår i: ISCRAM2008 Proceedings of the 5th International ISCRAM Conference. ; , s. 504-515
-
Konferensbidrag (refereegranskat)abstract
- ComPlan (A Combined, Collaborative Command and Control Planning tool) is an approach to providing knowledge-based decision support in the context of command and control. It combines technical research on automated planning tools with human-centered research on mission planning. At its core, ComPlan uses interconnected views of a planning situation to present and manipulate aspects of a scenario. By using domain knowledge flexibly, it presents immediate and directly visible feedback on constraint violations of a plan, facilitates mental simulation of events, and provides support for synchronization of concurrently working mission planners. The conceptual framework of ComPlan is grounded on three main principles from human-centered research on command and control: transparency, graceful regulation, and event-based feedback. As a result, ComPlan provides a model for applying a human-centered perspective on plan authoring tools for command and control, and a demonstration for how to apply that model in an integrated plan-authoring environment.
|
|
| 22. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Critique and Visualization as decision support for mass-casualty emergency management
- 2005
-
Ingår i: Proceedings of the Second International ISCRAM Conference. - Brussels, Belgium : Royal Flemish Academy of Belgium. - 9076971099 - 9789076971094 ; , s. 155-
-
Konferensbidrag (refereegranskat)abstract
- Emergency management in highly dynamic situations consists of exploring options to solve a planning problem. This task can be supported through the use of visual cues that are based on domain knowledge of the current domain. We present an approach to use visualization of critical constraints in timelines and hierarchical views as decision support in mass-casualty emergency situations.
|
|
| 23. |
- Leifler, Ola, et al.
(författare)
-
Curriculum integration of sustainability in engineering education - a national study of programme director perspectives
- 2020
-
Ingår i: International Journal of Sustainability in Higher Education. - : EMERALD GROUP PUBLISHING LTD. - 1467-6370 .- 1758-6739. ; 21:5, s. 877-894
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose This study aims to report on how programme directors address sustainability within engineering education at Swedish universities and engineering colleges. Design/methodology/approach The study was performed as a survey with follow-up interviews around the following core questions: to what extent do programme directors possess a deep understanding of the subject of sustainable development? Which are the core competencies in sustainable development that programme directors identify as important for their engineering students to acquire during their basic training? To what extent are those competencies integrated into engineering education today and what kind of support do programme directors receive from their department to integrate these competencies into the curriculum? Findings Programme directors believe that learning for sustainable development is important mainly based on their personal convictions. However, out of 10 potential learning objectives extracted from the literature, only four-six are implemented in degree programmes. Learning objectives and activities are not always aligned, as students are required to learn about interdisciplinary collaboration without working with students from other faculties. The programme directors receive some support from the department, but they express a need for additional support. Examples of support that they suggest are faculty training, efficient teaching material and incorporation of sustainability in the quality assessment instruments for degree programmes. Originality/value This study is the first comprehensive, national survey of what programme directors think about sustainability in higher education. Their views are important in the attempt to accelerate the integration of sustainability in higher education curricula.
|
|
| 24. |
|
|
| 25. |
- Leifler, Ola, et al.
(författare)
-
Development of Critiquing Systems in Networked Organizations
- 2004
-
Ingår i: Human Error, Safety and Systems Development. - Boston : Springer US. - 9781402081521 ; , s. 31-43
-
Konferensbidrag (refereegranskat)abstract
- Recently, network organizations have been suggested as a solution for future crisis management and warfare. This will, however, have consequences for the development of decision support and critiquing systems. This paper suggests that there are special conditions that need to be taken into account when providing the means for decision-making in networked organizations. Hence, three research problems are suggested that need to be investigated in order to develop useful critiquing systems for future command and control systems.
|
|
| 26. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Domain-specific knowledge management in a Semantic Desktop
- 2009
-
Ingår i: Proceedings of I-KNOW '09 9th International Conference on Knowledge Management and Knowledge Technologies. ; , s. 360-365
-
Konferensbidrag (refereegranskat)abstract
- Semantic Desktops hold promise to provide intelligent information-management environments that can respond to users’ needs. A critical requirement for creating such environments is that the underlying ontology reflects the context of work properly. For specialized work domains where people deal with rich information sources in a context-specific manner, there may be a significant amount of domain-specific information available in text documents, emails and other domain-dependent data sources. We propose that this can be exploited to great effect in a Semantic Desktop to provide much more effective knowledge management. In this paper, we present extensions to an existing semantic desktop through content- and structure-based information extraction, domain-specific ontological extensions as well as visualization of semantic entities. Our extensions are justified by needs in strategic decision making, where domain-specific, well-structured knowledge is available in documents and communications but scattered across the desktop. The consistent and efficient use of these resources by a group of co-workers is critical to success. With a domain-aware semantic desktop, we argue that decision makers will have a much better chance of successful sense making in strategic decision making.
|
|
| 27. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Introduction to the thematic collection
- 2020
-
Ingår i: Högre Utbildning. - : Cappelen Damm Akademisk. - 2000-7558. ; 10:2, s. 50-64
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- In the current thematic collection of Högre Utbildning, we present eight contributions on how to change higher education in ways which will enable students to contribute to a sustainable future, including three invited papers on the systematic transformation of higher education necessary to addressour existential societal challenges.
|
|
| 28. |
- Leifler, Ola, et al.
(författare)
-
Message classification as a basis for studying command and control communication : an evaluation of machine learning approaches
- 2012
-
Ingår i: Journal of Intelligent Information Systems. - Berlin : Springer. - 0925-9902 .- 1573-7675. ; 38:2, s. 299-320
-
Tidskriftsartikel (refereegranskat)abstract
- In military command and control, success relies on being able to perform key functions such as communicating intent. Most staff functions are carried out using standard means of text communication. Exactly how members of staff perform their duties, who they communicate with and how, and how they could perform better, is an area of active research. In command and control research, there is not yet a single model which explains all actions undertaken by members of staff well enough to prescribe a set of procedures for how to perform functions in command and control. In this context, we have studied whether automated classification approaches can be applied to textual communication to assist researchers who study command teams and analyze their actions. Specifically, we report the results from evaluating machine leaning with respect to two metrics of classification performance: (1) the precision of finding a known transition between two activities in a work process, and (2) the precision of classifying messages similarly to human researchers that search for critical episodes in a workflow. The results indicate that classification based on text only provides higher precision results with respect to both metrics when compared to other machine learning approaches, and that the precision of classifying messages using text-based classification in already classified datasets was approximately 50%. We present the implications that these results have for the design of support systems based on machine learning, and outline how to practically use text classification for analyzing team communications by demonstrating a specific prototype support tool for workflow analysis.
|
|
| 29. |
- Leifler, Ola, et al.
(författare)
-
Simulation as a tool for problem detection in rescue operation planning
- 2005
-
Ingår i: Proceedings of the Conference on Modeling and Simulation for Public Safety: SimSafe 2005.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Management and response in the case of an emergency is a very demanding task. Rescue missiuns can involve numerous individuals and teams working together to save lives and property. The outcome of a rescue mission depends to a large extent on the responding units' ability to co-operate and the overall co-ordination of their efforts. This in turn makes it imponant to investigate how to support the decision makers in emergency situations.
|
|
| 30. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Teaching sustainability, ethics and scientific writing : An integrated approach
- 2020
-
Ingår i: Proceedings of 2020 IEEE Frontiers in Education Conference (FIE). - : IEEE. - 9781728189611 - 9781728189628
-
Konferensbidrag (refereegranskat)abstract
- This Innovative Practice Full Paper presents an approach to integrate three critical elements in Computer Science education.The call to imbue computer science graduates with strategic skills needed to address our pressing global sustainability challenges is extremely important, and a great challenge to degree programmes in computer science and software engineering. Doing this successfully requires great care, and possibly several iterations across an entire curriculum. In this regard, learning for sustainability faces similar challenges as understanding scientific results and ethics. Improving skills in searching for, reading, and producing academic texts are often neglected, as are skills in understanding ethics; what norms and values that guide our choices of methods for solving problems. To handle the fact that these subjects (academic writing, ethics and sustainability) are treated separately, and thereby lowering student engagement with the topics, we have successfully integrated them into one coherent subject of Professionalism in Computer Science. By integrating the three subjects, we do three things: a) describe a multi-faceted but integrated engineering role; b) integrate the three aspects of the role we focus on in education and steer away from the view that these are add-ons; and c) increase the motivation of students to take on these aspects of the engineering role.Our approach uses a flipped-classroom style with students playing educational games, participating in discussion seminars and conducting critical analyses of other students’ choices in IT system design. Much emphasis is on the students academic writing abilities, including critical information search and a student peer-review procedure. Also, we do this using an integrated assessment format where teachers from different disciplinary backgrounds jointly assess material from students, which stimulates discussions among ourselves about what and how to assess, and provides a practical way to integrate assessments. We present results from attitude surveys, course evaluations and the contents of the students’ analyses in their final essays. In conclusion, our approach demonstrates a clear shift in how students perceive sustainability, showing that it is possible to achieve changes in attitude towards the subjects as such and their importance for computer scientists.
|
|
| 31. |
- Leifler, Ola, 1978-
(författare)
-
User-centric critiquing in command and control : the DKExpert and ComPlan approaches
- 2007
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis describes two approaches for using critiquing as decision support for military mission planning. In our work, we have drawn from both human-centered research as well as results on decision support systems research for military mission planning when devising approaches for knowledge acquisition and decision support for mission planning.Our two approaches build on a common set of requirements which have been developed as the consequence of both literature analyses as well as interview studies. In short, these criteria state that critiquing systems should be developed with transparency, ease of use and integration in traditional work flow in mind. The use of these criteria is illustrated in two approaches to decision support in two different settings: a collaborative real-time war-gaming simulation and a planning tool for training mission commanders.Our first approach is demonstrated by the DKExpert system, in which end-users can create feedback mechanisms for their own needs when playing a two-sided war-game scenario in the DKE simulation environment. In DKExpert, users can choose to trigger feedback during the game by instructing a rule engine to recognize critical situations. Our second approach, ComPlan, builds on the insights on knowledge and planning representation gained from DKExpert and introduces an explicit representation of planning operations, thereby allowing for better analysis of planning operations and user-controlled feedback. ComPlan also demonstrates a design for critiquing support systems that respects the traditional work practice of mission planners while allowing for intelligent analysis of military plans.
|
|
| 32. |
- Malmqvist, Johan, 1964, et al.
(författare)
-
The CDIO Syllabus 3.0 - An Updated Statement of Goals
- 2022
-
Ingår i: Proceedings of the International CDIO Conference. - : Chalmers University of Technology. - 2002-1593. ; , s. 18-36
-
Konferensbidrag (refereegranskat)abstract
- The CDIO Initiative is going through a process of reconsidering and updating the CDIO approach for engineering education development. Previous work resulted in substantial updates of the twelve CDIO standards and the introduction of “optional” CDIO standards. This paper reports on a similar review and update of the CDIO Syllabus to version 3.0. It has been developed by a working group consisting of four sub-groups and iterated and refined guided by feedback from the whole CDIO community. There are mainly three external drivers that motivate the changes: sustainability, digitalization, and acceleration. There is also an internal driver in the form of lessons learned within the CDIO community, from using the Syllabus in curriculum and course development. Approximately 70 updates are proposed, amongst them three additions on the X.X level, namely 1.4 Knowledge of Social Sciences and Humanities, 3.1 Teamwork and Collaboration, and 5.3 Research.
|
|
| 33. |
- Peters, Anne-Kathrin, Dr. 1984-, et al.
(författare)
-
Sustainability in Computing Education : A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : Association for Computing Machinery (ACM). - 1946-6226. ; 24:1
-
Forskningsöversikt (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
| 34. |
- Peters, Anne-Kathrin, et al.
(författare)
-
Sustainability in Computing Education: A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : ASSOC COMPUTING MACHINERY. - 1946-6226. ; 24:1
-
Forskningsöversikt (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
| 35. |
- Peters, Anne-Kathrin, Dr. 1984-, et al.
(författare)
-
Sustainability in Computing Education: A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : Association for Computing Machinery (ACM). - 1946-6226. ; 24
-
Tidskriftsartikel (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
