| 1. |
|
|
| 2. |
- Ahmad, Azeem, et al.
(författare)
-
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
- 2021
-
Ingår i: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE COMPUTER SOC. - 1530-1362. ; , s. 338-348
-
Konferensbidrag (refereegranskat)abstract
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
|
|
| 3. |
- Ahmad, Azeem, et al.
(författare)
-
An Industrial Study on the Challenges and Effects of Diversity-Based Testing in Continuous Integration
- 2023
-
Ingår i: IEEE International Conference on Software Quality, Reliability and Security, QRS. - 2693-9177. - 9798350319583
-
Konferensbidrag (refereegranskat)abstract
- Many test prioritisation techniques have been proposed in order to improve test effectiveness of Continuous Integration (CI) pipelines. Particularly, diversity-based testing (DBT) has shown promising and competitive results to improve test effectiveness. However, the technical and practical challenges of introducing test prioritisation in CI pipelines are rarely discussed, thus hindering the applicability and adoption of those proposed techniques. This research builds on our prior work in which we evaluated diversity-based techniques in an industrial setting. This work investigates the factors that influence the adoption of DBT both in connection to improvements in test cost-effectiveness, as well as the process and human related challenges to transfer and use DBT prioritisation in CI pipelines. We report on a case study considering the CI pipeline of Axis Communications in Sweden. We performed a thematic analysis of a focus group interview with senior practitioners at the company to identify the challenges and perceived benefits of using test prioritisation in their test process. Our thematic analysis reveals a list of ten challenges and seven perceived effects of introducing test prioritisation in CI cycles. For instance, our participants emphasized the importance of introducing comprehensible and transparent techniques that instill trust in its users. Moreover, practitioners prefer techniques compatible with their current test infrastructure (e.g., test framework and environments) in order to reduce instrumentation efforts and avoid disrupting their current setup. In conclusion, we have identified tradeoffs between different test prioritisation techniques pertaining to the technical, process and human aspects of regression testing in CI. We summarize those findings in a list of seven advantages that refer to specific stakeholder interests and describe the effects of adopting DBT in CI pipelines.
|
|
| 4. |
- Ahmad, Azeem, 1984-
(författare)
-
Contributions to Improving Feedback and Trust in Automated Testing and Continuous Integration and Delivery
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- An integrated release version (also known as a release candidate in software engineering) is produced by merging, building, and testing code on a regular basis as part of the Continuous Integration and Continuous Delivery (CI/CD) practices. Several benefits, including improved software quality and shorter release cycles, have been claimed for CI/CD. On the other hand, recent research has uncovered a plethora of problems and bad practices related to CI/CD adoption, necessitating some optimization. Some of the problems addressed in this work include the ability to respond to practitioners’ questions and obtain quick and trustworthy feedback in CI/CD. To be more specific, our effort concentrated on: 1) identifying the information needs of software practitioners engaged in CI/CD; 2) adopting test optimization approaches to obtain faster feedback that are realistic for use in CI/CD environments without introducing excessive technical requirements; 3) identifying perceived causes and automated root cause analysis of test flakiness, thereby providing developers with guidance on how to resolve test flakiness; and 4) identifying challenges in addressing information needs, providing faster and more trustworthy feedback. The findings of the research reported in this thesis are based on data from three single-case studies and three multiple-case studies. The research uses quantitative and qualitative data collected via interviews, site visits, and workshops. To perform our analyses, we used data from firms producing embedded software as well as open-source repositories. The following are major research and practical contributions. Information Needs: The initial contribution to research is a list of information needs in CI/CD. This list contains 27 frequently asked questions on continuous integration and continuous delivery by software practitioners. The identified information needs have been classified as related to testing, code & commit, confidence, bug, and artifacts. We investigated how companies deal with information needs, what tools they use to deal with them, and who is interested in them. We concluded that there is a discrepancy between the identified needs and the techniques employed to meet them. Since some information needs cannot be met by current tools, manual inspections are required, which adds time to the process. Information about code & commit, confidence level, and testing is the most frequently sought for and most important information. Evaluation of Diversity Based Techniques/Tool: The contribution is to conduct a detailed examination of diversity-based techniques using industry test cases to determine if there is a difference between diversity functions in selecting integrationlevel automated test. Additionally, how diversity-based testing compares to other optimization techniques used in industry in terms of fault detection rates, feature coverage, and execution time. This enables us to observe how coverage changes when we run fewer test cases. We concluded that some of the techniques can eliminate up to 85% of test cases (provided by the case company) while still covering all distinct features/requirements. The techniques are developed and made available as an open-source tool for further research and application. Test Flakiness Detection, Prediction & Automated Root Cause Analysis: We identified 19 factors that professionals perceive affect test flakiness. These perceived factors are divided into four categories: test code, system under test, CI/test infrastructure, and organizational. We concluded that some of the perceived factors of test flakiness in closed-source development are directly related to non-determinism, whereas other perceived factors concern different aspects e.g., lack of good properties of a test case (i.e., small, simple and robust), deviations from the established processes, etc. To see if the developers’ perceptions were in line with what they had labelled as flaky or not, we examined the test artifacts that were readily available. We verified that two of the identified perceived factors (i.e., test case size and simplicity) are indeed indicative of test flakiness. Furthermore, we proposed a light weight technique named trace-back coverage to detect flaky tests. Trace-back coverage was combined with other factors such as test smells indicating test flakiness, flakiness frequency and test case size to investigate the effect on revealing test flakiness. When all factors are taken into consideration, the precision of flaky test detection is increased from 57% (using single factor) to 86% (combination of different factors).
|
|
| 5. |
- Ahmad, Azeem, et al.
(författare)
-
Data visualisation in continuous integration and delivery : Information needs, challenges, and recommendations
- 2022
-
Ingår i: IET Software. - : WILEY. - 1751-8806 .- 1751-8814. ; 16:3, s. 331-349
-
Tidskriftsartikel (refereegranskat)abstract
- Several operations, ranging from regular code updates to compiling, building, testing, and distribution to customers, are consolidated in continuous integration and delivery. Professionals seek additional information to complete the mission at hand during these tasks. Developers who devote a large amount of time and effort to finding such information may become distracted from their work. We will better understand the processes, procedures, and resources used to deliver a quality product on time by defining the types of information that software professionals seek. A deeper understanding of software practitioners information needs has many advantages, including remaining competitive, growing knowledge of issues that can stymie a timely update, and creating a visualisation tool to assist practitioners in addressing their information needs. This is an extension of a previous work done by the authors. The authors conducted a multiple-case holistic study with six different companies (38 unique participants) to identify information needs in continuous integration and delivery. This study attempts to capture the importance, frequency, required effort (e.g. sequence of actions required to collect information), current approach to handling, and associated stakeholders with respect to identified needs. 27 information needs associated with different stakeholders (i.e. developers, testers, project managers, release team, and compliance authority) were identified. The identified needs were categorised as testing, code & commit, confidence, bug, and artefacts. Apart from identifying information needs, practitioners face several challenges in developing visualisation tools. Thus, 8 challenges that were faced by the practitioners to develop/maintain visualisation tools for the software team were identified. The recommendations from practitioners who are experts in developing, maintaining, and providing visualisation services to the software team were listed.
|
|
| 6. |
- Ahmad, Azeem, et al.
(författare)
-
Empirical analysis of practitioners perceptions of test flakiness factors
- 2021
-
Ingår i: Software testing, verification & reliability. - : Wiley-Blackwell. - 0960-0833 .- 1099-1689. ; 31:8
-
Tidskriftsartikel (refereegranskat)abstract
- Identifying the root causes of test flakiness is one of the challenges faced by practitioners during software testing. In other words, the testing of the software is hampered by test flakiness. Since the research about test flakiness in large-scale software engineering is scarce, the need for an empirical case-study where we can build a common and grounded understanding of the problem as well as relevant remedies that can later be evaluated in a large-scale context is a necessity. This study reports the findings from a multiple-case study. The authors conducted an online survey to investigate and catalogue the root causes of test flakiness and mitigation strategies. We attempted to understand how practitioners perceive test flakiness in closed-source development, such as how they define test flakiness and what practitioners perceive can affect test flakiness. The perceptions of practitioners were compared with the available literature. We investigated whether practitioners perceptions are reflected in the test artefacts such as what is the relationship between the perceived factors and properties of test artefacts. This study reported 19 factors that are perceived by professionals to affect test flakiness. These perceived factors are categorized as test code, system under test, CI/test infrastructure, and organization-related. The authors concluded that some of the perceived factors in test flakiness in closed-source development are directly related to non-determinism, whereas other perceived factors concern different aspects, for example, lack of good properties of a test case, deviations from the established processes, and ad hoc decisions. Given a data set from investigated cases, the authors concluded that two of the perceived factors (i.e., test case size and test case simplicity) have a strong effect on test flakiness.
|
|
| 7. |
- Ahmad, Azeem, et al.
(författare)
-
Identifying Randomness related Flaky Tests through Divergence and Execution Tracing
- 2022
-
Ingår i: 2022 IEEE 15TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW 2022). - : IEEE COMPUTER SOC. - 9781665496285 ; , s. 293-300
-
Konferensbidrag (refereegranskat)abstract
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, known as non-deterministic tests, change their outcomes without any changes in the codebase, thus reducing the trust of developers during a software release as well as in the quality of a product. While rerunning test cases is a common approach, it is resource intensive, unreliable, and does not uncover the actual cause of test flakiness. Our paper evaluates an approach to identify randomness-related flaky. This paper used a divergence algorithm and execution tracing techniques to identify flaky tests, which resulted in the FLAKYPY prototype. In addition, this paper discusses the cases where FLAKYPY successfully identified the flaky test as well as those cases where FLAKYPY failed. The papers discuss how the reporting mechanism of FLAKYPY can help developers in identifying the root cause of randomness-related test flakiness. Thirty-two open-source projects were used in this. We concluded that FLAKYPY can detect most of the randomness-related test flakiness. In addition, the reporting mechanism of FLAKYPY reveals sufficient information about possible root causes of test flakiness.
|
|
| 8. |
- Duboc, Leticia, et al.
(författare)
-
Requirements engineering for sustainability: an awareness framework for designing software systems for a better tomorrow
- 2020
-
Ingår i: Requirements Engineering. - : Springer Science and Business Media LLC. - 0947-3602 .- 1432-010X. ; 25:4, s. 469-492
-
Tidskriftsartikel (refereegranskat)abstract
- Integrating novel software systems in our society, economy and environment can have far-reaching effects. As a result, software systems should be designed in such a way as to maintain or improve the sustainability of their intended socio-technical systems. However, a paradigm shift is required to raise awareness of software professionals on the potential sustainability effects of software systems. While Requirements Engineering is considered the key for driving this change, requirements engineers lack the knowledge, experience and methodological support for acting as facilitators for a broader discussion on sustainability effects. This paper presents a question-based framework for raising awareness of the potential effects of software systems on sustainability, as the first step towards enabling the required paradigm shift. An evaluation study of the framework was conducted with four groups of computer science students. The results of the study indicate that the framework is applicable to different types of systems and helps to facilitate discussions about the potential effects that software systems could have on sustainability.
|
|
| 9. |
- Eidenskog, Maria, 1981-, et al.
(författare)
-
Changing the world one engineer at a time – unmaking the traditional engineering education when introducing sustainability subjects
- 2023
-
Ingår i: International Journal of Sustainability in Higher Education. - : EMERALD GROUP PUBLISHING LTD. - 1467-6370 .- 1758-6739. ; 24:9, s. 70-84
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose: The information technology (IT) sector has been seen as central to society's transformation to a more just and sustainable society, which underlines teachers’ responsibility to foster engineers who can contribute specifically to such ends. This study aims to report an effort to significantly update an existing engineering programme in IT with this ambition and to analyse the effects and challenges associated with the transformation.Design/methodology/approach: This study is based on a combination of action-oriented research based on implementing key changes to the curriculum; empirical investigations including surveys and interviews with students and teachers, and analysis of these; and a science and technology studies-inspired analysis.Findings: Respondents were generally positive towards adding topics relating to sustainability. However, in the unmaking of traditional engineering subjects, changes created a conflict between core versus soft subjects in which the core subjects tended to gain the upper hand. This conflict can be turned into productive discussions by focusing on what kinds of engineers the authors’ educate and how students can be introduced to societal problems as an integrated part of their education.Practical implications: This study can be helpful for educators in the engineering domain to support them in their efforts to transition from a (narrow) focus on traditional disciplines to one where the bettering of society is at the core.Originality/value: This study provides a novel approach to the transformation of engineering education through a theoretical analysis seldom used in studies of higher education on a novel case study.
|
|
| 10. |
- Johansson, Björn, Professor, PhD, 1973-, et al.
(författare)
-
Understanding the challenge of the energy crisis : Tackling system complexity with megagaming
- 2023
-
Ingår i: ECCE '23: Proceedings of the European Conference on Cognitive Ergonomics 2023. - New York, NY United States : ACM Digital Library. - 9798400708756
-
Konferensbidrag (refereegranskat)abstract
- Climate change is tightly coupled to the usage of energy in differentforms. Creating and using sustainable energy solutions is increasinglybeing recognized as the greatest challenge of our time. Theenergy crisis concerns everyone who consumes energy or servicesthat rely on energy to be performed, in practice almost everythingwe buy. However, the temporal and psychological distance to climatechange has led to a situation where the consequences of ourenergy usage often is considered a future problem or “someoneelse’s” problem. Current methods to increase awareness and understandingof climate change and the need to create sustainableenergy solutions are usually based on simulation or negotiationgames. Methods similar to crisis management exercises are uncommon.In this paper, we propose to use Megagames, large scale socialgames, as a method for increasing the awareness of the challengesrelated to creating sustainable energy solutions. Examples fromtwo Megagame test runs are provided and discussed.
|
|
| 11. |
- Leifler, Ola, et al.
(författare)
-
Curriculum integration of sustainability in engineering education - a national study of programme director perspectives
- 2020
-
Ingår i: International Journal of Sustainability in Higher Education. - : EMERALD GROUP PUBLISHING LTD. - 1467-6370 .- 1758-6739. ; 21:5, s. 877-894
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose This study aims to report on how programme directors address sustainability within engineering education at Swedish universities and engineering colleges. Design/methodology/approach The study was performed as a survey with follow-up interviews around the following core questions: to what extent do programme directors possess a deep understanding of the subject of sustainable development? Which are the core competencies in sustainable development that programme directors identify as important for their engineering students to acquire during their basic training? To what extent are those competencies integrated into engineering education today and what kind of support do programme directors receive from their department to integrate these competencies into the curriculum? Findings Programme directors believe that learning for sustainable development is important mainly based on their personal convictions. However, out of 10 potential learning objectives extracted from the literature, only four-six are implemented in degree programmes. Learning objectives and activities are not always aligned, as students are required to learn about interdisciplinary collaboration without working with students from other faculties. The programme directors receive some support from the department, but they express a need for additional support. Examples of support that they suggest are faculty training, efficient teaching material and incorporation of sustainability in the quality assessment instruments for degree programmes. Originality/value This study is the first comprehensive, national survey of what programme directors think about sustainability in higher education. Their views are important in the attempt to accelerate the integration of sustainability in higher education curricula.
|
|
| 12. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Introduction to the thematic collection
- 2020
-
Ingår i: Högre Utbildning. - : Cappelen Damm Akademisk. - 2000-7558. ; 10:2, s. 50-64
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- In the current thematic collection of Högre Utbildning, we present eight contributions on how to change higher education in ways which will enable students to contribute to a sustainable future, including three invited papers on the systematic transformation of higher education necessary to addressour existential societal challenges.
|
|
| 13. |
- Leifler, Ola, 1978-, et al.
(författare)
-
Teaching sustainability, ethics and scientific writing : An integrated approach
- 2020
-
Ingår i: Proceedings of 2020 IEEE Frontiers in Education Conference (FIE). - : IEEE. - 9781728189611 - 9781728189628
-
Konferensbidrag (refereegranskat)abstract
- This Innovative Practice Full Paper presents an approach to integrate three critical elements in Computer Science education.The call to imbue computer science graduates with strategic skills needed to address our pressing global sustainability challenges is extremely important, and a great challenge to degree programmes in computer science and software engineering. Doing this successfully requires great care, and possibly several iterations across an entire curriculum. In this regard, learning for sustainability faces similar challenges as understanding scientific results and ethics. Improving skills in searching for, reading, and producing academic texts are often neglected, as are skills in understanding ethics; what norms and values that guide our choices of methods for solving problems. To handle the fact that these subjects (academic writing, ethics and sustainability) are treated separately, and thereby lowering student engagement with the topics, we have successfully integrated them into one coherent subject of Professionalism in Computer Science. By integrating the three subjects, we do three things: a) describe a multi-faceted but integrated engineering role; b) integrate the three aspects of the role we focus on in education and steer away from the view that these are add-ons; and c) increase the motivation of students to take on these aspects of the engineering role.Our approach uses a flipped-classroom style with students playing educational games, participating in discussion seminars and conducting critical analyses of other students’ choices in IT system design. Much emphasis is on the students academic writing abilities, including critical information search and a student peer-review procedure. Also, we do this using an integrated assessment format where teachers from different disciplinary backgrounds jointly assess material from students, which stimulates discussions among ourselves about what and how to assess, and provides a practical way to integrate assessments. We present results from attitude surveys, course evaluations and the contents of the students’ analyses in their final essays. In conclusion, our approach demonstrates a clear shift in how students perceive sustainability, showing that it is possible to achieve changes in attitude towards the subjects as such and their importance for computer scientists.
|
|
| 14. |
- Malmqvist, Johan, 1964, et al.
(författare)
-
The CDIO Syllabus 3.0 - An Updated Statement of Goals
- 2022
-
Ingår i: Proceedings of the International CDIO Conference. - : Chalmers University of Technology. - 2002-1593. ; , s. 18-36
-
Konferensbidrag (refereegranskat)abstract
- The CDIO Initiative is going through a process of reconsidering and updating the CDIO approach for engineering education development. Previous work resulted in substantial updates of the twelve CDIO standards and the introduction of “optional” CDIO standards. This paper reports on a similar review and update of the CDIO Syllabus to version 3.0. It has been developed by a working group consisting of four sub-groups and iterated and refined guided by feedback from the whole CDIO community. There are mainly three external drivers that motivate the changes: sustainability, digitalization, and acceleration. There is also an internal driver in the form of lessons learned within the CDIO community, from using the Syllabus in curriculum and course development. Approximately 70 updates are proposed, amongst them three additions on the X.X level, namely 1.4 Knowledge of Social Sciences and Humanities, 3.1 Teamwork and Collaboration, and 5.3 Research.
|
|
| 15. |
- Peters, Anne-Kathrin, Dr. 1984-, et al.
(författare)
-
Sustainability in Computing Education : A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : Association for Computing Machinery (ACM). - 1946-6226. ; 24:1
-
Forskningsöversikt (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
| 16. |
- Peters, Anne-Kathrin, et al.
(författare)
-
Sustainability in Computing Education: A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : ASSOC COMPUTING MACHINERY. - 1946-6226. ; 24:1
-
Forskningsöversikt (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
| 17. |
- Peters, Anne-Kathrin, Dr. 1984-, et al.
(författare)
-
Sustainability in Computing Education: A Systematic Literature Review
- 2024
-
Ingår i: ACM Transactions on Computing Education. - : Association for Computing Machinery (ACM). - 1946-6226. ; 24
-
Tidskriftsartikel (refereegranskat)abstract
- Research shows that the global society as organized today, with our current technological and economic system, is impossible to sustain. We are living in an era in which human activities in highly industrialized countries are responsible for overshooting several planetary boundaries, with poorer communities contributing the least to the problems but being impacted the most. At the same time, technical and economic gains fail to provide society at large with equal opportunities and improved quality of life. This article describes approaches taken in computing education to address the issue of sustainability. It presents results of a systematic review of the literature on sustainability in computing education. From a set of 572 publications extracted from six large digital libraries plus snowballing, we distilled and analyzed 89 relevant primary studies. Using an inductive and deductive thematic analysis, we study (i) conceptions of sustainability, computing, and education; (ii) implementations of sustainability in computing education; and (iii) research on sustainability in computing education. We present a framework capturing learning objectives and outcomes as well as pedagogical methods for sustainability in computing education. These results can be mapped to existing standards and curricula in future work. We find that only a few of the articles engage with the challenges as calling for drastic systemic change, along with radically new understandings of computing and education. We suggest that future work should connect to the substantial body of critical theory, such as feminist theories of science and technology. Existing research on sustainability in computing education may be considered rather immature, as the majority of articles are experience reports with limited empirical research.
|
|
