| 1. |
- Li, Xiu, 1982-, et al.
(författare)
-
Linking Swedish Learning Materials to Exercises through an AI-Enhanced Recommender System
- 2023
-
Ingår i: Methodologies and Intelligent Systems for Technology Enhanced Learning, 13th International Conference. - Cham : Springer. - 9783031412257 - 9783031412264 ; , s. 96-107
-
Konferensbidrag (refereegranskat)abstract
- As an integral part of AI-enhanced learning, a content recommender automatically filters and recommends relevant learning materials to the learner or the instructor in a learning system. It can effectively help instructors in pedagogical practices and support students in self-regulated learning. Content recommendation technologies and applications have been studied extensively, however, the SOTA technologies have not adequately adapted to the education domain and there is very limited research on how different models and solutions can be applied in the Swedish context and for multiple subjects. In this paper, we develop a text similarity-based content recommender system. Specifically, given a quiz, we automatically recommend supportive learning resources as a reference to the answer and link back to the textbook sections where the examined knowledge points reside. We present a generic method for Swedish educational content recommendations using the most representative models, evaluate and analyze in multi-dimensions such as model types, pooling methods, subjects etc. The best results are obtained by Sentence-BERT (SBERT) with max paragraph-level pooling, outperforming traditional Natural Language Processing (NLP) models and knowledge graph-based models, obtaining on average 95% in Recall@3 and 82% in MRR, and outstanding in dealing with texts containing symbols, equations or calculations. This research provides empirical evidence and analysis, and can be used as a guidance when building a Swedish educational content recommender.
|
|
| 2. |
- Li, Xiu, 1982-, et al.
(författare)
-
Supporting Teaching-to-the-Curriculum by Linking Diagnostic Tests to Curriculum Goals : Using Textbook Content as Context for Retrieval-Augmented Generation with Large Language Models
- 2024
-
Ingår i: Artificial Intelligence in Education. - : Springer Nature. - 9783031643026 - 9783031643019 ; , s. 118-132
-
Konferensbidrag (refereegranskat)abstract
- Using AI for automatically linking exercises to curriculum goals can support many educational use cases and facilitate teaching-to-the-curriculum by ensuring that exercises adequately reflect and encompass the curriculum goals, ultimately enabling curriculum-based assessment. Here, we introduce this novel task and create a manually labeled dataset where two types of diagnostic tests are linked to curriculum goals for Biology G7-9 in Sweden. We cast the problem both as an information retrieval task and a multi-class text classification task and explore unsupervised approaches to both, as labeled data for such tasks is typically scarce. For the information retrieval task, we employ SOTA embedding model ADA-002 for semantic textual similarity (STS), while we prompt a large language model in the form of ChatGPT to classify diagnostic tests into curriculum goals. For both task formulations, we investigate different ways of using textbook content as a pivot and provide additional context for linking diagnostic tests to curriculum goals. We show that a combination of the two approaches in a retrieval-augmented generation model, whereby STS is used for retrieving textbook content as context to ChatGPT that then performs zero-shot classification, leads to the best classification accuracy (73.5%), outperforming both STS-based classification (67.5%) and LLM-based classification without context (71.5%). Finally, we showcase how the proposed method could be used in pedagogical practices.
|
|
| 3. |
- Wu, Yongchao, 1987-, et al.
(författare)
-
Towards Data-effective Educational Question Generation with Prompt-based Learning
- 2023
-
Konferensbidrag (refereegranskat)abstract
- Practice and exam-style questions, as essential educational tools, contribute to educators’ effective teaching. Automatic question generation (QG) is a promising technique that can eliminate the manual effort of constructing questions and boost technology-enhanced education systems. Recently, deep neural network-based question-generation approaches have significantly improved upon state-of-the-art of question generation. Nevertheless, these approaches are often developed based on huge and non-educational datasets consisting of over 100,000 examples, which negatively affect the scalability and reliability of the educational QG systems. This study proposes a prompt-based learning QG approach that could generate questions in a data-effective way. The proposed prompt-based learning QG approach is trained and evaluated on a general dataset SQuAD, and an educational dataset SciQ. Experiment results demonstrate that our approach outperforms existing best QG models by a vast margin in data-effective scenarios and could generate high-quality educational questions with as few as 1,000 training examples.
|
|
| 4. |
- Afzaal, Muhammad, 1989-
(författare)
-
Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Student Self-Regulation
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Self-regulated learning (SRL) is a cognitive ability with demonstrable significance in facilitating students’ ability to effectively strategize, monitor, and assess their own learning actions. Studies have indicated that a lack of selfregulated learning skills negatively impacts students’ academic performance. Effective data-driven feedback and action recommendations are considered crucial for SRL and significantly influence student learning and performance. However, the task of delivering personalised feedback to every student poses a significant challenge for teachers. Moreover, the task of identifying appropriate learning activities and resources for individualised recommendations poses a significant challenge for teachers, given the large number of students enrolled in most courses.To address these challenges, several studies have examined how learning analytics-based dashboards can support students’ self-regulation. These dashboards offered several visualisations (as feedback) on student success and failure. However, while such feedback may be beneficial, it does not offer insightful information or actionable recommendations to help students improve academically. Explainable artificial intelligence (xAI) approaches have been proposed to explain such feedback and generate insights from predictive models, with a focus on the relevant actions a student needs to take to improve in ongoing courses. Such intelligent activities could be offered to students as data-driven behavioural change recommendations.This thesis offers an xAI-based approach that predicts course performance and computes informative feedback and actionable recommendations to promote student self-regulation. Unlike previous research, this thesis integrates a predictive approach with an xAI approach to analyse and manipulate students’ learning trajectories. The aim is to offer detailed, data-driven actionable feedback to students by providing in-depth insights and explanations for the predictions provided by the approach. The technique provides students with more practical and useful knowledge compared to the predictions alone.The proposed approach was implemented in the form of a dashboard to support self-regulation by students in university courses, and it was evaluated to determine its effects on the students’ academic performance. The results revealed that the dashboard significantly enhanced students’ learning achievements and improved their self-regulated learning skills. Furthermore, it was found that the recommendations generated by the proposed approach positively affected students’ performance and assisted them in self-regulation.
|
|
| 5. |
- Afzaal, Muhammad, et al.
(författare)
-
Informative Feedback and Explainable AI-Based Recommendations to Support Students' Self-regulation
- 2024
-
Ingår i: Technology, Knowledge and Learning. - 2211-1662 .- 2211-1670. ; 29:1, s. 331-354
-
Tidskriftsartikel (refereegranskat)abstract
- Self-regulated learning is an essential skill that can help students plan, monitor, and reflect on their learning in order to achieve their learning goals. However, in situations where there is a lack of effective feedback and recommendations, it becomes challenging for students to self-regulate their learning. In this paper, we propose an explainable AI-based approach to provide automatic and intelligent feedback and recommendations that can support the self-regulation of students' learning in a data-driven manner, with the aim of improving their performance on their courses. Prior studies have predicted students' performance and have used these predicted outcomes as feedback, without explaining the reasons behind the predictions. Our proposed approach is based on an algorithm that explains the root causes behind a decline in student performance, and generates data-driven recommendations for taking appropriate actions. The proposed approach was implemented in the form of a dashboard to support self-regulation by students on a university course, and was evaluated to determine its effects on the students' academic performance. The results revealed that the dashboard significantly enhanced students' learning achievements and improved their self-regulated learning skills. Furthermore, it was found that the recommendations generated by the proposed approach positively affected students' performance and assisted them in self-regulation
|
|
| 6. |
- Hegestedt, Robert, et al.
(författare)
-
Data-driven school improvement and data-literacy in K-12 : Findings from a Swedish national program
- 2023
-
Ingår i: International Journal. - : International Association of Online Engineering. - 1868-8799 .- 1863-0383. ; 18:15, s. 189-208
-
Tidskriftsartikel (refereegranskat)abstract
- Data-driven school improvement has been proposed to improve and support edu-cational practices and more studies are emerging describing data-driven practices in schools and the effects of data-driven interventions. This paper reports on a study that has taken place within a national program where 15 schools from six different municipalities and organizations are working at classroom, school and municipality levels to improve educational practices using data-driven methods. The study aimed at understanding what educational problems teachers, principals and administrative staff in the project aimed to address through the utilization of data-driven methods and the challenges they face in doing so. Using a mixed method design, we identified four thematic areas that reflect the focused problem areas of the participants in the project, namely didactics, democracy, assessment and planning, and mental health. All development groups identified problems that can be solved with data-driven methods. Along with this, we also identified five challenges faced by the participants: time and resources, competence, ethics, digi-tal systems and common language. We conclude that the main challenge faced by the participants is data literacy, and that professional development is needed to support effective and successful data-driven practices in schools.
|
|
| 7. |
- Li, Xiu, et al.
(författare)
-
Automatic Educational Concept Extraction Using NLP
- 2022
-
Ingår i: Methodologies and Intelligent Systems for Technology Enhanced Learning, 12th International Conference. - Cham : Springer Nature. - 9783031206177 - 9783031206160 ; , s. 133-138
-
Konferensbidrag (refereegranskat)abstract
- Educational concepts are the core of teaching and learning. From the perspective of educational technology, concepts are essential meta-data, represen- tative terms that can connect different learning materials, and are the foundation for many downstream tasks. Some studies on automatic concept extraction have been conducted, but there are no studies looking at the K-12 level and focused on the Swedish language. In this paper, we use a state-of-the-art Swedish BERT model to build an automatic concept extractor for the Biology subject using fine- annotated digital textbook data that cover all content for K-12. The model gives a recall measure of 72% and has the potential to be used in real-world settings for use cases that require high recall. Meanwhile, we investigate how input data fea- tures influence model performance and provide guidance on how to effectively use text data to achieve the optimal results when building a named entity recognition (NER) model.
|
|
| 8. |
|
|
| 9. |
- Nouri, Jalal, et al.
(författare)
-
Efforts in Europe for Data-Driven Improvement of Education – A review of learning analytics research in seven countries
- 2019
-
Ingår i: International journal of learning analytics and artificial intelligence for education. - : International Association of Online Engineering (IAOE). - 2706-7564. ; 1:1, s. 8-27
-
Tidskriftsartikel (refereegranskat)abstract
- Information and communication technologies are increasingly mediating learning and teaching practices as well as how educational institutions are handling their administrative work. As such, students and teachers are leaving large amounts of digital footprints and traces in various educational apps and learning management platforms, and educational administrators register various processes and outcomes in digital administrative systems. It is against such a background we in recent years have seen the emergence of the fast-growing and multi-disciplinary field of learning analytics. In this paper, we examine the research efforts that have been conducted in the field of learning analytics in Austria, Denmark, Finland, Norway, Germany, Spain, and Sweden. More specifically, we report on developed national policies, infrastructures and competence centers, as well as major research projects and developed research strands within the selected countries. The main conclusions of this paper are that the work of researchers around Europe has not led to national adoption or European level strategies for learning analytics. Furthermore, most countries have not established national policies for learners’ data or guidelines that govern the ethical usage of data in research or education. We also conclude, that learning analytics research on pre-university level to high extent have been overlooked. In the same vein, learning analytics has not received enough focus form national and European national bodies. Such funding is necessary for taking steps towards data-driven development of education.
|
|
| 10. |
- Nouri, Jalal, 1982-
(författare)
-
Orchestrating scaffolded outdoor mobile learning activities
- 2014
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Since the beginning of time, technological innovations have formed the basis for the development of society and supported the most fundamental societal features. The educational system is no exception. This we have witnessed on many occasions, as for example in form of the transformations of learning and teaching introduced by the printing press, the calculator and computers. With the advance of mobile technology, we have received another technology that inspires research fields to study the learning and teaching potentials that mobile technology may present. It is from here this thesis takes its general starting point, namely, in the determination to critically examine the role mobile technology can play in supporting outdoor learning activities. More specifically, the thesis attempts to, on the one hand, develop an understanding of the challenges and limitations associated with scaffolding students’ mobile learning in outdoor environments. On the other hand, based on such a developed understanding, the thesis investigates how mobile technology-supported outdoor activities should be orchestrated to scaffold students learning. Orchestration is, in this thesis, understood as the process of productively coordinating supportive interventions across multiple learning activities occurring at multiple social levels involving multiple contexts, and multiple tools and media.The framework of design-based research has guided the methodological approach. Three design studies formed the empirical basis of the study of the issues. The results of the thesis indicate the difficulties and challenges in supporting students in outdoor contexts and delineate an understanding of how mobile outdoor learning activities can be orchestrated with students scaffolding needs taken into account.The thesis contributes with a conceptualization of and a model for orchestration of mobile learning activities, a framework for design-based research in mobile learning, as well as a critical perspective on the introduction of mobile technology in education.
|
|
| 11. |
- Saqr, Mohammed, et al.
(författare)
-
How networking and social capital influence performance : the role of long-term ties
- 2021
-
Ingår i: Networks in the Global World V: Proceedings of NetGloW 2020. - Cham : Springer. - 9783030648770 - 9783030648763 ; , s. 335-346
-
Konferensbidrag (refereegranskat)abstract
- Recently, students have become networked in many ways, and evidence is mounting that networking plays a significant role in how students learn, interact, and transfer information. Relationships could translate to opportunities; resources and support that help achieve the pursued goals and objectives. Although students exist, interact, and play different roles within social and information networks, networks have not received the due attention. This research aimed to study medical student’s friendship- and information exchange networks as well as assess the correlation between social capital and network position variables and the cumulative Grade Point Average (GPA) which is the average grade obtained over all the years. The relationships considered in our study are the long-term face-to-face and online ties that developed over the full duration of the study in the medical college. More specifically, we have studied face-to-face and information networks. Analysis of student’s networks included a combination of visual and social network analysis. The correlation with the performance was performed using resampling permutation correlation coefficient, linear regression, and 10-fold cross-validation of binary logistic regression. The results of correlation and linear regression tests demonstrated that student’s social capital was correlated with performance. The most significant factors were the power of close friends regarding connectedness and achievement scores. These findings were evident in the close friends’ network and the information network. The results of this study highlight the importance of social capital and networking ties in medical schools and the need to consider peer dynamics in class assignment and support services
|
|
| 12. |
- Scruggs, Richard, et al.
(författare)
-
Using Knowledge Tracing to Predict Students’ Performance in Cognitive Training and Math
- 2022
-
Ingår i: Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. - Cham : Springer. - 9783031116469 - 9783031116476 ; , s. 410-413
-
Bokkapitel (refereegranskat)abstract
- Cognitive training aims to improve skills such as working memory capacity and spatial ability, which have been linked to math skills. In this study, we fit Deep Knowledge Tracing with Transformers (DKTT), Dynamic Key-Value Memory Networks (DKVMN), and Knowledge Tracing Machines (KTM) to a large dataset from a cognitive training system. DKVMN achieved the highest AUC (0.739) of the algorithms. To explore connections between math skills and cognitive skills, the data was split into cognitive and math items. DKVMN’s AUC on the math items was higher (0.745) than on the cognitive (0.706). Notably, the split model AUCs did not differ from skill-level AUCs produced by a model trained on the entire dataset, suggesting that math performance did not improve DKVMN’s cognitive predictions and vice versa.
|
|
| 13. |
- Wickberg Hugerth, Mattias, et al.
(författare)
-
"I Should, but I Don't Feel Like It" : Overcoming Obstacles in Upper Secondary Students' Self-regulation Using Learning Analytics
- 2024
-
Ingår i: Studia Paedagogica. - 1803-7437 .- 2336-4521. ; 28:3, s. 89-111
-
Tidskriftsartikel (refereegranskat)abstract
- Även om forskning har bedrivits om självreglerat lärande i relation till lärandeanalys finns det fortfarande en kunskapslucka när det gäller de hinder som elever i gymnasieutbildningen möter i att reglera sitt eget lärande och hur lärandeanalys kan stödja deras självreglering. Denna artikel undersöker två frågor: 1) Vilka utmaningar upplever gymnasieelever i processen att reglera sitt eget lärande?, och 2) Vilken information och data behöver gymnasieelever för att bättre kunna reglera sitt eget lärande? Vi genomförde en studie på en medelstor gymnasieskola i Mellansverige för att bättre förstå hur dessa frågor manifesterar sig bland eleverna. Vi analyserade data som samlats in av skolan två gånger årligen mellan 2015 och 2022 och administrerade ett frågeformulär till 224 elever för att besvara forskningsfrågorna. Genom beskrivande statistik och en tematisk analys identifierar vi vanliga problem som elever stöter på samt den information som är nödvändig för att stötta självreglerat lärande. Vi diskuterar implikationerna av våra fynd för utformningen av system som förser elever med relevant data för att förbättra deras lärandeupplevelser.
|
|
| 14. |
- Wu, Yongchao, et al.
(författare)
-
Beyond Benchmarks : Spotting Key Topical Sentences While Improving Automated Essay Scoring Performance with Topic-Aware BERT
- 2023
-
Ingår i: Electronics. - : MDPI AG. - 2079-9292. ; 12:1
-
Tidskriftsartikel (refereegranskat)abstract
- Automated Essay Scoring (AES) automatically allocates scores to essays at scale and may help teachers reduce the heavy burden during grading activities. Recently, researchers have deployed neural-based AES approaches to improve upon the state-of-the-art AES performance. These neural-based AES methods mainly take student essays as the sole input and focus on learning the relationship between student essays and essay scores through deep neural networks. However, their only product, the predicted holistic score, is far from providing adequate pedagogical information, such as automated writing evaluation (AWE). In this work, we propose Topic-aware BERT, a new method of learning relations among scores, student essays, as well as topical information in essay instructions. Beyond improving the AES benchmark performance, Topic-aware BERT can automatically retrieve key topical sentences in student essays by probing self-attention maps in intermediate layers. We evaluate the performance of Topic-aware BERT of different variants to (i) perform AES and (ii) retrieve key topical sentences using the open dataset Automated Student Assessment Prize and a manually annotated dataset. Our experiments show that Topic-aware BERT achieves a strong AES performance compared with the previous best neural-based AES methods and demonstrates effectiveness in identifying key topical sentences in argumentative essays.
|
|
| 15. |
- Wu, Yongchao, et al.
(författare)
-
Retrieving Key Topical Sentences With Topic-aware BERT when Conducting Automated Essay Scoring
- 2022
-
Ingår i: Methodologies and Intelligent Systems for Technology Enhanced Learning, 12th International Conference. - Cham : Springer Nature. ; , s. 123-132
-
Konferensbidrag (refereegranskat)abstract
- Automated Essay Scoring (AES) automatically assigns scores to essays at scale and may help to support teachers' grading activities. Recently, AES methods based on deep neural networks (DNN) have significantly improved upon the state-of-the-art performance by learning relations between holistic essay scores and student essays. However, DNN-based AES methods function like black-box, negatively affecting the ability to provide automated writing evaluation (AWE). In this work, we proposed a new method, topic-aware BERT, based on fine-tuning the pre-trained language model to learn relations between essay scores and text representations of student essays as well as topical information in essay writing instructions. Moreover, we propose an approach to automatically retrieve key topical sentences in student essays by probing self-attention maps in intermediate layers of topic-aware BERT. We evaluate the performance of topic-aware BERT to (i) perform AES and (ii) retrieve key topical sentences using the open dataset Automated Student Assessment Prize and a manually annotated dataset, respectively. Our model achieves a strong AES performance compared with previous state-of-the-art DNN-based methods and shows effectiveness in identifying key topical sentences in argumentative essays.
|
|
| 16. |
- Wu, Yongchao, 1987-, et al.
(författare)
-
Towards Improving the Reliability and Transparency of ChatGPT for Educational Question Answering
- 2023
-
Ingår i: LNCS Springer Conference Proceedings. - : Springer. - 9783031426810
-
Konferensbidrag (refereegranskat)abstract
- Large language models (LLMs), such as ChatGPT, have shown remarkable performance on various natural language processing (NLP) tasks, including educational question answering (EQA). However, LLMs generate text entirely based on knowledge obtained during pre-training, which means they struggle with recent information or domain-specific knowledge bases. Moreover, only providing answers to questions posed to LLMs without any grounding materials makes it difficult for students to judge their validity.We therefore propose a method for integrating information retrieval systems with LLMs when developing EQA systems, which in addition to improving EQA performance grounds the answers in the educational context. Our experiments show that the proposed system outperforms vanilla ChatGPT with a vast margin of 110.9%, 67.8%, and 43.3% on BLEU, ROUGE, and METEOR scores. In addition, we argue that the use of the retrieved educational context enhances the transparency and reliability of the EQA process, making it easier to determine the correctness of the answers.
|
|
| 17. |
- Zhang, LeChen, 1987-, et al.
(författare)
-
Assessing K-9 Teachers' Computational Thinking Skills
- 2022
-
Ingår i: Research Anthology on Computational Thinking, Programming, and Robotics in the Classroom. - Hershey, Pennsylvania : IGI Global. - 9781668424124 - 9781668424117 ; , s. 467-487
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- Many national curricula have incorporated computational thinking (CT) into compulsory education. Teacher ability to deliver the revised curriculum determines whether these new skills can be successfully integrated into teaching. Therefore, it is crucial to examine teacher readiness. This study measured Swedish K-9 teacher CT skills through a CT test validated by an expert review panel and a principal component analysis. Additionally, we engaged statistical analyses to examine the relationship between the teachers' background and their CT test scores, as well as their self-reported ability to teach CT. The result demonstrated the teachers' proficiency in different types of CT skills. Another finding revealed that the type of programming language mastered by teachers was associated with both their CT test score and self-reported ability to teach CT. This CT test can support teachers to identify specific areas for professional development and may facilitate the school management to plan teachers' competence training strategically.
|
|
