| 1. |
|
|
| 2. |
- Aayesha, Aayesha, et al.
(författare)
-
An Ensemble Approach for Question-Level Knowledge Tracing
- 2021
-
Ingår i: Artificial Intelligence in Education. - Cham : Springer. - 9783030782702 - 9783030782696 ; , s. 433-437
-
Konferensbidrag (refereegranskat)abstract
- Knowledge tracing—where a machine models the students’ knowledge as they interact with coursework—is a well-established area in the field of Artificial Intelligence in Education. In this paper, an ensemble approach is proposed that addresses existing limitations in question-centric knowledge tracing and achieves the goal of predicting future question correctness. The proposed approach consists of two models; one is Light Gradient Boosting Machine (LightGBM) built by incorporating all relevant key features engineered from the data. The second model is a Multiheaded-Self-Attention Knowledge Tracing model (MSAKT) that extracts historical student knowledge of future question by calculating their contextual similarity with previously attempted questions. The proposed model’s effectiveness is evaluated by conducting experiments on a big Kaggle dataset achieving an Area Under ROC Curve (AUC) score of 0.84 with 84% accuracy using 10fold cross-validation.
|
|
| 3. |
- Afzaal, Muhammad, et al.
(författare)
-
Automatic and Intelligent Recommendations to Support Students’ Self-Regulation
- 2021
-
Ingår i: International Conference on Advanced Learning Technologies (ICALT). - 9781665441063 ; , s. 336-338
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we propose a counterfactual explanations-based approach to provide an automatic and intelligent recommendation that supports student's self-regulation of learning in a data-driven manner, aiming to improve their performance in courses. Existing work under the fields of learning analytics and AI in education predict students' performance and use the prediction outcome as feedback without explaining the reasons behind the prediction. Our proposed approach developed an algorithm that explains the root causes behind student's performance decline and generates data-driven recommendations for action. The effectiveness of the proposed predictive model that constitutes the intelligent recommendations is evaluated, with results demonstrating high accuracy.
|
|
| 4. |
- Afzaal, Muhammad, et al.
(författare)
-
Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Students Self-Regulation
- 2021
-
Ingår i: Frontiers in Artificial Intelligence. - : Frontiers Media SA. - 2624-8212. ; 4
-
Tidskriftsartikel (refereegranskat)abstract
- Formative feedback has long been recognised as an effective tool for student learning, and researchers have investigated the subject for decades. However, the actual implementation of formative feedback practices is associated with significant challenges because it is highly time-consuming for teachers to analyse students’ behaviours and to formulate and deliver effective feedback and action recommendations to support students’ regulation of learning. This paper proposes a novel approach that employs learning analytics techniques combined with explainable machine learning to provide automatic and intelligent feedback and action recommendations that support student’s self-regulation in a data-driven manner, aiming to improve their performance in courses. Prior studies within the field of learning analytics have predicted students’ performance and have used the prediction status as feedback without explaining the reasons behind the prediction. Our proposed method, which has been developed based on LMS data from a university course, extends this approach by explaining the root causes of the predictions and by automatically providing data-driven intelligent recommendations for action. Based on the proposed explainable machine learning-based approach, a dashboard that provides data-driven feedback and intelligent course action recommendations to students is developed, tested and evaluated. Based on such an evaluation, we identify and discuss the utility and limitations of the developed dashboard. According to the findings of the conducted evaluation, the dashboard improved students’ learning outcomes, assisted them in self-regulation and had a positive effect on their motivation.
|
|
| 5. |
- Afzaal, Muhammad, et al.
(författare)
-
Generation of Automatic Data-Driven Feedback to Students Using Explainable Machine Learning
- 2021
-
Ingår i: Artificial Intelligence in Education. - Cham : Springer. - 9783030782702 ; , s. 37-42
-
Konferensbidrag (refereegranskat)abstract
- This paper proposes a novel approach that employs learning analytics techniques combined with explainable machine learning to provide automatic and intelligent actionable feedback that supports students self-regulation of learning in a data-driven manner. Prior studies within the field of learning analytics predict students’ performance and use the prediction status as feedback without explaining the reasons behind the prediction. Our proposed method, which has been developed based on LMS data from a university course, extends this approach by explaining the root causes of the predictions and automatically provides data-driven recommendations for action. The underlying predictive model effectiveness of the proposed approach is evaluated, with the results demonstrating 90 per cent accuracy.
|
|
| 6. |
- Li, Xiu, et al.
(författare)
-
Automatic Educational Concept Extraction Using NLP
- 2022
-
Ingår i: Methodologies and Intelligent Systems for Technology Enhanced Learning, 12th International Conference. - Cham : Springer Nature. - 9783031206177 - 9783031206160 ; , s. 133-138
-
Konferensbidrag (refereegranskat)abstract
- Educational concepts are the core of teaching and learning. From the perspective of educational technology, concepts are essential meta-data, represen- tative terms that can connect different learning materials, and are the foundation for many downstream tasks. Some studies on automatic concept extraction have been conducted, but there are no studies looking at the K-12 level and focused on the Swedish language. In this paper, we use a state-of-the-art Swedish BERT model to build an automatic concept extractor for the Biology subject using fine- annotated digital textbook data that cover all content for K-12. The model gives a recall measure of 72% and has the potential to be used in real-world settings for use cases that require high recall. Meanwhile, we investigate how input data fea- tures influence model performance and provide guidance on how to effectively use text data to achieve the optimal results when building a named entity recognition (NER) model.
|
|
| 7. |
- Li, Xiu, et al.
(författare)
-
Evaluating Embeddings from Pre-Trained Language Models and Knowledge Graphs for Educational Content Recommendation
- 2024
-
Ingår i: Future Internet. - 1999-5903. ; 16:1, s. 1-21
-
Tidskriftsartikel (refereegranskat)abstract
- Educational content recommendation is a cornerstone of AI-enhanced learning. In particular, to facilitate navigating the diverse learning resources available on learning platforms, methods are needed for automatically linking learning materials, e.g. in order to recommend textbook content based on exercises. Such methods are typically based on semantic textual similarity (STS) and the use of embeddings for text representation. However, it remains unclear what types of embeddings should be used for this task. In this study, we carry out an extensive empirical evaluation of embeddings derived from three different types of models: (i) static embeddings trained using a concept-based knowledge graph, (ii) contextual embeddings from a pre-trained language model, and (iii) contextual embeddings from a large language model (LLM). In addition to evaluating the models individually, various ensembles are explored based on different strategies for combining two models in an early vs. late fusion fashion. The evaluation is carried out using digital textbooks in Swedish for three different subjects and two types of exercises. The results show that using contextual embeddings from an LLM leads to superior performance compared to the other models, and that there is no significant improvement when combining these with static embeddings trained using a knowledge graph. When using embeddings derived from a smaller language model, however, it helps to combine them with knowledge graph embeddings. The performance of the best-performing model is high for both types of exercises, resulting in a mean Recall@3 of 0.96 and 0.95 and a mean MRR of 0.87 and 0.86 for quizzes and study questions, respectively, demonstrating the feasibility of using STS based on text embeddings for educational content recommendation. The ability to link digital learning materials in an unsupervised manner -- relying only on readily available pre-trained models -- facilitates the development of AI-enhanced learning.
|
|
| 8. |
- Li, Xiu, 1982-, et al.
(författare)
-
Linking Swedish Learning Materials to Exercises through an AI-Enhanced Recommender System
- 2023
-
Ingår i: Methodologies and Intelligent Systems for Technology Enhanced Learning, 13th International Conference. - Cham : Springer. - 9783031412257 - 9783031412264 ; , s. 96-107
-
Konferensbidrag (refereegranskat)abstract
- As an integral part of AI-enhanced learning, a content recommender automatically filters and recommends relevant learning materials to the learner or the instructor in a learning system. It can effectively help instructors in pedagogical practices and support students in self-regulated learning. Content recommendation technologies and applications have been studied extensively, however, the SOTA technologies have not adequately adapted to the education domain and there is very limited research on how different models and solutions can be applied in the Swedish context and for multiple subjects. In this paper, we develop a text similarity-based content recommender system. Specifically, given a quiz, we automatically recommend supportive learning resources as a reference to the answer and link back to the textbook sections where the examined knowledge points reside. We present a generic method for Swedish educational content recommendations using the most representative models, evaluate and analyze in multi-dimensions such as model types, pooling methods, subjects etc. The best results are obtained by Sentence-BERT (SBERT) with max paragraph-level pooling, outperforming traditional Natural Language Processing (NLP) models and knowledge graph-based models, obtaining on average 95% in Recall@3 and 82% in MRR, and outstanding in dealing with texts containing symbols, equations or calculations. This research provides empirical evidence and analysis, and can be used as a guidance when building a Swedish educational content recommender.
|
|
| 9. |
- Li, Xiu, 1982-, et al.
(författare)
-
Supporting Teaching-to-the-Curriculum by Linking Diagnostic Tests to Curriculum Goals : Using Textbook Content as Context for Retrieval-Augmented Generation with Large Language Models
- 2024
-
Ingår i: Artificial Intelligence in Education. - : Springer Nature. - 9783031643026 - 9783031643019 ; , s. 118-132
-
Konferensbidrag (refereegranskat)abstract
- Using AI for automatically linking exercises to curriculum goals can support many educational use cases and facilitate teaching-to-the-curriculum by ensuring that exercises adequately reflect and encompass the curriculum goals, ultimately enabling curriculum-based assessment. Here, we introduce this novel task and create a manually labeled dataset where two types of diagnostic tests are linked to curriculum goals for Biology G7-9 in Sweden. We cast the problem both as an information retrieval task and a multi-class text classification task and explore unsupervised approaches to both, as labeled data for such tasks is typically scarce. For the information retrieval task, we employ SOTA embedding model ADA-002 for semantic textual similarity (STS), while we prompt a large language model in the form of ChatGPT to classify diagnostic tests into curriculum goals. For both task formulations, we investigate different ways of using textbook content as a pivot and provide additional context for linking diagnostic tests to curriculum goals. We show that a combination of the two approaches in a retrieval-augmented generation model, whereby STS is used for retrieving textbook content as context to ChatGPT that then performs zero-shot classification, leads to the best classification accuracy (73.5%), outperforming both STS-based classification (67.5%) and LLM-based classification without context (71.5%). Finally, we showcase how the proposed method could be used in pedagogical practices.
|
|
| 10. |
- Wu, Yongchao, et al.
(författare)
-
A Word Embeddings Based Clustering Approach for Collaborative Learning Group Formation
- 2021
-
Ingår i: Artificial Intelligence in Education. - Cham : Springer Nature. - 9783030782702 - 9783030782696 ; , s. 395-400
-
Konferensbidrag (refereegranskat)abstract
- Today, collaborative learning has become quite central as a method for learning, and over the past decades, a large number of studies have demonstrated the benefits from various theoretical and methodological perspectives. This study proposes a novel approach that utilises Natural Language Processing(NLP) methods, particularly pre-trained word embeddings, to automatically create homogeneous or heterogeneous groups of students in terms of knowledge and knowledge gaps expressed in assessments. The two different ways of creating groups serve two different pedagogical purposes: (1) homogeneous group formation based on students’ knowledge can support and make teachers’ pedagogical activities such as feedback provision more time efficient, and (2) the heterogeneous groups can support and enhance collaborative learning. We evaluate the performance of the proposed approach through experiments with a dataset from a university course in programming didactics.
|
|
