Sökning: WFRF:(Nouri Jalal 1982 ) >
Supporting Teaching...
Supporting Teaching-to-the-Curriculum by Linking Diagnostic Tests to Curriculum Goals : Using Textbook Content as Context for Retrieval-Augmented Generation with Large Language Models
-
- Li, Xiu, 1982- (författare)
- Stockholms universitet,Institutionen för data- och systemvetenskap
-
- Henriksson, Aron, 1985- (författare)
- Stockholms universitet,Institutionen för data- och systemvetenskap
-
- Duneld, Martin, 1971- (författare)
- Stockholms universitet,Institutionen för data- och systemvetenskap
-
visa fler...
-
- Nouri, Jalal, 1982- (författare)
- Stockholms universitet,Institutionen för data- och systemvetenskap
-
- Wu, Yongchao, 1987- (författare)
- Stockholms universitet,Institutionen för data- och systemvetenskap
-
visa färre...
-
(creator_code:org_t)
- Springer Nature, 2024
- 2024
- Engelska.
-
Ingår i: Artificial Intelligence in Education. - : Springer Nature. - 9783031643026 - 9783031643019 ; , s. 118-132
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Using AI for automatically linking exercises to curriculum goals can support many educational use cases and facilitate teaching-to-the-curriculum by ensuring that exercises adequately reflect and encompass the curriculum goals, ultimately enabling curriculum-based assessment. Here, we introduce this novel task and create a manually labeled dataset where two types of diagnostic tests are linked to curriculum goals for Biology G7-9 in Sweden. We cast the problem both as an information retrieval task and a multi-class text classification task and explore unsupervised approaches to both, as labeled data for such tasks is typically scarce. For the information retrieval task, we employ SOTA embedding model ADA-002 for semantic textual similarity (STS), while we prompt a large language model in the form of ChatGPT to classify diagnostic tests into curriculum goals. For both task formulations, we investigate different ways of using textbook content as a pivot and provide additional context for linking diagnostic tests to curriculum goals. We show that a combination of the two approaches in a retrieval-augmented generation model, whereby STS is used for retrieving textbook content as context to ChatGPT that then performs zero-shot classification, leads to the best classification accuracy (73.5%), outperforming both STS-based classification (67.5%) and LLM-based classification without context (71.5%). Finally, we showcase how the proposed method could be used in pedagogical practices.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Språkteknologi (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Language Technology (hsv//eng)
Nyckelord
- Teaching-to-the-Curriculum
- Semantic Textual Similarity
- Large Language Models
- ChatGPT
- Retrieval-Augmented Generation.
- data- och systemvetenskap
- Computer and Systems Sciences
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas