Sökning: onr:"swepub:oai:DiVA.org:ri-73777" >
GPT-SW3 :
GPT-SW3 : An Autoregressive Language Model for the Scandinavian Languages
-
- Ekgren, Ariel (författare)
- AI Sweden, Sweden
-
- Gyllensten, Amaru Cuba (författare)
- AI Sweden, Sweden
-
- Stollenwerk, Felix (författare)
- AI Sweden, Sweden
-
visa fler...
-
- Öhman, Joey (författare)
- AI Sweden, Sweden
-
- Isbister, Tim (författare)
- AI Sweden, Sweden
-
- Gogoulou, Evangelia (författare)
- RISE,Datavetenskap
-
- Carlsson, Fredrik (författare)
- RISE,Elektrifiering och pålitlighet
-
- Casademont, Judit (författare)
- AI Sweden, Sweden
-
- Sahlgren, Magnus (författare)
- AI Sweden, Sweden
-
visa färre...
-
(creator_code:org_t)
- European Language Resources Association (ELRA), 2024
- 2024
- Engelska.
-
Ingår i: <em>2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings</em>. - : European Language Resources Association (ELRA). ; , s. 7886-7900
- Relaterad länk:
-
https://urn.kb.se/re...
Abstract
Ämnesord
Stäng
- This paper details the process of developing the first native large generative language model for the North Germanic languages, GPT-SW3. We cover all parts of the development process, from data collection and processing, training configuration and instruction finetuning, to evaluation, applications, and considerations for release strategies. We discuss pros and cons of developing large language models for smaller languages and in relatively peripheral regions of the globe, and we hope that this paper can serve as a guide and reference for other researchers that undertake the development of large generative models for smaller languages.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
Nyckelord
- Computational linguistics; Auto-regressive; Data collection; Development process; Generative model; Language model; Large language model; Low resource languages; Multilinguality; Peripheral regions; Release strategies; Data handling
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)