Sökning: onr:"swepub:oai:DiVA.org:su-201394" >
Probing Multilingua...
Probing Multilingual Language Models for Discourse
-
- Kurfali, Murathan, 1990- (författare)
- Stockholms universitet,Institutionen för lingvistik
-
- Östling, Robert, 1986- (författare)
- Stockholms universitet,Institutionen för lingvistik
-
(creator_code:org_t)
- 2021
- 2021
- Engelska.
- Relaterad länk:
-
https://urn.kb.se/re...
Abstract
Ämnesord
Stäng
- Pre-trained multilingual language models have become an important building block in multilingual natural language processing. In the present paper, we investigate a range of such models to find out how well they transfer discourse-level knowledge across languages. This is done with a systematic evaluation on a broader set of discourse-level tasks than has been previously been assembled. We find that the XLM-RoBERTa family of models consistently show the best performance, by simultaneously being good monolingual models and degrading relatively little in a zero-shot setting. Our results also indicate that model distillation may hurt the ability of cross-lingual transfer of sentence representations, while language dissimilarity at most has a modest effect. We hope that our test suite, covering 5 tasks with a total of 22 languages in 10 distinct families, will serve as a useful evaluation platform for multilingual performance at and beyond the sentence level.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Språkteknologi (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Language Technology (hsv//eng)
Nyckelord
- Computational Linguistics
- datorlingvistik
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)