onr:"swepub:oai:DiVA.org:uu-497148"
Search: onr:"swepub:oai:DiVA.org:uu-497148" > Probing Pre-trained...
- 1 of 1
- Previous record
- Next record
- To hitlist
Probing Pre-trained Language Models for Semantic Attributes and their Values
-
- Beloucif, Meriem (author)
- MIN Faculty, Universität Hamburg,Computational Linguistics
-
- Biemann, Chris (author)
- MIN Faculty, Universität Hamburg
- (creator_code:org_t)
- Stroudsburg, PA, USA : Association for Computational Linguistics, 2021
- 2021
- English.
- In: Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021. - Stroudsburg, PA, USA : Association for Computational Linguistics. - 9781955917100 ; , s. 2554-2559
- Conference paper (peer-reviewed)
Abstract
Subject headings
Close
- Pretrained Language Models (PTLMs) yield state-of-the-art performance on many Natural Language Processing tasks, including syntax, semantics and commonsense reasoning. In this paper, we focus on identifying to what extent do PTLMs capture semantic attributes and their values, e.g. the relation between rich and high net worth. We use PTLMs to predict masked tokens using patterns and lists of items from Wikidata in order to verify how likely PTLMs encode semantic attributes along with their values. Such inferences based on semantics are intuitive for us humans as part of our language understanding. Since PTLMs are trained on large amounts of Wikipedia data, we would assume that they can generate similar predictions. However, our findings reveal that PTLMs perform still much worse than humans on this task. We show an analysis which explains how to exploit our methodology to integrate better context and semantics into PTLMs using knowledge bases.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Språkteknologi (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Language Technology (hsv//eng)
Keyword
- BERT
- Pretrained Language Models
- Computational Linguistics
- Machine learning
- Maskininlärning
- Linguistics
- Lingvistik
Publication and Content Type
- ref (subject category)
- kon (subject category)
Find in a library
- Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual E... (Search for host publication in LIBRIS)
To the university's database
- 1 of 1
- Previous record
- Next record
- To hitlist
Find more in SwePub
- By the author/editor
- Beloucif, Meriem
- Biemann, Chris
- About the subject
-
- NATURAL SCIENCES
- NATURAL SCIENCES
- and Computer and Inf ...
- and Language Technol ...
- Articles in the publication
- Findings of the ...
- By the university
- Uppsala University
Search outside SwePub
- Extend your search to:
- Google Book Search
- Google Scholar
Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.
- Copyright © LIBRIS - National Library Systems
- LIBRIS.kb.se
