Sökning: onr:"swepub:oai:DiVA.org:uu-484791" >
Finding Structural ...
Finding Structural Knowledge in Multimodal-BERT
-
- Milewski, Victor (författare)
- Katholieke Univ Leuven, Dept Comp Sci, Leuven, Belgium.
-
- de Lhoneux, Miryam, 1990- (författare)
- Uppsala universitet,Institutionen för lingvistik och filologi,Katholieke Univ Leuven, Dept Comp Sci, Leuven, Belgium.;Univ Copenhagen, Dept Comp Sci, Copenhagen, Denmark.
-
- Moens, Marie-Francine (författare)
- Katholieke Univ Leuven, Dept Comp Sci, Leuven, Belgium.
-
Katholieke Univ Leuven, Dept Comp Sci, Leuven, Belgium Institutionen för lingvistik och filologi (creator_code:org_t)
- Association for Computational Linguistics, 2022
- 2022
- Engelska.
-
Ingår i: PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1. - : Association for Computational Linguistics. - 9781955917216 ; , s. 5658-5671
- Relaterad länk:
-
https://urn.kb.se/re...
Abstract
Ämnesord
Stäng
- In this work, we investigate the knowledge learned in the embeddings of multimodal-BERT models. More specifically, we probe their capabilities of storing the grammatical structure of linguistic data and the structure learned over objects in visual data. To reach that goal, we first make the inherent structure of language and visuals explicit by a dependency parse of the sentences that describe the image and by the dependencies between the object regions in the image, respectively. We call this explicit visual structure the scene tree, that is based on the dependency tree of the language description. Extensive probing experiments show that the multimodal-BERT models do not encode these scene trees.
Ämnesord
- HUMANIORA -- Språk och litteratur -- Jämförande språkvetenskap och allmän lingvistik (hsv//swe)
- HUMANITIES -- Languages and Literature -- General Language Studies and Linguistics (hsv//eng)
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas