Sökning: onr:"swepub:oai:DiVA.org:kth-334544" >
Decorrelate Irrelev...
Decorrelate Irrelevant, Purify Relevant : Overcome Textual Spurious Correlations from a Feature Perspective
-
- Dou, Shihan (författare)
- School of Computer Science, Fudan University, Shanghai, China
-
- Zheng, Rui (författare)
- School of Computer Science, Fudan University, Shanghai, China
-
- Wu, Ting (författare)
- School of Computer Science, Fudan University, Shanghai, China
-
visa fler...
-
- Gao, Songyang (författare)
- School of Computer Science, Fudan University, Shanghai, China
-
- Shan, Junjie (författare)
- KTH,Skolan för elektroteknik och datavetenskap (EECS)
-
- Zhang, Qi (författare)
- School of Computer Science, Fudan University, Shanghai, China; Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University
-
- Wu, Yueming (författare)
- Nanyang Technological University, Singapore
-
- Huang, Xuanjing (författare)
- School of Computer Science, Fudan University, Shanghai, China
-
visa färre...
-
(creator_code:org_t)
- Association for Computational Linguistics (ACL), 2022
- 2022
- Engelska.
- Relaterad länk:
-
https://urn.kb.se/re...
Abstract
Ämnesord
Stäng
- Natural language understanding (NLU) models tend to rely on spurious correlations (i.e., dataset bias) to achieve high performance on in-distribution datasets but poor performance on out-of-distribution ones. Most of the existing debiasing methods often identify and weaken these samples with biased features (i.e., superficial surface features that cause such spurious correlations). However, down-weighting these samples obstructs the model in learning from the non-biased parts of these samples. To tackle this challenge, in this paper, we propose to eliminate spurious correlations in a fine-grained manner from a feature space perspective. Specifically, we introduce Random Fourier Features and weighted re-sampling to decorrelate the dependencies between features to mitigate spurious correlations. After obtaining decorrelated features, we further design a mutual-information-based method to purify them, which forces the model to learn features that are more relevant to tasks. Extensive experiments on two well-studied NLU tasks demonstrate that our method is superior to other comparative approaches.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)