| 1. |
- Han, Ridong, et al.
(författare)
-
Document-level Relation Extraction with Relation Correlations
- 2024
-
Ingår i: Neural Networks. - Oxford : Elsevier. - 0893-6080 .- 1879-2782. ; 171, s. 14-24
-
Tidskriftsartikel (refereegranskat)abstract
- Document-level relation extraction faces two often overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above challenges. In this paper, we analyze the co-occurrence correlation of relations, and introduce it into the document-level relation extraction task for the first time. We argue that the correlations can not only transfer knowledge between data-rich relations and data-scarce ones to assist in the training of long-tailed relations, but also reflect semantic distance guiding the classifier to identify semantically close relations for multi-label entity pairs. Specifically, we use relation embedding as a medium, and propose two co-occurrence prediction sub-tasks from both coarse- and fine-grained perspectives to capture relation correlations. Finally, the learned correlation-aware embeddings are used to guide the extraction of relational facts. Substantial experiments on two popular datasets (i.e., DocRED and DWIE) are conducted, and our method achieves superior results compared to baselines. Insightful analysis also demonstrates the potential of relation correlations to address the above challenges. The data and code are released at https://github.com/RidongHan/DocRE-Co-Occur. © 2023 Elsevier Ltd
|
|
| 2. |
- Li, Jianquan, et al.
(författare)
-
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk
- 2023
-
Ingår i: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). - Stroudsburg, PA : Association for Computational Linguistics. - 9781959429722 ; , s. 7581-7596
-
Konferensbidrag (refereegranskat)abstract
- Language is the principal tool for human communication, in which humor is one of the most attractive parts. Producing natural language like humans using computers, a.k.a, Natural Language Generation (NLG), has been widely used for dialogue systems, chatbots, text summarization, as well as AI-Generated Content (AIGC), e.g., idea generation, and scriptwriting. However, the humor aspect of natural language is relatively under-investigated, especially in the age of pre-trained language models. In this work, we aim to preliminarily test whether NLG can generate humor as humans do. We build the largest dataset consisting of numerous Chinese Comical Crosstalk scripts (called C3 in short), which is for a popular Chinese performing art called 'Xiangsheng' or '相声' since 1800s. We benchmark various generation approaches including training-from-scratch Seq2seq, fine-tuned middle-scale PLMs, and large-scale PLMs with and without fine-tuning. Moreover, we also conduct a human assessment, showing that 1) large-scale pretraining largely improves crosstalk generation quality; and 2) even the scripts generated from the best PLM is far from what we expect. We conclude humor generation could be largely improved using large-scale PLMs, but it is still in its infancy. The data and benchmarking code are publicly available in https://github.com/anonNo2/crosstalk-generation. © 2023 Association for Computational Linguistics.
|
|
| 3. |
- Sun, Le, et al.
(författare)
-
Few-Shot Class-Incremental Learning for Medical Time Series Classification
- 2024
-
Ingår i: IEEE journal of biomedical and health informatics. - Piscataway, N.J. : IEEE. - 2168-2194 .- 2168-2208. ; 28:4, s. 1872-1882
-
Tidskriftsartikel (refereegranskat)abstract
- Continuously analyzing medical time series as new classes emerge is meaningful for health monitoring and medical decision-making. Few-shot class-incremental learning (FSCIL) explores the classification of few-shot new classes without forgetting old classes. However, little of the existing research on FSCIL focuses on medical time series classification, which is more challenging to learn due to its large intra-class variability. In this paper, we propose a framework, the Meta self-Attention Prototype Incrementer (MAPIC) to address these problems. MAPIC contains three main modules: an embedding encoder for feature extraction, a prototype enhancement module for increasing inter-class variation, and a distance-based classifier for reducing intra-class variation. To mitigate catastrophic forgetting, MAPIC adopts a parameter protection strategy in which the parameters of the embedding encoder module are frozen at incremental stages after being trained in the base stage. The prototype enhancement module is proposed to enhance the expressiveness of prototypes by perceiving inter-class relations using a self-attention mechanism. We design a composite loss function containing the sample classification loss, the prototype non-overlapping loss, and the knowledge distillation loss, which work together to reduce intra-class variations and resist catastrophic forgetting. Experimental results on three different time series datasets show that MAPIC significantly outperforms state-of-the-art approaches by 27.99%, 18.4%, and 3.95%, respectively. IEEE
|
|
| 4. |
- Wang, Benyou, et al.
(författare)
-
Pre-trained Language Models in Biomedical Domain : A Systematic Survey
- 2024
-
Ingår i: ACM Computing Surveys. - New York, NY : Association for Computing Machinery (ACM). - 0360-0300 .- 1557-7341. ; 56:3
-
Forskningsöversikt (refereegranskat)abstract
- Pre-trained language models (PLMs) have been the de facto paradigm for most natural language processing tasks. This also benefits the biomedical domain: researchers from informatics, medicine, and computer science communities propose various PLMs trained on biomedical datasets, e.g., biomedical text, electronic health records, protein, and DNA sequences for various biomedical tasks. However, the cross-discipline characteristics of biomedical PLMs hinder their spreading among communities; some existing works are isolated from each other without comprehensive comparison and discussions. It is nontrivial to make a survey that not only systematically reviews recent advances in biomedical PLMs and their applications but also standardizes terminology and benchmarks. This article summarizes the recent progress of pre-trained language models in the biomedical domain and their applications in downstream biomedical tasks. Particularly, we discuss the motivations of PLMs in the biomedical domain and introduce the key concepts of pre-trained language models. We then propose a taxonomy of existing biomedical PLMs that categorizes them from various perspectives systematically. Plus, their applications in biomedical downstream tasks are exhaustively discussed, respectively. Last, we illustrate various limitations and future trends, which aims to provide inspiration for the future research. © 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
|
|
