| 1. |
- Beck, Daniel, et al.
(author)
-
Learning Structural Kernels for Natural Language Processing
- 2015
-
In: Transactions of the Association for Computational Linguistics. - Stroudsburg, PA : Association for Computational Linguistics. - 2307-387X. ; 3, s. 461-473
-
Journal article (peer-reviewed)abstract
- Structural kernels are a flexible learning paradigm that has been widely used in Natural Language Processing. However, the problem of model selection in kernel-based methods is usually overlooked. Previous approaches mostly rely on setting default values for kernel hyperparameters or using grid search, which is slow and coarse-grained. In contrast, Bayesian methods allow efficient model selection by maximizing the evidence on the training data through gradient-based methods. In this paper we show how to perform this in the context of structural kernels by using Gaussian Processes. Experimental results on tree kernels show that this procedure results in better prediction performance compared to hyperparameter optimization via grid search. The framework proposed in this paper can be adapted to other structures besides trees, e.g., strings and graphs, thereby extending the utility of kernel-based methods.
|
|
| 2. |
- Bevacqua, Luca, et al.
(author)
-
Event and Entity Coreference Across Five Languages: Effects of Context and Referring Expression
- 2021
-
In: Dialogue and Discourse. - 2152-9620. ; 12:2, s. 192-226
-
Journal article (peer-reviewed)abstract
- Current work on coreference focuses primarily on entities, often leaving unanalysed the use of anaphors to corefer with antecedents such as events and textual segments. Moreover, the anaphoric forms that speakers use for entity and non-entity coreference are not mutually exclusive. This ambiguity has been the subject of recent work in English, with evidence of a split between comprehenders' preferential interpretation of personal versus demonstrative pronouns. In addition, comprehenders are shown to be sensitive to antecedent complexity and aspectual status, two verb-driven cues that signal how an event is being portrayed. Here we extend this work via a comparison across five languages (English, French, German, Italian, and Spanish). With a story-continuation experiment, we test how different referring expressions corefer with entity and event antecedents and whether verbal features such as argument structure and aspect influence this choice. Our results show widely consistent, not categorical biases across languages: entity coreference is favoured for personal pronouns and event coreference for demonstratives. Antecedent complexity increases the rate at which anaphors are taken to corefer with an event antecedent, but portraying an event as completed does not reach statistical significance (though showing quite uniform patterns). Lastly, we report a comparison of the same referring expressions to refer to entity and event antecedents in a trilingual parallel corpus annotated with coreference. Together, the results provide a first crosslingual picture of coreference preferences beyond the restricted entity-only patterns targeted by most existing work on coreference. The five languages are all shown to allow gradable use of pronouns for entity and event coreference, with biases that align with existing generalizations about the link between prominence and the use of reduced referring expressions. The studies also show the feasibility of manipulating targeted verb-driven cues across multiple languages to support crosslingual comparisons.
|
|
| 3. |
- Callin, Jimmy, et al.
(author)
-
Part-of-Speech Driven Cross-Lingual Pronoun Prediction with Feed-Forward Neural Networks
- 2015
-
In: Proceedings of the Second Workshop on Discourse in Machine Translation (DiscoMT). - Stroudsburg, PA : Association for Computational Linguistics. - 9781941643327 ; , s. 59-64
-
Conference paper (peer-reviewed)abstract
- For some language pairs, pronoun translation is a discourse-driven task which requires information that lies beyond its local context. This motivates the task of predicting the correct pronoun given a source sentence and a target translation, where the translated pronouns have been replaced with placeholders. For cross-lingual pronoun prediction, we suggest a neural network-based model using preceding nouns and determiners as features for suggesting antecedent candidates. Our model scores on par with similar models while having a simpler architecture.
|
|
| 4. |
- Devinney, Hannah, 1995-
(author)
-
Gender and representation : investigations of bias in natural language processing
- 2024
-
Doctoral thesis (other academic/artistic)abstract
- Natural Language Processing (NLP) technologies are a part of our every day realities. They come in forms we can easily see as ‘language technologies’ (auto-correct, translation services, search results) as well as those that fly under our radar (social media algorithms, 'suggested reading' recommendations on news sites, spam filters). NLP fuels many other tools under the Artificial Intelligence umbrella – such as algorithms approving for loan applications – which can have major material effects on our lives. As large language models like ChatGPT have become popularized, we are also increasingly exposed to machine-generated texts.Machine Learning (ML) methods, which most modern NLP tools rely on, replicate patterns in their training data. Typically, these language data are generated by humans, and contain both overt and underlying patterns that we consider socially undesirable, comprising stereotypes and other reflections of human prejudice. Such patterns (often termed 'bias') are picked up and repeated, or even made more extreme, by ML systems. Thus, NLP technologies become a part of the linguistic landscapes in which we humans transmit stereotypes and act on our prejudices. They may participate in this transmission by, for example, translating nurses as women (and doctors as men) or systematically preferring to suggest promoting men over women. These technologies are tools in the construction of power asymmetries not only through the reinforcement of hegemony, but also through the distribution of material resources when they are included in decision-making processes such as screening job applications.This thesis explores gendered biases, trans and nonbinary inclusion, and queer representation within NLP through a feminist and intersectional lens. Three key areas are investigated: the ways in which “gender” is theorized and operationalized by researchers investigating gender bias in NLP; gendered associations within datasets used for training language technologies; and the representation of queer (particularly trans and nonbinary) identities in the output of both low-level NLP models and large language models (LLMs). The findings indicate that nonbinary people/genders are erased by both bias in NLP tools/datasets, and by research/ers attempting to address gender biases. Men and women are also held to cisheteronormative standards (and stereotypes), which is particularly problematic when considering the intersection of gender and sexuality. Although it is possible to mitigate some of these issues in particular circumstances, such as addressing erasure by adding more examples of nonbinary language to training data, the complex nature of the socio-technical landscape which NLP technologies are a part of means that simple fixes may not always be sufficient. Additionally, it is important that ways of measuring and mitigating 'bias' remain flexible, as our understandings of social categories, stereotypes and other undesirable norms, and 'bias' itself will shift across contexts such as time and linguistic setting.
|
|
| 5. |
|
|
| 6. |
|
|
| 7. |
- Guillou, Liane, et al.
(author)
-
Automatic Reference-Based Evaluation of Pronoun Translation Misses the Point
- 2018
-
In: 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018). - : Association for Computational Linguistics. - 9781948087841 ; , s. 4797-4802
-
Conference paper (peer-reviewed)abstract
- We compare the performance of the APT and AutoPRF metrics for pronoun translation against a manually annotated dataset comprising human judgements as to the correctness of translations of the PROTEST test suite. Although there is some correlation with the human judgements, a range of issues limit the performance of the automated metrics. Instead, we recommend the use of semiautomatic metrics and test suites in place of fully automatic metrics.
|
|
| 8. |
- Guillou, Liane, et al.
(author)
-
Findings of the 2016 WMT Shared Taskon Cross-lingual Pronoun Prediction
- 2016
-
In: Proceedings of the First Conference on Machine Translation. ; , s. 525-542
-
Conference paper (other academic/artistic)abstract
- We describe the design, the evaluation setup, and the results of the 2016 WMT shared task on cross-lingual pronoun prediction. This is a classification task in which participants are asked to provide predictions on what pronoun class label should replace a placeholder value in the target-language text, provided in lemmatised and PoS-tagged form. We provided four subtasks, for the English–French and English–German language pairs, in both directions. Eleven teams participated in the shared task; nine for the English–French subtask, five for French–English, nine for English–German, and six for German–English. Most of the submissions outperformed two strong language-model-based baseline systems, with systems using deep recurrent neural networks outperforming those using other architectures for most language pairs.
|
|
| 9. |
- Guillou, Liane, et al.
(author)
-
ParCor 1.0 : A Parallel Pronoun-Coreference Corpus to Support Statistical MT
- 2014
-
In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). - Paris : European Language Resources Association. - 9782951740884 ; , s. 3191-3198
-
Conference paper (peer-reviewed)abstract
- We present ParCor, a parallel corpus of texts in which pronoun coreference – reduced coreference in which pronouns are used as referringexpressions – has been annotated. The corpus is intended to be used both as a resource from which to learn systematic differences inpronoun use between languages and ultimately for developing and testing informed Statistical Machine Translation systems aimed ataddressing the problem of pronoun coreference in translation. At present, the corpus consists of a collection of parallel English-Germandocuments from two different text genres: TED Talks (transcribed planned speech), and EU Bookshop publications (written text). Alldocuments in the corpus have been manually annotated with respect to the type and location of each pronoun and, where relevant, itsantecedent. We provide details of the texts that we selected, the guidelines and tools used to support annotation and some corpus statistics.The texts in the corpus have already been translated into many languages, and we plan to expand the corpus into these other languages, aswell as other genres, in the future.
|
|
| 10. |
- Guillou, Liane, et al.
(author)
-
PROTEST : A Test Suite for Evaluating Pronouns in Machine Translation
- 2016
-
In: LREC 2016. - : EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA. ; , s. 636-643
-
Conference paper (peer-reviewed)abstract
- We present PROTEST, a test suite for the evaluation of pronoun translation by MT systems. The test suite comprises 250 hand-selected pronoun tokens and an automatic evaluation method which compares the translations of pronouns in MT output with those in the reference translation. Pronoun translations that do not match the reference are referred for manual evaluation. PROTEST is designed to support analysis of system performance at the level of individual pronoun groups, rather than to provide a single aggregate measure over all pronouns. We wish to encourage detailed analyses to highlight issues in the handling of specific linguistic mechanisms by MT systems, thereby contributing to a better understanding of those problems involved in translating pronouns. We present two use cases for PROTEST: a) for measuring improvement/degradation of an incremental system change, and b) for comparing the performance of a group of systems whose design may be largely unrelated. Following the latter use case, we demonstrate the application of PROTEST to the evaluation of the systems submitted to the DiscoMT 2015 shared task on pronoun translation.
|
|
