| 1. |
- Ballesteros, Miguel, et al.
(författare)
-
MaltOptimizer : Fast and Effective Parser Optimization
- 2016
-
Ingår i: Natural Language Engineering. - 1351-3249 .- 1469-8110. ; 22:2, s. 187-213
-
Tidskriftsartikel (refereegranskat)abstract
- Statistical parsers often require careful parameter tuning and feature selection. This is a nontrivial task for application developers who are not interested in parsing for its own sake, and it can be time-consuming even for experienced researchers. In this paper we present MaltOptimizer, a tool developed to automatically explore parameters and features for MaltParser, a transition-based dependency parsing system that can be used to train parser's given treebank data. MaltParser provides a wide range of parameters for optimization, including nine different parsing algorithms, an expressive feature specification language that can be used to define arbitrarily rich feature models, and two machine learning libraries, each with their own parameters. MaltOptimizer is an interactive system that performs parser optimization in three stages. First, it performs an analysis of the training set in order to select a suitable starting point for optimization. Second, it selects the best parsing algorithm and tunes the parameters of this algorithm. Finally, it performs feature selection and tunes machine learning parameters. Experiments on a wide range of data sets show that MaltOptimizer quickly produces models that consistently outperform default settings and often approach the accuracy achieved through careful manual optimization.
|
|
| 2. |
|
|
| 3. |
- Basirat, Ali, 1982-, et al.
(författare)
-
Real-valued Syntactic Word Vectors (RSV) for Greedy Neural Dependency Parsing
- 2017
-
Konferensbidrag (refereegranskat)abstract
- We show that a set of real-valued word vectors formed by right singular vectors of a transformed co-occurrence matrix are meaningful for determining different types of dependency relations between words. Our experimental results on the task of dependency parsing confirm the superiority of the word vectors to the other sets of word vectors generated by popular methods of word embedding. We also study the effect of using these vectors on the accuracy of dependency parsing in different languages versus using more complex parsing architectures.
|
|
| 4. |
|
|
| 5. |
|
|
| 6. |
|
|
| 7. |
- Constant, Matthieu, et al.
(författare)
-
A Transition-Based System for Joint Lexical and Syntactic Analysis
- 2016
-
Ingår i: PROCEEDINGS OF THE 54TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1. ; , s. 161-171
-
Konferensbidrag (refereegranskat)abstract
- We present a transition-based system that jointly predicts the syntactic structure and lexical units of a sentence by building two structures over the input words: a syntactic dependency tree and a forest of lexical units including multiword expressions (MWEs). This combined representation allows us to capture both the syntactic and semantic structure of MWEs, which in turn enables deeper downstream semantic analysis, especially for semi-compositional MWEs. The proposed system extends the arc-standard transition system for dependency parsing with transitions for building complex lexical units. Experiments on two different data sets show that the approach significantly improves MWE identification accuracy (and sometimes syntactic accuracy) compared to existing joint approaches.
|
|
| 8. |
- de Lhoneux, Miryam, 1990-, et al.
(författare)
-
Arc-Hybrid Non-Projective Dependency Parsing with a Static-Dynamic Oracle
- 2017
-
Ingår i: IWPT 2017 15th International Conference on Parsing Technologies. - Pisa, Italy : Association for Computational Linguistics. - 9781945626739 ; , s. 99-104
-
Konferensbidrag (refereegranskat)abstract
- We extend the arc-hybrid transition system for dependency parsing with a SWAP transition that enables reordering of the words and construction of non-projective trees. Although this extension potentially breaks the arc-decomposability of the transition system, we show that the existing dynamic oracle can be modified and combined with a static oracle for the SWAP transition. Experiments on five languages with different degrees of non-projectivity show that the new system gives competitive accuracy and is significantly better than a system trained with a purely static oracle.
|
|
| 9. |
- de Lhoneux, Miryam, 1990-, et al.
(författare)
-
From raw text to Universal Dependencies : look, no tags!
- 2017
-
Ingår i: Proceedings of the CoNLL 2017 Shared Task. - Vancouver, Canada : Association for Computational Linguistics. - 9781945626708 ; , s. 207-217
-
Konferensbidrag (refereegranskat)abstract
- We present the Uppsala submission to the CoNLL 2017 shared task on parsing from raw text to universal dependencies. Our system is a simple pipeline consisting of two components. The first performs joint word and sentence segmentation on raw text; the second predicts dependency trees from raw words. The parser bypasses the need for part-of-speech tagging, but uses word embeddings based on universal tag distributions. We achieved a macroaveraged LAS F1 of 65.11 in the official test run and obtained the 2nd best result for sentence segmentation with a score of 89.03. After fixing two bugs, we obtained an unofficial LAS F1 of 70.49.
|
|
| 10. |
- de Lhoneux, Miryam, 1990-, et al.
(författare)
-
Recursive Subtree Composition in LSTM-Based Dependency Parsing
- 2019
-
Ingår i: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. - Stroudsburg : Association for Computational Linguistics. - 9781950737130 ; , s. 1566-1576
-
Konferensbidrag (refereegranskat)abstract
- The need for tree structure modelling on top of sequence modelling is an open issue in neural dependency parsing. We investigate the impact of adding a tree layer on top of a sequential model by recursively composing subtree representations (composition) in a transition-based parser that uses features extracted by a BiLSTM. Composition seems superfluous with such a model, suggesting that BiLSTMs capture information about subtrees. We perform model ablations to tease out the conditions under which composition helps. When ablating the backward LSTM, performance drops and composition does not recover much of the gap. When ablating the forward LSTM, performance drops less dramatically and composition recovers a substantial part of the gap, indicating that a forward LSTM and composition capture similar information. We take the backward LSTM to be related to lookahead features and the forward LSTM to the rich history-based features both crucial for transition-based parsers. To capture history-based information, composition is better than a forward LSTM on its own, but it is even better to have a forward LSTM as part of a BiLSTM. We correlate results with language properties, showing that the improved lookahead of a backward LSTM is especially important for head-final languages.
|
|
| 11. |
- de Lhoneux, Miryam, 1990-, et al.
(författare)
-
What Should/Do/Can LSTMs Learn When Parsing Auxiliary Verb Constructions?
- 2019
-
Ingår i: CoRR. ; abs/1907.07950
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- This article is a linguistic investigation of a neural parser. We look at transitivity and agreement information of auxiliary verb constructions (AVCs) in comparison to finite main verbs (FMVs). This comparison is motivated by theoretical work in dependency grammar and in particular the work of Tesnière (1959) where AVCs and FMVs are both instances of a nucleus, the basic unit of syntax. An AVC is a dissociated nucleus, it consists of at least two words, and a FMV is its non-dissociated counterpart, consisting of exactly one word. We suggest that the representation of AVCs and FMVs should capture similar information. We use diagnostic classifiers to probe agreement and transitivity information in vectors learned by a transition-based neural parser in four typologically different languages. We find that the parser learns different information about AVCs and FMVs if only sequential models (BiLSTMs) are used in the architecture but similar information when a recursive layer is used. We find explanations for why this is the case by looking closely at how information is learned in the network and looking at what happens with different dependency representations of AVCs.
|
|
| 12. |
- de Marneffe, Marie-Catherine, et al.
(författare)
-
Dependency Grammar
- 2019
-
Ingår i: Annual review of linguistics. - : ANNUAL REVIEWS. - 2333-9691 .- 2333-9683. ; 5, s. 197-218
-
Tidskriftsartikel (refereegranskat)abstract
- Dependency grammar is a descriptive and theoretical tradition in linguistics that can be traced back to antiquity. It has long been influential in the European linguistics tradition and has more recently become a mainstream approach to representing syntactic and semantic structure in natural language processing. In this review, we introduce the basic theoretical assumptions of dependency grammar and review some key aspects in which different dependency frameworks agree or disagree. We also discuss advantages and disadvantages of dependency representations and introduce Universal Dependencies, a framework for multilingual dependency-based morphosyntactic annotation that has been applied to more than 60 languages.
|
|
| 13. |
|
|
| 14. |
- Dobrovoljc, Kaja, et al.
(författare)
-
The Universal Dependencies Treebank of Spoken Slovenian
- 2016
-
Ingår i: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). - 9782951740891 ; , s. 1566-1573
-
Konferensbidrag (refereegranskat)abstract
- This paper presents the construction of an open-source dependency treebank of spoken Slovenian, the first syntactically annotated collection of spontaneous speech in Slovenian. The treebank has been manually annotated using the Universal Dependencies annotation scheme, a one-layer syntactic annotation scheme with a high degree of cross-modality, cross-framework and cross-language interoperability. In this original application of the scheme to spoken language transcripts, we address a wide spectrum of syntactic particularities in speech, either by extending the scope of application of existing universal labels or by proposing new speech-specific extensions. The initial analysis of the resulting treebank and its comparison with the written Slovenian UD treebank confirms significant syntactic differences between the two language modalities, with spoken data consisting of shorter and more elliptic sentences, less and simpler nominal phrases, and more relations marking disfluencies, interaction, deixis and modality.
|
|
| 15. |
|
|
| 16. |
- Dubremetz, Marie, 1988-, et al.
(författare)
-
Rhetorical Figure Detection : Chiasmus, Epanaphora, Epiphora
- 2018
-
Ingår i: Frontiers in Digital Humanities. - : Frontiers Media SA. - 2297-2668. ; 5:10
-
Tidskriftsartikel (refereegranskat)abstract
- Rhetorical figures are valuable linguistic data for literary analysis. In this article, we target the detection of three rhetorical figures that belong to the family of repetitive figures: chiasmus (I go where I please, and I please where I go.), epanaphora also called anaphora (“Poor old European Commission! Poor old European Council.”) and epiphora (“This house is mine. This car is mine. You are mine.”). Detecting repetition of words is easy for a computer but detecting only the ones provoking a rhetorical effect is difficult because of many accidental and irrelevant repetitions. For all figures, we train a log-linear classifier on a corpus of political debates. The corpus is only very partially annotated, but we nevertheless obtain good results, with more than 50% precision for all figures. We then apply our models to totally different genres and perform a comparative analysis, by comparing corpora of fiction, science and quotes. Thanks to the automatic detection of rhetorical figures, we discover that chiasmus is more likely to appear in the scientific context whereas epanaphora and epiphora are more common in fiction.
|
|
| 17. |
|
|
| 18. |
|
|
| 19. |
|
|
| 20. |
|
|
| 21. |
|
|
| 22. |
|
|
| 23. |
- Kulmizev, Artur, et al.
(författare)
-
Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing – A Tale of Two Parsers Revisited
- 2019
-
Ingår i: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ; , s. 2755-2768
-
Konferensbidrag (refereegranskat)abstract
- Transition-based and graph-based dependency parsers have previously been shown to have complementary strengths and weaknesses: transition-based parsers exploit rich structural features but suffer from error propagation, while graph-based parsers benefit from global optimization but have restricted feature scope. In this paper, we show that, even though some details of the picture have changed after the switch to neural networks and continuous representations, the basic trade-off between rich features and global optimization remains essentially the same. Moreover, we show that deep contextualized word embeddings, which allow parsers to pack information about global sentence structure into local feature representations, benefit transition-based parsers more than graph-based parsers, making the two approaches virtually equivalent in terms of both accuracy and error profile. We argue that the reason is that these representations help prevent search errors and thereby allow transitionbased parsers to better exploit their inherent strength of making accurate local decisions. We support this explanation by an error analysis of parsing experiments on 13 languages.
|
|
| 24. |
|
|
| 25. |
|
|
| 26. |
- Nivre, Joakim, 1962-
(författare)
-
Om datorer och språkförståelse
- 2015
-
Ingår i: Å…rsbok 2015. - : Kungliga Vetenskaps-Societeten i Uppsala. ; , s. 75-82
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)
|
|
| 27. |
- Nivre, Joakim, 1962-, et al.
(författare)
-
Universal Dependencies v1 : A Multilingual Treebank Collection
- 2016
-
Ingår i: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). - Paris : EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA. - 9782951740891 ; , s. 1659-1666
-
Konferensbidrag (refereegranskat)abstract
- Cross-linguistically consistent annotation is necessary for sound comparative evaluation and cross-lingual learning experiments. It is also useful for multilingual system development and comparative linguistic studies. Universal Dependencies is an open community effort to create cross-linguistically consistent treebank annotation for many languages within a dependency-based lexicalist framework. In this paper, we describe v1 of the universal guidelines, the underlying design principles, and the currently available treebanks for 33 languages.
|
|
| 28. |
- Nivre, Joakim, 1962-, et al.
(författare)
-
Universal Dependency Evaluation
- 2017
-
Ingår i: Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017). - 9789176855010 ; , s. 86-95
-
Konferensbidrag (refereegranskat)
|
|
| 29. |
|
|
| 30. |
|
|
| 31. |
|
|
| 32. |
|
|
| 33. |
- Seraji, Mojgan, et al.
(författare)
-
Universal Dependencies for Persian
- 2016
-
Ingår i: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). - Paris : EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA. - 9782951740891 ; , s. 2361-2365
-
Konferensbidrag (refereegranskat)abstract
- The Persian Universal Dependency Treebank (Persian UD) is a recent effort of treebanking Persian with Universal Dependencies (UD), an ongoing project that designs unified and cross-linguistically valid grammatical representations including part-of-speech tags, morphological features, and dependency relations. The Persian UD is the converted version of the Uppsala Persian Dependency Treebank (UPDT) to the universal dependencies framework and consists of nearly 6,000 sentences and 152,871 word tokens with an average sentence length of 25 words. In addition to the universal dependencies syntactic annotation guidelines, the two treebanks differ in tokenization. All words containing unsegmented clitics (pronominal and copula clitics) annotated with complex labels in the UPDT have been separated from the clitics and appear with distinct labels in the Persian UD. The treebank has its original syntactic annotation scheme based on Stanford Typed Dependencies. In this paper, we present the approaches taken in the development of the Persian UD.
|
|
| 34. |
- Shao, Yan, 1990-, et al.
(författare)
-
Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF
- 2017
-
Ingår i: Proceedings of the The 8th International Joint Conference on Natural Language Processing. - Taipei : Asian Federation of Natural Language Processing. ; , s. 173-183
-
Konferensbidrag (refereegranskat)abstract
- We present a character-based model for joint segmentation and POS tagging for Chinese. The bidirectional RNN-CRF architecture for general sequence tagging is adapted and applied with novel vector representations of Chinese characters that capture rich contextual information and sub-character level features. The proposed model is extensively evaluated and compared with a state-of-the-art tagger respectively on CTB5, CTB9 and UD Chinese. The experimental results indicate that our model is accurate and robust across datasets in different sizes, genres and annotation schemes. We obtain stateof-the-art performance on CTB5, achieving 94.38 F1-score for joint segmentation and POS tagging.
|
|
| 35. |
- Shao, Yan, 1990-, et al.
(författare)
-
Recall is the Proper Evaluation Metric for Word Segmentation
- 2017
-
Ingår i: Proceedings of the The 8th International Joint Conference on Natural Language Processing. - Taipei : Asian Federation of Natural Language Processing. ; , s. 86-90
-
Konferensbidrag (refereegranskat)abstract
- We extensively analyse the correlations and drawbacks of conventionally employed evaluation metrics for word segmentation. Unlike in standard information retrieval, precision favours under-splitting systems and therefore can be misleading in word segmentation. Overall, based on both theoretical and experimental analysis, we propose that precision should be excluded from the standard evaluation metrics and that the evaluation score obtained by using only recall is sufficient and better correlated with the performance of word segmentation systems.
|
|
| 36. |
- Shao, Yan, 1990-, et al.
(författare)
-
Universal Word Segmentation : Implementation and Interpretation
- 2018
-
Ingår i: Transactions of the Association for Computational Linguistics. - 2307-387X. ; 6, s. 421-435
-
Tidskriftsartikel (refereegranskat)abstract
- Word segmentation is a low-level NLP taskt hat is non-trivial for a considerable number of languages. In this paper, we present asequence tagging framework and apply it to word segmentation for a wide range of languages with different writing systems and typological characteristics. Additionally, we investigate the correlations between various typological factors and word segmentation accuracy. The experimental results indicate that segmentation accuracy is positively related to word boundary markers and negatively to the number of unique non-segmental terms. Based on the analysis, we design a small set of language-specific settings and extensively evaluate the segmentation system on the Universal Dependencies datasets. Our model obtains state-of-the-art accuracies on all the UD languages. It performs substantially better on languages that are non-trivial to segment, such as Chinese, Japanese, Arabic and Hebrew, when compared to previous work.
|
|
| 37. |
|
|
| 38. |
- Smith, Aaron, 1985-, et al.
(författare)
-
An Investigation of the Interactions Between Pre-Trained Word Embeddings, Character Models and POS Tags in Dependency Parsing
- 2018
-
Ingår i: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. - : Association for Computational Linguistics. - 9781948087841 ; , s. 2711-2720
-
Konferensbidrag (refereegranskat)abstract
- We provide a comprehensive analysis of the interactions between pre-trained word embeddings, character models and POS tags in a transition-based dependency parser. While previous studies have shown POS information to be less important in the presence of character models, we show that in fact there are complex interactions between all three techniques. In isolation each produces large improvements over a baseline system using randomly initialised word embeddings only, but combining them quickly leads to diminishing returns. We categorise words by frequency, POS tag and language in order to systematically investigate how each of the techniques affects parsing quality. For many word categories, applying any two of the three techniques is almost as good as the full combined system. Character models tend to be more important for low-frequency open-class words, especially in morphologically rich languages, while POS tags can help disambiguate high-frequency function words. We also show that large character embedding sizes help even for languages with small character sets, especially in morphologically rich languages.
|
|
| 39. |
- Stymne, Sara, 1977-, et al.
(författare)
-
Parser Training with Heterogeneous Treebanks
- 2018
-
Ingår i: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). - : Association for Computational Linguistics. - 9781948087346 ; , s. 619-625
-
Konferensbidrag (refereegranskat)abstract
- How to make the most of multiple heterogeneous treebanks when training a monolingual dependency parser is an open question. We start by investigating previouslysuggested, but little evaluated, strategiesfor exploiting multiple treebanks based onconcatenating training sets, with or without fine-tuning. We go on to propose anew method based on treebank embeddings. We perform experiments for severallanguages and show that in many casesfine-tuning and treebank embeddings leadto substantial improvements over singletreebanks or concatenation, with averagegains of 2.0–3.5 LAS points. We arguethat treebank embeddings should be preferred due to their conceptual simplicity,flexibility and extensibility.
|
|
| 40. |
- Tang, Gongbo, et al.
(författare)
-
An analysis of Attention Mechanism: The Case of Word Sense Disambiguation in Neural Machine Translation
- 2018
-
Ingår i: Proceedings of the Third Conference on Machine Translation. ; , s. 26-35
-
Konferensbidrag (refereegranskat)abstract
- Recent work has shown that the encoder-decoder attention mechanisms in neural ma-chine translation (NMT) are different from theword alignment in statistical machine trans-lation.In this paper, we focus on analyz-ing encoder-decoder attention mechanisms, inthe case of word sense disambiguation (WSD)in NMT models. We hypothesize that atten-tion mechanisms pay more attention to contexttokens when translating ambiguous words.We explore the attention distribution patternswhen translating ambiguous nouns. Counter-intuitively, we find that attention mechanismsare likely to distribute more attention to theambiguous noun itself rather than context to-kens, in comparison to other nouns. We con-clude that attention is not the main mecha-nism used by NMT models to incorporate con-textual information for WSD. The experimen-tal results suggest that NMT models learn toencode contextual information necessary forWSD in the encoder hidden states. For the at-tention mechanism in Transformer models, wereveal that the first few layers gradually learnto “align” source and target tokens and the lastfew layers learn to extract features from the re-lated but unaligned context tokens
|
|
| 41. |
- Tang, Gongbo, et al.
(författare)
-
An evaluation of neural machine translation models on historical spelling normalization
- 2018
-
Ingår i: Proceedings of the 27th International Conference on Computational Linguistics. ; , s. 1320-1331
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages: English, German, Hungarian, Icelandic, and Swedish. The NMT models are at different levels, have different attention mechanisms, and different neural network architectures. Our results show that NMT models are much better than SMT models in terms of character error rate. The vanilla RNNs are competitive to GRUs/LSTMs in historical spelling normalization. Transformer models perform better only when provided with more training data. We also find that subword-level models with a small subword vocabulary are better than character-level models. In addition, we propose a hybrid method which further improves the performance of historical spelling normalization.
|
|
| 42. |
- Tang, Gongbo, et al.
(författare)
-
Encoders Help You Disambiguate Word Senses in Neural Machine Translation
- 2019
-
Ingår i: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ; , s. 1429-1435
-
Konferensbidrag (refereegranskat)abstract
- Neural machine translation (NMT) has achieved new state-of-the-art performance in translating ambiguous words. However, it is still unclear which component dominates the process of disambiguation. In this paper, we explore the ability of NMT encoders and decoders to disambiguate word senses by evaluating hidden states and investigating the distributions of self-attention. We train a classifier to predict whether a translation is correct given the representation of an ambiguous noun. We find that encoder hidden states outperform word embeddings significantly which indicates that encoders adequately encode relevant information for disambiguation into hidden states. In contrast to encoders, the effect of decoder is different in models with different architectures. Moreover, the attention weights and attention entropy show that self-attention can detect ambiguous nouns and distribute more attention to the context.
|
|
| 43. |
|
|
| 44. |
|
|
