| 1. |
- Ariu, Kaito, et al.
(author)
-
Optimal clustering from noisy binary feedback
- 2024
-
In: Machine Learning. - : Springer Nature. - 0885-6125 .- 1573-0565. ; 113:5, s. 2733-2764
-
Journal article (peer-reviewed)abstract
- We study the problem of clustering a set of items from binary user feedback. Such a problem arises in crowdsourcing platforms solving large-scale labeling tasks with minimal effort put on the users. For example, in some of the recent reCAPTCHA systems, users clicks (binary answers) can be used to efficiently label images. In our inference problem, items are grouped into initially unknown non-overlapping clusters. To recover these clusters, the learner sequentially presents to users a finite list of items together with a question with a binary answer selected from a fixed finite set. For each of these items, the user provides a noisy answer whose expectation is determined by the item cluster and the question and by an item-specific parameter characterizing the hardness of classifying the item. The objective is to devise an algorithm with a minimal cluster recovery error rate. We derive problem-specific information-theoretical lower bounds on the error rate satisfied by any algorithm, for both uniform and adaptive (list, question) selection strategies. For uniform selection, we present a simple algorithm built upon the K-means algorithm and whose performance almost matches the fundamental limits. For adaptive selection, we develop an adaptive algorithm that is inspired by the derivation of the information-theoretical error lower bounds, and in turn allocates the budget in an efficient way. The algorithm learns to select items hard to cluster and relevant questions more often. We compare the performance of our algorithms with or without the adaptive selection strategy numerically and illustrate the gain achieved by being adaptive.
|
|
| 2. |
|
|
| 3. |
- Boulieris, Petros, et al.
(author)
-
Fraud detection with natural language processing
- 2023
-
In: Machine Learning. - 0885-6125 .- 1573-0565.
-
Journal article (peer-reviewed)abstract
- Automated fraud detection can assist organisations to safeguard user accounts, a task that is very challenging due to the great sparsity of known fraud transactions. Many approaches in the literature focus on credit card fraud and ignore the growing field of online banking. However, there is a lack of publicly available data for both. The lack of publicly available data hinders the progress of the field and limits the investigation of potential solutions. With this work, we: (a) introduce FraudNLP, the first anonymised, publicly available dataset for online fraud detection, (b) benchmark machine and deep learning methods with multiple evaluation measures, (c) argue that online actions do follow rules similar to natural language and hence can be approached successfully by natural language processing methods.
|
|
| 4. |
- Cacciarelli, Davide, et al.
(author)
-
Active learning for data streams: a survey
- 2024
-
In: Machine Learning. - : Springer Nature. - 0885-6125 .- 1573-0565. ; 113:1, s. 185-239
-
Research review (peer-reviewed)abstract
- Online active learning is a paradigm in machine learning that aims to select the most informative data points to label from a data stream. The problem of minimizing the cost associated with collecting labeled observations has gained a lot of attention in recent years, particularly in real-world applications where data is only available in an unlabeled form. Annotating each observation can be time-consuming and costly, making it difficult to obtain large amounts of labeled data. To overcome this issue, many active learning strategies have been proposed in the last decades, aiming to select the most informative observations for labeling in order to improve the performance of machine learning models. These approaches can be broadly divided into two categories: static pool-based and stream-based active learning. Pool-based active learning involves selecting a subset of observations from a closed pool of unlabeled data, and it has been the focus of many surveys and literature reviews. However, the growing availability of data streams has led to an increase in the number of approaches that focus on online active learning, which involves continuously selecting and labeling observations as they arrive in a stream. This work aims to provide an overview of the most recently proposed approaches for selecting the most informative observations from data streams in real time. We review the various techniques that have been proposed and discuss their strengths and limitations, as well as the challenges and opportunities that exist in this area of research.
|
|
| 5. |
- De Raedt, Luc, 1964-, et al.
(author)
-
Probabilistic (logic) programming concepts
- 2015
-
In: Machine Learning. - : Springer-Verlag New York. - 0885-6125 .- 1573-0565. ; 100:1, s. 5-47
-
Journal article (peer-reviewed)abstract
- A multitude of different probabilistic programming languages exists today, allextending a traditional programming language with primitives to support modeling ofcomplex, structured probability distributions. Each of these languages employs its own prob-abilistic primitives, and comes with a particular syntax, semantics and inference procedure.This makes it hard to understand the underlying programming concepts and appreciate thedifferences between the different languages. To obtain a better understanding of probabilisticprogramming, we identify a number of core programming concepts underlying the primi-tives used by various probabilistic languages, discuss the execution mechanisms that theyrequire and use these to position and survey state-of-the-art probabilistic languages and theirimplementation. While doing so, we focus on probabilistic extensions oflogicprogramminglanguages such as Prolog, which have been considered for over 20 years.
|
|
| 6. |
- Dimitrakakis, Christos, 1975, et al.
(author)
-
Rollout sampling approximate policy iteration
- 2008
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 72:3, s. 157-171
-
Journal article (peer-reviewed)abstract
- Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.
|
|
| 7. |
- Ebberstein, Victor, 1995, et al.
(author)
-
A unified framework for online trip destination prediction
- 2022
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 111:10, s. 3839-3865
-
Journal article (peer-reviewed)abstract
- Trip destination prediction is an area of increasing importance in many applications such as trip planning, autonomous driving and electric vehicles. Even though this problem could be naturally addressed in an online learning paradigm where data is arriving in a sequential fashion, the majority of research has rather considered the offline setting. In this paper, we present a unified framework for trip destination prediction in an online setting, which is suitable for both online training and online prediction. For this purpose, we develop two clustering algorithms and integrate them within two online prediction models for this problem. We investigate the different configurations of clustering algorithms and prediction models on a real-world dataset. We demonstrate that both the clustering and the entire framework yield consistent results compared to the offline setting. Finally, we propose a novel regret metric for evaluating the entire online framework in comparison to its offline counterpart. This metric makes it possible to relate the source of erroneous predictions to either the clustering or the prediction model. Using this metric, we show that the proposed methods converge to a probability distribution resembling the true underlying distribution with a lower regret than all of the baselines.
|
|
| 8. |
- Fernandes, Sofia, et al.
(author)
-
WINTENDED : WINdowed TENsor decomposition for Densification Event Detection in time-evolving networks
- 2023
-
In: Machine Learning. - New York, NY : Springer. - 0885-6125 .- 1573-0565. ; 112:2, s. 459-481
-
Journal article (peer-reviewed)abstract
- Densification events in time-evolving networks refer to instants in which the network density, that is, the number of edges, is substantially larger than in the remaining. These events can occur at a global level, involving the majority of the nodes in the network, or at a local level involving only a subset of nodes.While global densification events affect the overall structure of the network, the same does not hold in local densification events, which may remain undetectable by the existing detection methods. In order to address this issue, we propose WINdowed TENsor decomposition for Densification Event Detection (WINTENDED) for the detection and characterization of both global and local densification events. Our method combines a sliding window decomposition with statistical tools to capture the local dynamics of the network and automatically find the irregular behaviours. According to our experimental evaluation, WINTENDED is able to spot global densification events at least as accurately as its competitors, while also being able to find local densification events, on the contrary to its competitors. © 2021, The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature.
|
|
| 9. |
|
|
| 10. |
- Grinberg, Nastasiya F., et al.
(author)
-
An evaluation of machine-learning for predicting phenotype: studies in yeast, rice, and wheat
- 2020
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 109:2, s. 251-277
-
Journal article (peer-reviewed)abstract
- In phenotype prediction the physical characteristics of an organism are predicted from knowledge of its genotype and environment. Such studies, often called genome-wide association studies, are of the highest societal importance, as they are of central importance to medicine, crop-breeding, etc. We investigated three phenotype prediction problems: one simple and clean (yeast), and the other two complex and real-world (rice and wheat). We compared standard machine learning methods; elastic net, ridge regression, lasso regression, random forest, gradient boosting machines (GBM), and support vector machines (SVM), with two state-of-the-art classical statistical genetics methods; genomic BLUP and a two-step sequential method based on linear regression. Additionally, using the clean yeast data, we investigated how performance varied with the complexity of the biological mechanism, the amount of observational noise, the number of examples, the amount of missing data, and the use of different data representations. We found that for almost all the phenotypes considered, standard machine learning methods outperformed the methods from classical statistical genetics. On the yeast problem, the most successful method was GBM, followed by lasso regression, and the two statistical genetics methods; with greater mechanistic complexity GBM was best, while in simpler cases lasso was superior. In the wheat and rice studies the best two methods were SVM and BLUP. The most robust method in the presence of noise, missing data, etc. was random forests. The classical statistical genetics method of genomic BLUP was found to perform well on problems where there was population structure. This suggests that standard machine learning methods need to be refined to include population structure information when this is present. We conclude that the application of machine learning methods to phenotype prediction problems holds great promise, but that determining which methods is likely to perform well on any given problem is elusive and non-trivial.
|
|
| 11. |
- Gummesson Svensson, Hampus, 1996, et al.
(author)
-
Utilizing reinforcement learning for de novo drug design
- 2024
-
In: MACHINE LEARNING. - 0885-6125 .- 1573-0565.
-
Journal article (peer-reviewed)abstract
- Deep learning-based approaches for generating novel drug molecules with specific properties have gained a lot of interest in the last few years. Recent studies have demonstrated promising performance for string-based generation of novel molecules utilizing reinforcement learning. In this paper, we develop a unified framework for using reinforcement learning for de novo drug design, wherein we systematically study various on- and off-policy reinforcement learning algorithms and replay buffers to learn an RNN-based policy to generate novel molecules predicted to be active against the dopamine receptor DRD2. Our findings suggest that it is advantageous to use at least both top-scoring and low-scoring molecules for updating the policy when structural diversity is essential. Using all generated molecules at an iteration seems to enhance performance stability for on-policy algorithms. In addition, when replaying high, intermediate, and low-scoring molecules, off-policy algorithms display the potential of improving the structural diversity and number of active molecules generated, but possibly at the cost of a longer exploration phase. Our work provides an open-source framework enabling researchers to investigate various reinforcement learning methods for de novo drug design.
|
|
| 12. |
- Haghir Chehreghani, Morteza, 1982, et al.
(author)
-
Learning representations from dendrograms
- 2020
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 109:9-10, s. 1779-1802
-
Journal article (peer-reviewed)abstract
- We propose unsupervised representation learning and feature extraction from dendrograms. The commonly used Minimax distance measures correspond to building a dendrogram with single linkage criterion, with defining specific forms of a level function and a distance function over that. Therefore, we extend this method to arbitrary dendrograms. We develop a generalized framework wherein different distance measures and representations can be inferred from different types of dendrograms, level functions and distance functions. Via an appropriate embedding, we compute a vector-based representation of the inferred distances, in order to enable many numerical machine learning algorithms to employ such distances. Then, to address the model selection problem, we study the aggregation of different dendrogram-based distances respectively in solution space and in representation space in the spirit of deep representations. In the first approach, for example for the clustering problem, we build a graph with positive and negative edge weights according to the consistency of the clustering labels of different objects among different solutions, in the context of ensemble methods. Then, we use an efficient variant of correlation clustering to produce the final clusters. In the second approach, we investigate the combination of different distances and features sequentially in the spirit of multi-layered architectures to obtain the final features. Finally, we demonstrate the effectiveness of our approach via several numerical studies.
|
|
| 13. |
- Haghir Chehreghani, Morteza, 1982
(author)
-
Shift of pairwise similarities for data clustering
- 2023
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 112:6, s. 2025-2051
-
Journal article (peer-reviewed)abstract
- Several clustering methods (e.g., Normalized Cut and Ratio Cut) divide the Min Cut cost function by a cluster dependent factor (e.g., the size or the degree of the clusters), in order to yield a more balanced partitioning. We, instead, investigate adding such regularizations to the original cost function. We first consider the case where the regularization term is the sum of the squared size of the clusters, and then generalize it to adaptive regularization of the pairwise similarities. This leads to shifting (adaptively) the pairwise similarities which might make some of them negative. We then study the connection of this method to Correlation Clustering and then propose an efficient local search optimization algorithm with fast theoretical convergence rate to solve the new clustering problem. In the following, we investigate the shift of pairwise similarities on some common clustering methods, and finally, we demonstrate the superior performance of the method by extensive experiments on different datasets.
|
|
| 14. |
- Haghir Chehreghani, Morteza, 1982
(author)
-
Unsupervised representation learning with Minimax distance measures
- 2020
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 109:11, s. 2063-2097
-
Journal article (peer-reviewed)abstract
- We investigate the use of Minimax distances to extract in a nonparametric way the features that capture the unknown underlying patterns and structures in the data. We develop a general-purpose and computationally efficient framework to employ Minimax distances with many machine learning methods that perform on numerical data. We study both computing the pairwise Minimax distances for all pairs of objects and as well as computing the Minimax distances of all the objects to/from a fixed (test) object. We first efficiently compute the pairwise Minimax distances between the objects, using the equivalence of Minimax distances over a graph and over a minimum spanning tree constructed on that. Then, we perform an embedding of the pairwise Minimax distances into a new vector space, such that their squared Euclidean distances in the new space equal to the pairwise Minimax distances in the original space. We also study the case of having multiple pairwise Minimax matrices, instead of a single one. Thereby, we propose an embedding via first summing up the centered matrices and then performing an eigenvalue decomposition to obtain the relevant features. In the following, we study computing Minimax distances from a fixed (test) object which can be used for instance in K-nearest neighbor search. Similar to the case of all-pair pairwise Minimax distances, we develop an efficient and general-purpose algorithm that is applicable with any arbitrary base distance measure. Moreover, we investigate in detail the edges selected by the Minimax distances and thereby explore the ability of Minimax distances in detecting outlier objects. Finally, for each setting, we perform several experiments to demonstrate the effectiveness of our framework.
|
|
| 15. |
- Johansson, Ulf, et al.
(author)
-
Efficient Venn predictors using random forests
- 2019
-
In: Machine Learning. - : SPRINGER. - 0885-6125 .- 1573-0565. ; 108:3, s. 535-550
-
Journal article (peer-reviewed)abstract
- Successful use of probabilistic classification requires well-calibrated probability estimates, i.e., the predicted class probabilities must correspond to the true probabilities. In addition, a probabilistic classifier must, of course, also be as accurate as possible. In this paper, Venn predictors, and its special case Venn-Abers predictors, are evaluated for probabilistic classification, using random forests as the underlying models. Venn predictors output multiple probabilities for each label, i.e., the predicted label is associated with a probability interval. Since all Venn predictors are valid in the long run, the size of the probability intervals is very important, with tighter intervals being more informative. The standard solution when calibrating a classifier is to employ an additional step, transforming the outputs from a classifier into probability estimates, using a labeled data set not employed for training of the models. For random forests, and other bagged ensembles, it is, however, possible to use the out-of-bag instances for calibration, making all training data available for both model learning and calibration. This procedure has previously been successfully applied to conformal prediction, but was here evaluated for the first time for Venn predictors. The empirical investigation, using 22 publicly available data sets, showed that all four versions of the Venn predictors were better calibrated than both the raw estimates from the random forest, and the standard techniques Platt scaling and isotonic regression. Regarding both informativeness and accuracy, the standard Venn predictor calibrated on out-of-bag instances was the best setup evaluated. Most importantly, calibrating on out-of-bag instances, instead of using a separate calibration set, resulted in tighter intervals and more accurate models on every data set, for both the Venn predictors and the Venn-Abers predictors.
|
|
| 16. |
- Johansson, Ulf, et al.
(author)
-
Regression conformal prediction with random forests
- 2014
-
In: Machine Learning. - : Springer-Verlag New York. - 0885-6125 .- 1573-0565. ; 97:1-2, s. 155-176
-
Journal article (peer-reviewed)abstract
- Regression conformal prediction produces prediction intervals that are valid, i.e., the probability of excluding the correct target value is bounded by a predefined confidence level. The most important criterion when comparing conformal regressors is efficiency; the prediction intervals should be as tight (informative) as possible. In this study, the use of random forests as the underlying model for regression conformal prediction is investigated and compared to existing state-of-the-art techniques, which are based on neural networks and k-nearest neighbors. In addition to their robust predictive performance, random forests allow for determining the size of the prediction intervals by using out-of-bag estimates instead of requiring a separate calibration set. An extensive empirical investigation, using 33 publicly available data sets, was undertaken to compare the use of random forests to existing stateof- the-art conformal predictors. The results show that the suggested approach, on almost all confidence levels and using both standard and normalized nonconformity functions, produced significantly more efficient conformal predictors than the existing alternatives.
|
|
| 17. |
|
|
| 18. |
- Kolb, Samuel, et al.
(author)
-
Learning constraints in spreadsheets and tabular data
- 2017
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; :106, s. 1441-1468
-
Journal article (peer-reviewed)abstract
- Spreadsheets, comma separated value files and other tabular data representations are in wide use today. However, writing, maintaining and identifying good formulas for tabular data and spreadsheets can be time-consuming and error-prone. We investigate the automatic learning of constraints (formulas and relations) in raw tabular data in an unsupervised way. We represent common spreadsheet formulas and relations through predicates and expressions whose arguments must satisfy the inherent properties of the constraint. The challenge is to automatically infer the set of constraints present in the data, without labeled examples or user feedback. We propose a two-stage generate and test method where the first stage uses constraint solving techniques to efficiently reduce the number of candidates, based on the predicate signatures. Our approach takes inspiration from inductive logic programming, constraint learning and constraint satisfaction. We show that we are able to accurately discover constraints in spreadsheets from various sources.
|
|
| 19. |
- Kolb, Samuel, et al.
(author)
-
Predictive spreadsheet autocompletion with constraints
- 2020
-
In: Machine Learning. - : Springer-Verlag New York. - 0885-6125 .- 1573-0565. ; 109:2, s. 307-325
-
Journal article (peer-reviewed)abstract
- Spreadsheets are arguably the most accessible data-analysis tool and are used by millions of people. Despite the fact that they lie at the core of most business practices, working with spreadsheets can be error prone, usage of formulas requires training and, crucially, spreadsheet users do not have access to state-of-the-art analysis techniques offered by machine learning. To tackle these issues, we introduce the novel task of predictive spreadsheet autocompletion, where the goal is to automatically predict the missing entries in the spreadsheets. This task is highly non-trivial: cells can hold heterogeneous data types and there might be unobserved relationships between their values, such as constraints or probabilistic dependencies. Critically, the exact prediction task itself is not given. We consider a simplified, yet non-trivial, setting and propose a principled probabilistic model to solve it. Our approach combines black-box predictive models specialized for different predictive tasks (e.g., classification, regression) and constraints and formulas detected by a constraint learner, and produces a maximally likely prediction for all target cells that is consistent with the constraints. Overall, our approach brings us one step closer to allowing end users to leverage machine learning in their workflows without writing a single line of code.
|
|
| 20. |
- Kuratomi Hernandez, Alejandro, et al.
(author)
-
Ijuice : integer JUstIfied counterfactual explanations
- 2024
-
In: Machine Learning. - 0885-6125 .- 1573-0565.
-
Journal article (peer-reviewed)abstract
- Counterfactual explanations modify the feature values of an instance in order to alter its prediction from an undesired to a desired label. As such, they are highly useful for providing trustworthy interpretations of decision-making in domains where complex and opaque machine learning algorithms are utilized. To guarantee their quality and promote user trust, they need to satisfy the faithfulness desideratum, when supported by the data distribution. We hereby propose a counterfactual generation algorithm for mixed-feature spaces that prioritizes faithfulness through k-justification, a novel counterfactual property introduced in this paper. The proposed algorithm employs a graph representation of the search space and provides counterfactuals by solving an integer program. In addition, the algorithm is classifier-agnostic and is not dependent on the order in which the feature space is explored. In our empirical evaluation, we demonstrate that it guarantees k-justification while showing comparable performance to state-of-the-art methods in feasibility, sparsity, and proximity.
|
|
| 21. |
- Larranaga, P., et al.
(author)
-
Machine Learning : Editorial
- 2005
-
In: Machine Learning. - : Springer-Verlag New York. - 0885-6125 .- 1573-0565. ; 59:3, s. 211-212
-
Journal article (other academic/artistic)
|
|
| 22. |
- Nitti, Davide, et al.
(author)
-
Planning in hybrid relational MDPs
- 2017
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; 106:12, s. 1905-1932
-
Journal article (peer-reviewed)abstract
- We study planning in relational Markov decision processes involving discrete and continuous states and actions, and an unknown number of objects. This combination of hybrid relational domains has so far not received a lot of attention. While both relational and hybrid approaches have been studied separately, planning in such domains is still challenging and often requires restrictive assumptions and approximations. We propose HYPE: a sample-based planner for hybrid relational domains that combines model-based approaches with state abstraction. HYPE samples episodes and uses the previous episodes as well as the model to approximate the Q-function. In addition, abstraction is performed for each sampled episode, this removes the complexity of symbolic approaches for hybrid relational domains. In our empirical evaluations, we show that HYPE is a general and widely applicable planner in domains ranging from strictly discrete to strictly continuous to hybrid ones, handles intricacies such as unknown objects and relational models. Moreover, empirical results showed that abstraction provides significant improvements.
|
|
| 23. |
- Nitti, Davide, et al.
(author)
-
Probabilistic logic programming for hybrid relational domains
- 2016
-
In: Machine Learning. - Boston : Springer-Verlag New York. - 0885-6125 .- 1573-0565. ; 103:3, s. 407-449
-
Journal article (peer-reviewed)abstract
- We introduce a probabilistic language and an efficient inference algorithm based on distributional clauses for static and dynamic inference in hybrid relational domains. Static inference is based on sampling, where the samples represent (partial) worlds (with discrete and continuous variables). Furthermore, we use backward reasoning to determine which facts should be included in the partial worlds. For filtering in dynamic models we combine the static inference algorithm with particle filters and guarantee that the previous partial samples can be safely forgotten, a condition that does not hold in most logical filtering frameworks. Experiments show that the proposed framework can outperform classic sampling methods for static and dynamic inference and that it is promising for robotics and vision applications. In addition, it provides the correct results in domains in which most probabilistic programming languages fail.
|
|
| 24. |
- Orhobor, Oghenejokpeme I., et al.
(author)
-
Imbalanced regression using regressor-classifier ensembles
- 2023
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 112:4, s. 1365-1387
-
Journal article (peer-reviewed)abstract
- We present an extension to the federated ensemble regression using classification algorithm, an ensemble learning algorithm for regression problems which leverages the distribution of the samples in a learning set to achieve improved performance. We evaluated the extension using four classifiers and four regressors, two discretizers, and 119 responses from a wide variety of datasets in different domains. Additionally, we compared our algorithm to two resampling methods aimed at addressing imbalanced datasets. Our results show that the proposed extension is highly unlikely to perform worse than the base case, and on average outperforms the two resampling methods with significant differences in performance.
|
|
| 25. |
- Orhobor, Oghenejokpeme I., et al.
(author)
-
Predicting rice phenotypes with meta and multi-target learning
- 2020
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 109:11, s. 2195-2212
-
Journal article (peer-reviewed)abstract
- The features in some machine learning datasets can naturally be divided into groups. This is the case with genomic data, where features can be grouped by chromosome. In many applications it is common for these groupings to be ignored, as interactions may exist between features belonging to different groups. However, including a group that does not influence a response introduces noise when fitting a model, leading to suboptimal predictive accuracy. Here we present two general frameworks for the generation and combination of meta-features when feature groupings are present. Furthermore, we make comparisons to multi-target learning, given that one is typically interested in predicting multiple phenotypes. We evaluated the frameworks and multi-target learning approaches on a genomic rice dataset where the regression task is to predict plant phenotype. Our results demonstrate that there are use cases for both the meta and multi-target approaches, given that overall, they significantly outperform the base case.
|
|
| 26. |
- Orsini, Francesco, et al.
(author)
-
kProbLog : an algebraic Prolog for machine learning
- 2017
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; :106, s. 1933-1969
-
Journal article (peer-reviewed)abstract
- We introduce kProbLog as a declarative logical language for machine learning. kProbLog is a simple algebraic extension of Prolog with facts and rules annotated by semi-ring labels. It allows to elegantly combine algebraic expressions with logic programs. We introduce the semantics of kProbLog, its inference algorithm, its implementation and provide convergence guarantees. We provide several code examples to illustrate its potential for a wide range of machine learning techniques. In particular, we show the encodings of state-of-the-art graph kernels such as Weisfeiler-Lehman graph kernels, propagation kernels and an instance of graph invariant kernels, a recent framework for graph kernels with continuous attributes. However, kProbLog is not limited to kernel methods and it can concisely express declarative formulations of tensor-based algorithms such as matrix factorization and energy-based models, and it can exploit semirings of dual numbers to perform algorithmic differentiation. Furthermore, experiments show that kProbLog is not only of theoretical interest, but can also be applied to real-world datasets. At the technical level, kProbLog extends aProbLog (an algebraic Prolog) by allowing multiple semirings to coexist in a single program and by introducing meta-functions for manipulating algebraic values.
|
|
| 27. |
- Paramonov, Sergey, et al.
(author)
-
Relational data factorization
- 2017
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; 106:12, s. 1867-1904
-
Journal article (peer-reviewed)abstract
- Motivated by an analogy with matrix factorization, we introduce the problem of factorizing relational data. In matrix factorization, one is given a matrix and has to factorize it as a product of other matrices. In relational data factorization (ReDF), the task is to factorize a given relation as a conjunctive query over other relations, i.e., as a combination of natural join operations. Given a conjunctive query and the input relation, the problem is to compute the extensions of the relations used in the query. Thus, relational data factorization is a relational analog of matrix factorization; it is also a form inverse querying as one has to compute the relations in the query from the result of the query. The result of relational data factorization is neither necessarily unique nor required to be a lossless decomposition of the original relation. Therefore, constraints can be imposed on the desired factorization and a scoring function is used to determine its quality (often similarity to the original data). Relational data factorization is thus a constraint satisfaction and optimization problem. We show how answer set programming can be used for solving relational data factorization problems.
|
|
| 28. |
- Pavlopoulos, Ioannis, 1983-, et al.
(author)
-
Automotive fault nowcasting with machine learning and natural language processing
- 2024
-
In: Machine Learning. - 0885-6125 .- 1573-0565. ; 113:2, s. 843-861
-
Journal article (peer-reviewed)abstract
- Automated fault diagnosis can facilitate diagnostics assistance, speedier troubleshooting, and better-organised logistics. Currently, most AI-based prognostics and health management in the automotive industry ignore textual descriptions of the experienced problems or symptoms. With this study, however, we propose an ML-assisted workflow for automotive fault nowcasting that improves on current industry standards. We show that a multilingual pre-trained Transformer model can effectively classify the textual symptom claims from a large company with vehicle fleets, despite the task’s challenging nature due to the 38 languages and 1357 classes involved. Overall, we report an accuracy of more than 80% for high-frequency classes and above 60% for classes with reasonable minimum support, bringing novel evidence that automotive troubleshooting management can benefit from multilingual symptom text classification.
|
|
| 29. |
- Verreet, V., et al.
(author)
-
Modeling PU learning using probabilistic logic programming
- 2024
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; 113:3, s. 1351-1372
-
Journal article (peer-reviewed)abstract
- The goal of learning from positive and unlabeled (PU) examples is to learn a classifier that predicts the posterior class probability. The challenge is that the available labels in the data are determined by (1) the true class, and (2) the labeling mechanism that selects which positive examples get labeled, where often certain examples have a higher probability to be selected than others. Incorrectly assuming an unbiased labeling mechanism leads to learning a biased classifier. Yet, this is what most existing methods do. A handful of methods makes more realistic assumptions, but they are either so general that it is impossible to distinguish between the effects of the true classification and of the labeling mechanism, or too restrictive to correctly model the real situation, or require knowledge that is typically unavailable. This paper studies how to formulate and integrate more realistic assumptions for learning better classifiers, by exploiting the strengths of probabilistic logic programming (PLP). Concretely, (1) we propose PU ProbLog: a PLP-based general method that allows to (partially) model the labeling mechanism. (2) We show that our method generalizes existing methods, in the sense that it can model the same assumptions. (3) Thanks to the use of PLP, our method supports also PU learning in relational domains. (4) Our empirical analysis shows that partially modeling the labeling bias, improves the learned classifiers.
|
|
| 30. |
- Wang, Yuxuan, et al.
(author)
-
Extrapolation is not the same as interpolation
- 2024
-
In: Machine Learning. - 0885-6125 .- 1573-0565. ; In Press
-
Journal article (peer-reviewed)abstract
- We propose a new machine learning formulation designed specifically for extrapolation. The textbook way to apply machine learning to drug design is to learn a univariate function that when a drug (structure) is input, the function outputs a real number (the activity): f(drug) → activity. However, experience in real-world drug design suggests that this formulation of the drug design problem is not quite correct. Specifically, what one is really interested in is extrapolation: predicting the activity of new drugs with higher activity than any existing ones. Our new formulation for extrapolation is based on learning a bivariate function that predicts the difference in activities of two drugs F(drug1, drug2) → difference in activity, followed by the use of ranking algorithms. This formulation is general and agnostic, suitable for finding samples with target values beyond the target value range of the training set. We applied the formulation to work with support vector machines, random forests, and Gradient Boosting Machines. We compared the formulation with standard regression on thousands of drug design datasets, gene expression datasets and material property datasets. The test set extrapolation metric was the identification of examples with greater values than the training set, and top-performing examples (within the top 10% of the whole dataset). On this metric our pairwise formulation vastly outperformed standard regression. Its proposed variations also showed a consistent outperformance. Its application in the stock selection problem further confirmed the advantage of this pairwise formulation.
|
|
| 31. |
- Wang, Zhendong, et al.
(author)
-
Glacier : guided locally constrained counterfactual explanations for time series classification
- 2024
-
In: Machine Learning. - 0885-6125 .- 1573-0565.
-
Journal article (peer-reviewed)abstract
- In machine learning applications, there is a need to obtain predictive models of high performance and, most importantly, to allow end-users and practitioners to understand and act on their predictions. One way to obtain such understanding is via counterfactuals, that provide sample-based explanations in the form of recommendations on which features need to be modified from a test example so that the classification outcome of a given classifier changes from an undesired outcome to a desired one. This paper focuses on the domain of time series classification, more specifically, on defining counterfactual explanations for univariate time series. We propose Glacier, a model-agnostic method for generating locally-constrained counterfactual explanations for time series classification using gradient search either on the original space or on a latent space that is learned through an auto-encoder. An additional flexibility of our method is the inclusion of constraints on the counterfactual generation process that favour applying changes to particular time series points or segments while discouraging changing others. The main purpose of these constraints is to ensure more reliable counterfactuals, while increasing the efficiency of the counterfactual generation process. Two particular types of constraints are considered, i.e., example-specific constraints and global constraints. We conduct extensive experiments on 40 datasets from the UCR archive, comparing different instantiations of Glacier against three competitors. Our findings suggest that Glacier outperforms the three competitors in terms of two common metrics for counterfactuals, i.e., proximity and compactness. Moreover, Glacier obtains comparable counterfactual validity compared to the best of the three competitors. Finally, when comparing the unconstrained variant of Glacier to the constraint-based variants, we conclude that the inclusion of example-specific and global constraints yields a good performance while demonstrating the trade-off between the different metrics.
|
|
| 32. |
- Yang, Wen-Chi, et al.
(author)
-
Lifted model checking for relational MDPs
- 2022
-
In: Machine Learning. - : Springer. - 0885-6125 .- 1573-0565. ; :111, s. 3797-3838
-
Journal article (peer-reviewed)abstract
- Probabilistic model checking has been developed for verifying systems that have stochastic and nondeterministic behavior. Given a probabilistic system, a probabilistic model checker takes a property and checks whether or not the property holds in that system. For this reason, probabilistic model checking provide rigorous guarantees. So far, however, probabilistic model checking has focused on propositional models where a state is represented by a symbol. On the other hand, it is commonly required to make relational abstractions in planning and reinforcement learning. Various frameworks handle relational domains, for instance, STRIPS planning and relational Markov Decision Processes. Using propositional model checking in relational settings requires one to ground the model, which leads to the well known state explosion problem and intractability. We present pCTL-REBEL, a lifted model checking approach for verifying pCTL properties of relational MDPs. It extends REBEL, a relational model-based reinforcement learning technique, toward relational pCTL model checking. PCTL-REBEL is lifted, which means that rather than grounding, the model exploits symmetries to reason about a group of objects as a whole at the relational level. Theoretically, we show that pCTL model checking is decidable for relational MDPs that have a possibly infinite domain, provided that the states have a bounded size. Practically, we contribute algorithms and an implementation of lifted relational model checking, and we show that the lifted approach improves the scalability of the model checking approach.
|
|
| 33. |
- Åkerblom, Niklas, 1987, et al.
(author)
-
Online Learning of Network Bottlenecks via Minimax Paths
- 2023
-
In: Machine Learning. - : Springer Science and Business Media LLC. - 0885-6125 .- 1573-0565. ; 112:1, s. 131-150
-
Journal article (peer-reviewed)abstract
- In this paper, we study bottleneck identification in networks via extracting minimax paths. Many real-world networks have stochastic weights for which full knowledge is not available in advance. Therefore, we model this task as a combinatorial semi-bandit problem to which we apply a combinatorial version of Thompson Sampling and establish an upper bound on the corresponding Bayesian regret. Due to the computational intractability of the problem, we then devise an alternative problem formulation which approximates the original objective. Finally, we experimentally evaluate the performance of Thompson Sampling with the approximate formulation on real-world directed and undirected networks.
|
|
| 34. |
|
|
| 35. |
|
|
| 36. |
- Henriksson, Roger, et al.
(author)
-
Impact of therapy on quality of life, neurocognitive function and their correlates in glioblastoma multiforme : a review
- 2011
-
In: Journal of Neuro-Oncology. - Boston : Nijhoff. - 0167-594X .- 1573-7373. ; 104:3, s. 639-646
-
Research review (peer-reviewed)abstract
- The maintenance of quality of life (QoL) in patients with high-grade glioma is an important endpoint during treatment, particularly in those with glioblastoma multiforme (GBM) given its dismal prognosis despite limited advances in standard therapy. It has proven difficult to identify new therapies that extend survival in patients with recurrent GBM, so one of the primary aims of new therapies is to reduce morbidity, restore or preserve neurologic functions, and the capacity to perform daily activities. Apart from temozolomide, cytotoxic chemotherapeutic agents do not appear to significantly impact response or survival, but produce toxicity that is likely to negatively impact QoL. New biological agents, such as bevacizumab, can induce a clinically meaningful proportion of durable responses among patients with recurrent GBM with an acceptable safety profile. Emerging evidence suggests that bevacizumab produces an improvement or preservation of neurocognitive function in GBM patients, suggestive of QoL improvement, in most poor-prognosis patients who would otherwise be expected to show a sudden and rapid deterioration in QoL.
|
|
| 37. |
|
|
