| 1. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
Empirical Study : Visual Analytics for Comparing Stacking to Blending Ensemble Learning
- 2021
-
In: Proceedings of the 23rd International Conference on Control Systems and Computer Science (CSCS23), 26–28 May 2021, Bucharest, Romania. - : IEEE. - 9781665439404 - 9781665439398 ; , s. 1-8
-
Conference paper (other academic/artistic)abstract
- Stacked generalization (also called stacking) is an ensemble method in machine learning that uses a metamodel to combine the predictive results of heterogeneous base models arranged in at least one layer. K-fold cross-validation is employed at the various stages of training in this method. Nonetheless, another validation strategy is to try out several splits of data leading to different train and test sets for the base models and then use only the latter to train the metamodel—this is known as blending. In this work, we present a modification of an existing visual analytics system, entitled StackGenVis, that now supports the process of composing robust and diverse ensembles of models with both aforementioned methods. We have built multiple ensembles using our system with the two respective methods, and we tested the performance with six small- to large-sized data sets. The results indicate that stacking is significantly more powerful than blending based on three performance metrics. However, the training times of the base models and the final ensembles are lower and more stable during various train/test splits in blending rather than stacking.
|
|
| 2. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
FeatureEnVi : Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches
- 2022
-
In: IEEE Transactions on Visualization and Computer Graphics. - : IEEE. - 1077-2626 .- 1941-0506. ; 28:4, s. 1773-1791
-
Journal article (peer-reviewed)abstract
- The machine learning (ML) life cycle involves a series of iterative steps, from the effective gathering and preparation of the data—including complex feature engineering processes—to the presentation and improvement of results, with various algorithms to choose from in every step. Feature engineering in particular can be very beneficial for ML, leading to numerous improvements such as boosting the predictive results, decreasing computational times, reducing excessive noise, and increasing the transparency behind the decisions taken during the training. Despite that, while several visual analytics tools exist to monitor and control the different stages of the ML life cycle (especially those related to data and algorithms), feature engineering support remains inadequate. In this paper, we present FeatureEnVi, a visual analytics system specifically designed to assist with the feature engineering process. Our proposed system helps users to choose the most important feature, to transform the original features into powerful alternatives, and to experiment with different feature generation combinations. Additionally, data space slicing allows users to explore the impact of features on both local and global scales. FeatureEnVi utilizes multiple automatic feature selection techniques; furthermore, it visually guides users with statistical evidence about the influence of each feature (or subsets of features). The final outcome is the extraction of heavily engineered features, evaluated by multiple validation metrics. The usefulness and applicability of FeatureEnVi are demonstrated with two use cases and a case study. We also report feedback from interviews with two ML experts and a visualization researcher who assessed the effectiveness of our system.
|
|
| 3. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
StackGenVis : Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics
- 2021
-
In: IEEE Transactions on Visualization and Computer Graphics. - : IEEE Computer Society Digital Library. - 1077-2626 .- 1941-0506. ; 27:2, s. 1547-1557
-
Journal article (peer-reviewed)abstract
- In machine learning (ML), ensemble methods—such as bagging, boosting, and stacking—are widely-established approaches that regularly achieve top-notch predictive performance. Stacking (also called "stacked generalization") is an ensemble method that combines heterogeneous base models, arranged in at least one layer, and then employs another metamodel to summarize the predictions of those models. Although it may be a highly-effective approach for increasing the predictive performance of ML, generating a stack of models from scratch can be a cumbersome trial-and-error process. This challenge stems from the enormous space of available solutions, with different sets of data instances and features that could be used for training, several algorithms to choose from, and instantiations of these algorithms using diverse parameters (i.e., models) that perform differently according to various metrics. In this work, we present a knowledge generation model, which supports ensemble learning with the use of visualization, and a visual analytics system for stacked generalization. Our system, StackGenVis, assists users in dynamically adapting performance metrics, managing data instances, selecting the most important features for a given data set, choosing a set of top-performant and diverse algorithms, and measuring the predictive performance. In consequence, our proposed tool helps users to decide between distinct models and to reduce the complexity of the resulting stack by removing overpromising and underperforming models. The applicability and effectiveness of StackGenVis are demonstrated with two use cases: a real-world healthcare data set and a collection of data related to sentiment/stance detection in texts. Finally, the tool has been evaluated through interviews with three ML experts.
|
|
| 4. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
- 2020
-
In: Computer graphics forum (Print). - : John Wiley & Sons. - 0167-7055 .- 1467-8659. ; 39:3, s. 713-756
-
Journal article (peer-reviewed)abstract
- Machine learning (ML) models are nowadays used in complex applications in various domains such as medicine, bioinformatics, and other sciences. Due to their black box nature, however, it may sometimes be hard to understand and trust the results they provide. This has increased the demand for reliable visualization tools related to enhancing trust in ML models, which has become a prominent topic of research in the visualization community over the past decades. To provide an overview and present the frontiers of current research on the topic, we present a State-of-the-Art Report (STAR) on enhancing trust in ML models with the use of interactive visualization. We define and describe the background of the topic, introduce a categorization for visualization techniques that aim to accomplish this goal, and discuss insights and opportunities for future research directions. Among our contributions is a categorization of trust against different facets of interactive ML, expanded and improved from previous research. Our results are investigated from different analytical perspectives: (a) providing a statistical overview, (b) summarizing key findings, (c) performing topic analyses, and (d) exploring the data sets used in the individual papers, all with the support of an interactive web-based survey browser. We intend this survey to be beneficial for visualization researchers whose interests involve making ML models more trustworthy, as well as researchers and practitioners from other disciplines in their search for effective visualization techniques suitable for solving their tasks with confidence and conveying meaning to their data.
|
|
| 5. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
VisEvol : Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
- 2021
-
In: Computer graphics forum (Print). - : John Wiley & Sons. - 0167-7055 .- 1467-8659. ; 40:3, s. 201-214
-
Journal article (peer-reviewed)abstract
- During the training phase of machine learning (ML) models, it is usually necessary to configure several hyperparameters. This process is computationally intensive and requires an extensive search to infer the best hyperparameter set for the given problem. The challenge is exacerbated by the fact that most ML models are complex internally, and training involves trial-and-error processes that could remarkably affect the predictive result. Moreover, each hyperparameter of an ML algorithm is potentially intertwined with the others, and changing it might result in unforeseeable impacts on the remaining hyperparameters. Evolutionary optimization is a promising method to try and address those issues. According to this method, performant models are stored, while the remainder are improved through crossover and mutation processes inspired by genetic algorithms. We present VisEvol, a visual analytics tool that supports interactive exploration of hyperparameters and intervention in this evolutionary procedure. In summary, our proposed tool helps the user to generate new models through evolution and eventually explore powerful hyperparameter combinations in diverse regions of the extensive hyperparameter space. The outcome is a voting ensemble (with equal rights) that boosts the final predictive performance. The utility and applicability of VisEvol are demonstrated with two use cases and interviews with ML experts who evaluated the effectiveness of the tool.
|
|
| 6. |
- Chatzimparmpas, Angelos, 1994-, et al.
(author)
-
Visualization for Trust in Machine Learning Revisited : The State of the Field in 2023
- 2024
-
In: IEEE Computer Graphics and Applications. - : IEEE. - 0272-1716 .- 1558-1756. ; 44:3, s. 99-113
-
Journal article (peer-reviewed)abstract
- Visualization for explainable and trustworthy machine learning remains one of the most important and heavily researched fields within information visualization and visual analytics with various application domains, such as medicine, finance, and bioinformatics. After our 2020 state-of-the-art report comprising 200 techniques, we have persistently collected peer-reviewed articles describing visualization techniques, categorized them based on the previously established categorization schema consisting of 119 categories, and provided the resulting collection of 542 techniques in an online survey browser. In this survey article, we present the updated findings of new analyses of this dataset as of fall 2023 and discuss trends, insights, and eight open challenges for using visualizations in machine learning. Our results corroborate the rapidly growing trend of visualization techniques for increasing trust in machine learning models in the past three years, with visualization found to help improve popular model explainability methods and check new deep learning architectures, for instance.
|
|
| 7. |
- Fatemi, Masoud, 1990-, et al.
(author)
-
Self-Similarity of Twitter Users
- 2021
-
In: Proceedings of the 2021 Swedish Workshop on Data Science (SweDS). - : IEEE. - 9781665418300
-
Conference paper (peer-reviewed)abstract
- Earlier studies have established that the (perceived) similarity of users is highly subjective and reflects more on how people respect/admire others rather than their characteristics or behavioral similarities. We study this phenomenon among Twitter users, and while confirm that it is indeed the case, we further explore the components of similarity by investigating it using data from three categories (interactions between egos and alters, profile-based activity history, and linguistic content in the messages). We use interactions as estimation for admiration and observe that it has more impact and a higher correlation to the perceived similarity than other objective measures, including similarity based on user profiles and their use of hashtags.
|
|
| 8. |
- Huang, Zeyang, 1998-, et al.
(author)
-
Towards a Visual Analytics System for Emotion Trajectories in Multiparty Conversations
- 2024
-
In: Poster Proceedings of the 26th Eurographics Conference on Visualization (EuroVis 2024 Posters). - : Eurographics - European Association for Computer Graphics. - 9783038682585
-
Conference paper (peer-reviewed)abstract
- Visualizing sentiments in textual data has received growing interest; however, representing emotions within interlocutor relationships and associating them with the temporal progression of dialogues remains challenging. In this poster abstract, we describe the ongoing work on a visual analytics tool designed for analyzing emotion trajectories within dialogue collections composed of utterances from multiple speakers. The proposed tool provides exploration at different levels of detail to complex multigraphs, where edges represent direct responses between speakers through their utterances. Our approach includes several selection strategies for connecting different views: summaries of emotion transitions across dialogue groups, detailed analyses of individual utterances within specific dialogues of interest in interlocutor networks, and close reading. The tool aims to support model development in natural language processing by allowing users to explore text corpora interactively.
|
|
| 9. |
- Huang, Zeyang, 1998-, et al.
(author)
-
Towards an Exploratory Visual Analytics System for Multivariate Subnetworks in Social Media Analysis
- 2022
-
In: Poster Abstracts, IEEE Visualization and Visual Analytics (VIS '22). - : IEEE.
-
Conference paper (peer-reviewed)abstract
- Identifying sociolinguistic attributes of inter-community interactions is essential for understanding the polarization of social network communities. A wide range of computational text and network analysis methods may be applicable for this task, however, interpretation of the respective results and investigation of particularly interesting cases and subnetworks are difficult due to the scale and complexity of the data, e.g., for the Reddit platform. In this poster paper, we present an interactive visual analysis interface that facilitates network exploration and comparison at different topological and multivariate attribute scales. Users are able to investigate text- and network-based properties of social network community interactions, identify anomalies of conflict starters, or gain insight into multivariate anomalies behind groups of negative social media posts.
|
|
| 10. |
- Huang, Zeyang, 1998-, et al.
(author)
-
VA + Embeddings STAR : A State-of-the-Art Report on the Use of Embeddings in Visual Analytics
- 2023
-
In: Computer graphics forum (Print). - : John Wiley & Sons. - 0167-7055 .- 1467-8659. ; 42:3, s. 539-571
-
Journal article (peer-reviewed)abstract
- Over the past years, an increasing number of publications in information visualization, especially within the field of visual analytics, have mentioned the term “embedding” when describing the computational approach. Within this context, embeddings are usually (relatively) low-dimensional, distributed representations of various data types (such as texts or graphs), and since they have proven to be extremely useful for a variety of data analysis tasks across various disciplines and fields, they have become widely used. Existing visualization approaches aim to either support exploration and interpretation of the embedding space through visual representation and interaction, or aim to use embeddings as part of the computational pipeline for addressing downstream analytical tasks. To the best of our knowledge, this is the first survey that takes a detailed look at embedding methods through the lens of visual analytics, and the purpose of our survey article is to provide a systematic overview of the state of the art within the emerging field of embedding visualization. We design a categorization scheme for our approach, analyze the current research frontier based on peer-reviewed publications, and discuss existing trends, challenges, and potential research directions for using embeddings in the context of visual analytics. Furthermore, we provide an interactive survey browser for the collected and categorized survey data, which currently includes 122 entries that appeared between 2007 and 2023.
|
|
