| 1. |
- Akram Hassan, Kahin, 1990-
(författare)
-
It’s About Time : User-centered Evaluation of Visual Representations for Temporal Data
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The primary goal for collecting and analyzing temporal data differs between individuals and their domain of expertise e.g., forecasting might be the goal in meteorology, anomaly detection might be the goal in finance. While the goal differs, one common denominator is the need for exploratory analysis of the temporal data, as this can aid the search for useful information. However, as temporal data can be challenging to understand and visualize, selecting appropriate visual representations for the domain and data at hand becomes a challenge. Moreover, many visual representations can show a single variable that changes over time, displaying multiple variables in a clear and easily accessible way is much harder, and inference-making and pattern recognition often require visualization of multiple variables. Additionally, as visualization aims to gain insight, it becomes crucial to investigate whether the representations used help users gain this insight. Furthermore, to create effective and efficient visual analysis tools, it is vital to understand the structure of the data, how this data can be represented, and have a clear understanding of the user needs. Developing useful visual representations can be challenging, but through close collaboration and involvement of end-users in the entire process, useful results can be accomplished. This thesis aims to investigate the usability of different visual representations for different types of multivariate temporal data, users, and tasks. Five user studies have been conducted to investigate different representation spaces, layouts, and interaction methods for investigating representations’ ability to facilitate users when analyzing and exploring such temporal datasets. The first study investigated and evaluated the experience of different radial design ideas for finding and comparison tasks when presenting hourly data based on an analog clock metaphor. The second study investigated 2D and 3D parallel coordinates for pattern finding. In the third study, the usability of three linear visual representations for presenting indoor climate data was investigated with domain experts. The fourth study continued on the third study and developed and evaluated a visual analytics tool with different visual representations and interaction techniques with domain experts. Finally, in the fifth study, another visual analytics tool presenting visual representations of temporal data was developed and evaluated with domain experts working and conducting experiments in Antarctica. The research conducted within the scope of this thesis concludes that it is vital to understand the characteristics of the temporal data and user needs for selecting the optimal representations. Without this knowledge, it becomes much harder to choose visual representations to help users gain insight from the data. It is also crucial to evaluate the perception and usability of the chosen visual representations.
|
|
| 2. |
- Besançon, Lonni, et al.
(författare)
-
Open up: a survey on open and non-anonymized peer reviewing
- 2020
-
Ingår i: BMC Research Integrity and Peer Review. - : BioMed Central. - 2058-8615. ; 5:8, s. 1-11
-
Tidskriftsartikel (refereegranskat)abstract
- BackgroundOur aim is to highlight the benefits and limitations of open and non-anonymized peer review. Our argument is based on the literature and on responses to a survey on the reviewing process of alt.chi, a more or less open review track within the so-called Computer Human Interaction (CHI) conference, the predominant conference in the field of human-computer interaction. This track currently is the only implementation of an open peer review process in the field of human-computer interaction while, with the recent increase in interest in open scientific practices, open review is now being considered and used in other fields.MethodsWe ran an online survey with 30 responses from alt.chi authors and reviewers, collecting quantitative data using multiple-choice questions and Likert scales. Qualitative data were collected using open questions.ResultsOur main quantitative result is that respondents are more positive to open and non-anonymous reviewing for alt.chi than for other parts of the CHI conference. The qualitative data specifically highlight the benefits of open and transparent academic discussions. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.ConclusionWhile the benefits are quite clear and the system is generally well-liked by alt.chi participants, they remain reluctant to see it used in other venues. This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation.
|
|
| 3. |
- Elmquist, Elias, 1996-, et al.
(författare)
-
OpenSpace Sonification: Complementing Visualization of the Solar System with Sound
- 2021
-
Ingår i: Proceedings of the 26th International Conference on Auditory Display (ICAD 2021). - icad.org : The International Community for Auditory Display. - 0967090474 - 9780967090474 ; , s. 135-142
-
Konferensbidrag (refereegranskat)abstract
- Data visualization software is commonly used to explore outer space in a planetarium environment, where the visuals of the software is typically accompanied with a narrator and supplementary background music. By letting sound take a bigger role in these kinds of presentations, a more informative and immersive experience can be achieved. The aim of the present study was to explore how sonification can be used as a complement to the visualization software OpenSpace to convey information about the Solar System, as well as increasing the perceived immersiveness for the audience in a planetarium environment. This was investigated by implementing a sonification that conveyed planetary properties, such as the size and orbital period of a planet, by mapping this data to sonification parameters. With a user-centered approach, the sonification was designed iteratively and evaluated in both an online and planetarium environment. The results of the evaluations show that the participants found the sonification informative and interesting, which suggest that sonification can be beneficially used as a complement to visualization in a planetarium environment.
|
|
| 4. |
- Elmquist, Elias, 1996-, et al.
(författare)
-
Parallel Chords: an audio-visual analytics design for parallel coordinates
- 2024
-
Ingår i: Personal and Ubiquitous Computing. - : Springer. - 1617-4909 .- 1617-4917.
-
Tidskriftsartikel (refereegranskat)abstract
- One of the commonly used visualization techniques for multivariate data is the parallel coordinates plot. It provides users with a visual overview of multivariate data and the possibility to interactively explore it. While pattern recognition is a strength of the human visual system, it is also a strength of the auditory system. Inspired by the integration of the visual and auditory perception in everyday life, we introduce an audio-visual analytics design named Parallel Chords combining both visual and auditory displays. Parallel Chords lets users explore multivariate data using both visualization and sonification through the interaction with the axes of a parallel coordinates plot. To illustrate the potential of the design, we present (1) prototypical data patterns where the sonification helps with the identification of correlations, clusters, and outliers, (2) a usage scenario showing the sonification of data from non-adjacent axes, and (3) a controlled experiment on the sensitivity thresholds of participants when distinguishing the strength of correlations. During this controlled experiment, 35 participants used three different display types, the visualization, the sonification, and the combination of these, to identify the strongest out of three correlations. The results show that all three display types enabled the participants to identify the strongest correlation — with visualization resulting in the best sensitivity. The sonification resulted in sensitivities that were independent from the type of displayed correlation, and the combination resulted in increased enjoyability during usage.
|
|
| 5. |
- Elmquist, Elias, 1996-, et al.
(författare)
-
SonAir: the design of a sonification of radar data for air traffic control
- 2023
-
Ingår i: Journal on Multimodal User Interfaces. - : SPRINGER. - 1783-7677 .- 1783-8738. ; 17:3, s. 137-149
-
Tidskriftsartikel (refereegranskat)abstract
- Along with the increase of digitalization and automation, a new kind of working environment is emerging in the field of air traffic control. Instead of situating the control tower at the airport, it is now possible to remotely control the airport at any given location, i.e. in a remote tower center (RTC). However, by controlling the airport remotely, the situational awareness and sense of presence might be compromised. By using directional sound, a higher situational awareness could potentially be achieved while also offloading the visual perception which is heavily used in air traffic control. Suitable use cases for sonification in air traffic control were found through workshops with air traffic controllers. A sonification design named SonAir was developed based on the outcome of the workshops, and was integrated with an RTC simulator for evaluating to what degree SonAir could support air traffic controllers in their work. The results suggest that certain aspects of SonAir could be useful for air traffic controllers. A continuous sonification where the spatial positioning of aircraft were conveyed was experienced to be partially useful, but the intrusiveness of SonAir should be further considered to fit the air traffic controllers’ needs. An earcon that conveyed when an aircraft enters the airspace and from which direction was considered useful to support situational awareness.
|
|
| 6. |
- Enge, Kajetan, et al.
(författare)
-
Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization
- 2024
-
Ingår i: Computer graphics forum (Print). - : Wiley-Blackwell Publishing Inc.. - 0167-7055 .- 1467-8659. ; 43:3
-
Tidskriftsartikel (refereegranskat)abstract
- The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non-speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a collection of work that bridges the borders of the two communities, hence a collection of work that aims to integrate the two techniques into one form of audiovisual display, which we argue to be “more than the sum of the two.” We introduce and motivate a classification system applicable to such audiovisual displays and categorize a corpus of 57 academic publications that appeared between 2011 and 2023 in categories such as reading level, dataset type, or evaluation system, to mention a few. The corpus also enables a meta-analysis of the field, including regularly occurring design patterns such as type of visualization and sonification techniques, or the use of visual and auditory channels, showing an overall diverse field with different designs. An analysis of a co-author network of the field shows individual teams without many interconnections. The body of work covered in this STAR also relates to three adjacent topics: audiovisual monitoring, accessibility, and audiovisual data art. These three topics are discussed individually in addition to the systematically conducted part of this research. The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field.
|
|
| 7. |
- Fredriksson Häägg, Anna, et al.
(författare)
-
Insight-based Evaluation of a Map-based Dashboard
- 2022
-
Konferensbidrag (refereegranskat)abstract
- Map-based dashboards are used for data exploration every day. The present study used an insight-based methodology for evaluating a map-based dashboard that presents research findings of water management and ecosystem services in the Amazon. In addition to analyzing the insights gained from using the dashboard, the evaluation method was compared to standardized questionnaires and task-based evaluations. The result suggests that the dashboard enabled the participants to gain domain-relevant, complex insights regarding the topic presented. Furthermore, the insight-based analysis highlighted unexpected insights and hypotheses regarding causes and potential adaptation strategies for remediation. Although time- and resource-consuming, the insight-based methodology was shown to have the potential of thoroughly analyzing how end users can utilize map-based dashboards for data exploration and decision making. Finally, the insight-based methodology is argued to evaluate tools in scenarios more similar to real-life usage, compared to task-based evaluation methods.
|
|
| 8. |
- Gorenko, Ivar, et al.
(författare)
-
Supporting Astrophysical Visualization with Sonification
- 2024
-
Konferensbidrag (refereegranskat)abstract
- This poster presents initial design steps exploring how sonification can be used to support visualization for comprehension of space and time in astronomical data. Radio signals travel at the speed of light. With a visualization of the universe, it is possible to travel faster than light and pass the radio waves leaving earth. We can then travel back in time. We propose to use sonification consisting of songs representing each year as a musical journey through space and time to create an engaging experience.
|
|
| 9. |
- Hassan, Kahin Akram, et al.
(författare)
-
A Study on 2D and 3D Parallel Coordinates for Pattern Identification in Temporal Multivariate Data
- 2019
-
Konferensbidrag (refereegranskat)abstract
- Parallel coordinates are commonly used for non-temporal multivariate data, but there is little support for their usability for displaying temporal multivariate data. In this paper, we introduce a study evaluating the usability of 2D and 3D parallel coordinates for pattern identification in temporal multivariate data. The results indicate that 3D parallel coordinates have higher usability, as measured with higher accuracy and faster response time as well as subjective ratings, compared to 2D.
|
|
| 10. |
- Hassan, Kahin Akram, et al.
(författare)
-
A Study on Visual Representations for Active Plant Wall Data Analysis
- 2019
-
Ingår i: DATA. - : MDPI. - 2306-5729. ; 4:2
-
Tidskriftsartikel (refereegranskat)abstract
- The indoor climate is closely related to human health, well-being, and comfort. Thus, an understanding of the indoor climate is vital. One way to improve the indoor climates is to place an aesthetically pleasing active plant wall in the environment. By collecting data using sensors placed in and around the plant wall both the indoor climate and the status of the plant wall can be monitored and analyzed. This manuscript presents a user study with domain experts in this field with a focus on the representation of such data. The experts explored this data with a Line graph, a Horizon graph, and a Stacked area graph to better understand the status of the active plant wall and the indoor climate. Qualitative measures were collected with Think-aloud protocol and semi-structured interviews. The study resulted in four categories of analysis tasks: Overview, Detail, Perception, and Complexity. The Line graph was found to be preferred for use in providing an overview, and the Horizon graph for detailed analysis, revealing patterns and showing discernible trends, while the Stacked area graph was generally not preferred. Based on these findings, directions for future research are discussed and formulated. The results and future directions of this research can facilitate the analysis of multivariate temporal data, both for domain users and visualization researchers.
|
|
| 11. |
- Niklas, Rönnberg, 1973-, et al.
(författare)
-
Photone: Exploring modal synergy in photographic images and music
- 2018
-
Ingår i: International Conference on Auditory Display, 2018 [30]. - Arlington, Virginia : Georgia Institute of Technology. ; , s. 73-79
-
Konferensbidrag (refereegranskat)abstract
- We present Photone, an interactive installation combining photographic images and musical sonification. An image is displayed, and a dynamic musical score is generated based on the overall color properties of the image and the color value of the pixel under the cursor. Hence, the music changes as the user moves the cursor. This simple approach turns out to have interesting experiential qualities in use. The composition of images and music invites the user to explore the combination of hues and textures, and musical sounds. We characterize the resulting experience in Photone as one of modal synergy where visual and auditory output combine holistically with the chosen interaction technique. This tentative finding is potentially relevant to further research in auditory displays and multimodal interaction.
|
|
| 12. |
- Niklas, Rönnberg, 1973-
(författare)
-
Sonification Enhances Perception of Color Intensity
- 2017
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This poster presents an interactive sonification experiment, designed to evaluate possible benefits of sonification in information visual- ization. The aim of the present study was to explore the use of composed and deliberately designed musical sounds to enhance perception of color intensity in visual representations. It was hypoth- esized, that by using musical sounds for sonification perception of color intensity would be improved. In this evaluation, sonification was mapped to color intensity in visual representations, and the par- ticipants had to identify and mark the highest color intensity, as well as answer a questionnaire about their experience. Both quantitative and qualitative preliminary results suggest a benefit of sonification, and indicate that sonification is useful in data exploration.
|
|
| 13. |
|
|
| 14. |
- Rind, Alexander, et al.
(författare)
-
Integrating Sonification and Visualization – But Why?
- 2024
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- The research communities studying visualization and sonification share exceptionally similar goals, essentially aiming to make data interpretable to humans. One community uses visual representations, while the other employs auditory (nonspeech) repre- sentations of data. Although the two communities have much in common, they developed mostly in parallel, with only compara- tively few examples of integrated audiovisual data analysis idioms presented so far. This panel brings together researchers from both the fields of visualization and sonification to collectively discuss the question: ‘Integrating Sonification and Visualization – but why?’ In the panel discussion, we will tackle this question along two main hypotheses: Combining the modalities to (1) increase the “bandwidth from data to brain,” or (2) to increase a user’s personal engagement during the data analysis process. On the one hand, designers might aim to communicate more data in less time or gain more and more complex insights from the data by using a multi-modal display. This argument follows an understanding that two senses should be capable of processing more information than “just” one. On the other hand, sometimes, a more engaged analysis of the represented data is desirable. Engagement with data visualization stands as a crucial topic in numerous contexts within our field, encouraging “deeper” thinking by expert analysts, readers of data journalism articles, and students in educational settings. We hypothesize that integrating visualization with sonification holds the potential to enhance user engagement during analysis. Through the panel discussion, we want to delve into the spectrum between aiming for bandwidth and engagement, seeking to understand the opportunities and challenges of integrating sonification and visualization.
|
|
| 15. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Adverse listening conditions affect short-term memory storage and processing of speech for older adults with hearing impairment
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Purpose: Previous work has shown an effect of noise type on memory for intelligible speech. The aim ofthis study was to investigate the effect of background noise on memory performance of intelligible speech for older adults with hearing impairment using the Auditory Inference Span Test (AIST).Method: Twenty participants with ages between 67 and 80 years with symmetrical hearing loss (29 to 47dB HL) performed the AIST, which requires processing of five-word sentences at three memoryload levels (MLLs) in three listening conditions: Quiet, steady-state noise (SSN), and backgroundvoices (ISTS). Individualized SNRs targeted 90% speech intelligibility. AIST performance reflects the amount of cognitive capacity occupied in listening, and consequently indicates the amount of listening effort. Working memory capacity (WMC) was assessed using the reading span test, and updating ability (UA) was assessed using the letter memory test.Results: AIST performance decreased in background noise and with increasing MLL. It was related to UA and age but not to WMC. Response times on questions designed to probe sentence recognition increased with the addition of background noise.Conclusions: The results demonstrate that the addition of background noise requires more cognitive resourcesto maintain speech recognition performance, leading to higher demands on the cognitive capacity,higher listening effort as measured by poorer memory performance, and longer AIST responsetimes. However, the type of background noise, SSN or ISTS, affected memory performance similarly.
|
|
| 16. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
AIST - Ett test av lyssningsansträngning
- 2011
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Hörapparatsanpassning kan ses som en process för att minska en persons lyssningsansträngning, men det är oklart hur man bäst mäter lyssningsansträngning på ett objektivt sätt. Auditory Inference Span Test (AIST) syftar därför till att utvecklas till ett kliniskt instrument att använda vid hörapparatsanpassning för att mäta en patients ansträngning att uppfatta tal. AIST är ett kombinerat hörsel-, minnes- och bearbetningstest. Testet bygger på idén att ju mer kognitiva resurser som går åt för att bearbeta och förstå tal, desto mindre kognitiva resurser finns kvar för att minnas och lagra talinformation. Testet använder Hagermans meningar i brus, och försökspersonen behöver minnas och bearbeta informationen i talmaterialet för att kunna besvara frågor om innehållet. Poäng på frågorna samt reaktionstid mäts som mått på lyssningsansträngning. Data från pilottester visar på att AIST kan bli ett väl anpassat test för kliniskt bruk för att mäta lyssningsansträngning.
|
|
| 17. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
An objective measure of listening effort : The Auditory Inference Span Test
- 2011
-
Konferensbidrag (refereegranskat)abstract
- One aim of hearing aid fitting is to ease the patient’s effort in understanding speech, i.e. the listening effort needed to perceive speech in different sound environments. To obtain a good hearing aid fitting, knowledge about the patient’s auditory as well as cognitive abilities seems to be important. However, listening effort is usually not included as a fitting criterion, partly as it is not clear how to measure listening effort objectively. The Auditory Inference Span Test (AIST) is a dual-task hearing-in-noise test, that combines auditory and memory processing. The basis for the test is that when more cognitive resources are required for understanding speech, less cognitive resources are available for storage and processing of the speech information. In AIST, Hagerman sentences are presented in noise and the subject is required to recall and process the sentence information. Recall ability is tested with different cognitive loads. Button-press responses are recorded and used as an estimate of listening effort. In a pilot study, listeners showed decreasing accuracy with increasing cognitive load and longer reaction time at maximum cognitive load, suggesting that the test may be suited as a clinical test for listening effort. In an ongoing study, the AIST is being evaluated in relation to other auditory and cognitive measures: baseline audiometry (audiogram) and speech in noise test (Hagerman sentences) as well as text based dual processing and storage test (reading span) and updating (letter memory test), as well as subjective rating of listening effort. Data from this study will be presented.
|
|
| 18. |
- Rönnberg, Niklas, 1973-
(författare)
-
Assessing cognitive spare capacity as a measure of listening effort using the Auditory Inference Span Test
- 2014
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Hearing loss has a negative effect on the daily life of 10-15% of the world’s population. One of the most common ways to treat a hearing loss is to fit hearing aids which increases audibility by providing amplification. Hearing aids thus improve speech reception in quiet, but listening in noise is nevertheless often difficult and stressful. Individual differences in cognitive capacity have been shown to be linked to differences in speech recognition performance in noise. An individual’s cognitive capacity is limited and is gradually consumed by increasing demands when listening in noise. Thus, fewer cognitive resources are left to interpret and process the information conveyed by the speech. Listening effort can therefore be explained by the amount of cognitive resources occupied with speech recognition. A well fitted hearing aid improves speech reception and leads to less listening effort, therefore an objective measure of listening effort would be a useful tool in the hearing aid fitting process.In this thesis the Auditory Inference Span Test (AIST) was developed to assess listening effort by measuring an individual’s cognitive spare capacity, the remaining cognitive resources available to interpret and encode linguistic content of incoming speech input while speech understanding takes place. The AIST is a dual-task hearing-innoise test, combining auditory and memory processing, and requires executive processing of speech at different memory load levels. The AIST was administered to young adults with normal hearing and older adults with hearing impairment. The aims were 1) to develop the AIST; 2) to investigate how different signal-to-noise ratios (SNRs) affect memory performance for perceived speech; 3) to explore if this performance would interact with cognitive capacity; 4) to test if different background noise types would interact differently with memory performance for young adults with normal hearing; and 5) to examine if these relationships would generalize to older adults with hearing impairment.The AIST is a new test of cognitive spare capacity which uses existing speech material that is available in several countries, and manipulates simultaneously cognitive load and SNR. Thus, the design of AIST pinpoints potential interactions between auditory and cognitive factors. The main finding of this thesis was the interaction between noise type and SNR showing that decreased SNR reduced cognitive spare capacity more in speech-like noise compared to speech-shaped noise, even though speech intelligibility levels were similar between noise types. This finding applied to young adults with normal hearing but there was a similar effect for older adults with hearing impairment with the addition of background noise compared to no background noise. Task demands, MLLs, interacted with cognitive capacity, thus, individuals with less cognitive capacity were more sensitive to increased cognitive load. However, MLLs did not interact with noise type or with SNR, which shows that different memory load levels were not affected differently in different noise types or in different SNRs. This suggests that different cognitive mechanisms come into play for storage and processing of speech information in AIST and for listening to speech in noise. Thus, the results suggested that a test of cognitive spare capacity seems to be a useful way to assess listening effort, even though the AIST, in the design used in this thesis, might be too cognitively demanding to provide reliable results for all individuals.
|
|
| 19. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Designing the user experience of musical sonification in public and semi-public spaces
- 2021
-
Ingår i: SoundEffects. - Århus C, Denmark : Århus Universitet. - 1904-500X. ; 10:1, s. 125-141
-
Tidskriftsartikel (refereegranskat)abstract
- Sonification refers to sonic expression of data or information. It is often thought of as an auditory complement, providing additional information about data which can reveal patterns and facilitate interpretation and understanding of the data. Hence, the listening space created by a sonifi cation is always a hybrid where auditory augmentation complements other information modalities and, in some cases, also spatial qualities. In this work, we focus on sonifi cation in public and semi-public spaces, and specifi cally on musical sonifi cation – the use of musical sounds to create a sonic environment, augmenting or complementing a physical shared space. We draw upon established approaches in interaction design to focus our work on the user experience of musical sonifi cation in public and semi-public spaces. Specifi cally, we fi rst identify the experiential qualities of sonic atmosphere and performativity as important aspects of sonifi cation in public and semi-public spaces, then use those experiential qualities generatively in the speculative design of a musical sonifi cation sketch. The design sketch comprises a dynamic musical sonifi cation of air quality data, intending to give citizens an awareness and an enhanced individual and interpersonal understanding of air quality in their city.
|
|
| 20. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Interactive Sonification for Visual Dense Data Displays
- 2016
-
Ingår i: Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016. - : CITEC, Bielefeld University. ; , s. 63-67
-
Konferensbidrag (refereegranskat)abstract
- This paper presents an experiment designed to evaluate the possible benefits of sonification in information visualization to give rise to further research challenges. It is hypothesized, that by using musical sounds for sonification when visualizing complex data, interpretation and comprehension of the visual representation could be increased by interactive sonification.This hypothesis is evaluated by testing sonification in parallel coordinates and scatter plots. The participants had to identify and mark different density areas in the representations, where amplitude of the sonification was mapped to the density in the data sets. Both quantitative and qualitative results suggest a benefit of sonification. These results indicate that sonification might be useful for data exploration, and give rise to new research questions and challenges.
|
|
| 21. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Measurement and sonification of construction site noise and particle pollution data
- 2023
-
Ingår i: Smart and Sustainable Built Environment. - Bingley, United Kingdom : Emerald Group Publishing Limited. - 2046-6099 .- 2046-6102. ; 12:4, s. 742-764
-
Tidskriftsartikel (refereegranskat)abstract
- Purpose – The noise and dust particles caused by the construction transport are by most stakeholders experienced as disturbing. The purpose of this study is to explore how sonification can support visualization in construction planning to decrease construction transport disturbances.Design/methodology/approach – This paper presents an interdisciplinary research project, combining research on construction logistics, internet of things and sonification. First, a data recording device, including sound, particle, temperature and humidity sensors, was implemented and deployed in a development project. Second, the collected data were used in a sonification design, which was, third, evaluated with potential users.Findings – The results showed that the low-cost sensors used could capture “good enough” data, and that the use of sonification for representing these data is interesting and a possible useful tool in urban and construction transport planning.Research limitations/implications – There is a need to further evolve the sonification design and better communicate the aim of the sounds used to potential users. Further testing is also needed.Practical implications – This study introduces new ideas of how to support visualization with sonification planning the construction work and its impact on the vicinity of the site. Currently, urban planning and construction planning focus on visualizing the final result, with little focus on how to handle disturbances during the construction process.Originality/value – Showing the potentials of using low-cost sensor data in sonification, and using sonification together with visualization, is the result of a novel interdisciplinary research area combination.
|
|
| 22. |
- Rönnberg, Niklas, 1973-
(författare)
-
Musical Elements in Sonification Support Visual Perception
- 2019
-
Ingår i: Proceedings of the 31st European Conference on Cognitive Ergonomics (ECCE 2019). - New York, NY, USA : Association for Computing Machinery (ACM). - 9781450371667 ; , s. 114-117
-
Konferensbidrag (refereegranskat)abstract
- Visual representations of data are commonly used to communicate research results. However, such representations might introduce several possible challenges for the human visual perception system, for example in perceiving brightness levels. Sonification, adding sound to the visual representation, might be used to overcome these challenges. As sonification provides additional information, sonification could be useful in supporting interpretations of a visual perception. In the present study, usefulness in terms of accuracy of sonification was investigated with an interactive sonification test. In the experiment, participants were asked to identify the highest brightness level in a monochrome visual representation. The task was performed in four conditions, one with no sonification and three with different sonification settings. The results show that sonification is useful, as measured by higher task accuracy.
|
|
| 23. |
- Rönnberg, Niklas, 1973-
(författare)
-
Musical sonification supports visual discrimination of color intensity
- 2019
-
Ingår i: Behaviour & Information Technology. - : Taylor & Francis. - 0144-929X. ; 38:10, s. 1028-1037
-
Tidskriftsartikel (refereegranskat)abstract
- Visual representations of data introduce several possible challenges for the human visual perception system in perceiving brightness levels. Overcoming these challenges might be simplified by adding sound to the representation. This is called sonification. As sonification provides additional information to the visual information, sonification could be useful in supporting the visual perception. In the present study, usefulness (in terms of accuracy and response time) of sonification was investigated with an interactive sonification test. In the test, participants were asked to identify the highest brightness level in a monochrome visual representation. The task was performed in four conditions, one with no sonification and three with different sonification settings. The results show that sonification is useful, as measured by higher task accuracy, and that the participant's musicality facilitates the use of sonification with better performance when sonification was used. The results were also supported by subjective measurements, where participants reported an experienced benefit of sonification.
|
|
| 24. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Questionnaires assessing usability of audio-visual representations
- 2022
-
Konferensbidrag (refereegranskat)abstract
- In this position paper we argue for the need of using standardized questionnaires for assessing usability in audio-visual representations. By using standardized measures of usability, comparability and validity of research studies in this field will be improved. However, it is not clear which questionnaire that is most suitable for assessing usability in audio-visual representations, neither when assessing the modalities individually or the combination. We present a variety of different questionnaires available, and argue for the need of combining different measures as well as developing new.
|
|
| 25. |
- Rönnberg, Niklas, 1973-
(författare)
-
Sonification for Conveying Data and Emotion
- 2021
-
Ingår i: AM '21: Audio Mostly 2021. - New York, NY, USA : ACM. - 9781450385695 ; , s. 56-63
-
Konferensbidrag (refereegranskat)abstract
- In the present study a sonification of running data was evaluated. The aim of the sonification was to both convey information about the data and convey a specific emotion. The sonification was evaluated in three parts, firstly as an auditory graph, secondly together with additional text information, and thirdly together with an animated visualization, with a total of 150 responses. The results suggest that the sonification could convey an emotion similar to that intended, but at the cost of less good representation of the data. The addition of visual information supported understanding of the sonification, and the auditory representation of data. The results thus suggest that it is possible to design sonification that is perceived as both interesting and fun, and convey an emotional impression, but that there may be a trade off between musical experience and clarity in sonification.
|
|
| 26. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Sonification Support for Information Visualization Dense Data Displays
- 2016
-
Ingår i: InfoVis Papers 2016.
-
Konferensbidrag (refereegranskat)abstract
- This poster presents an experiment designed to evaluate the possible benefits of sonification in information visualization. It is hypothesized, that by using musical sounds for sonification when visualizing complex data, interpretation and comprehension of the visual representation could be increased. In this evaluation of sonification in parallel coordinates and scatter plots, participants had to identify and mark different density areas in the representations. Both quantitative and qualitative results suggest a benefit of sonification. These results indicate that sonification might be useful for data exploration, and give rise to new research questions and challenges.
|
|
| 27. |
- Rönnberg, Niklas, 1973-
(författare)
-
Sonification supports perception of brightness contrast
- 2019
-
Ingår i: Journal on Multimodal User Interfaces. - : Springer. - 1783-7677 .- 1783-8738. ; 13:13, s. 373-381
-
Tidskriftsartikel (refereegranskat)abstract
- In complex visual representations, there are several possible challenges for the visual perception that might be eased by adding sound as a second modality (i.e. sonification). It was hypothesized that sonification would support visual perception when facing challenges such as simultaneous brightness contrast or the Mach band phenomena. This hypothesis was investigated with an interactive sonification test, yielding objective measures (accuracy and response time) as well as subjective measures of sonification benefit. In the test, the participant’s task was to mark the vertical pixel line having the highest intensity level. This was done in a condition without sonification and in three conditions where the intensity level was mapped to different musical elements. The results showed that there was a benefit of sonification, with higher accuracy when sonification was used compared to no sonification. This result was also supported by the subjective measurement. The results also showed longer response times when sonification was used. This suggests that the use and processing of the additional information took more time, leading to longer response times but also higher accuracy. There were no differences between the three sonification conditions.
|
|
| 28. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Sonifying the periphery: Supporting the formation of gestalt in air traffic control
- 2016
-
Ingår i: Proceedings of ISon 2016, 5th Interactive Sonification Workshop, CITEC, Bielefeld University, Germany, December 16, 2016. - : CITEC, Bielefeld University. ; , s. 23-27
-
Konferensbidrag (refereegranskat)abstract
- We report a design-led exploration of sonification to provide peripheral awareness in air traffic control centers. Our assumption is that by using musical sounds for sonification of peripheral events, it is possible to create a dynamic soundscape that complements the visual information to support the formation and maintenance of an airspace Gestalt throughout the air traffic controller’s interaction. An interactive sonification concept was designed, focusing on one controlled sector of airspace with inbound and outbound aircraft. A formative assessment of the sonification concept suggests that our approach might facilitate the air traffic controller’s work by providing complementary auditory information about inbound and outbound aircraft, particularly in situations where the traffic volume is moderate to low.
|
|
| 29. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Testing effort for speech comprehension using the individuals’ cognitive spare capacity - the Auditory Inference Span test
- 2010
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Modern hearing aids use a multitude of parameters to give the user an optimal speech signal. Fitting of the hearing aid becomes a handiwork due to the limited data of the patients hearing status (primarily an audiogram). A hearing in noise test (SNR threshold) is often used to evaluate the fitting. However, testing the SNR threshold as done in clinical use today is not ecological valid. Another way to think about hearing aid fitting is to ease the listening effort. Therefore, we propose the Auditory Inference Span Test (AIST) as a clinical tool during hearing aid fitting to assess the patient’s effort to understand speech. AIST is a combined auditory, memory, and processing test. It relies on the idea that the more cognitive resources that are required to process and understand speech, less cognitive resources are available for storage of the speech information. In AIST, sentences are presented in noise and afterwards the patient is required to recall and process the information from the sentences. Correctness and answering speed is measured and scores correlate to the effort required to understand the speech. Data from piloting tests indicate that the AIST is well suited as a clinical test for listening effort.
|
|
| 30. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Testing listening effort for speech comprehension
- 2011
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- One aim of hearing aid fitting is to reduce the effort of understanding speech, especially in noisy environments. For a good hearing aid fitting, knowledge about the patient’s auditory abilities is necessary, but knowledge about cognitive abilities may also be important. The Auditory Inference Span Test (AIST) is a dual-task hearing-in-noise test, that combines auditory and memory processing. In AIST, Hagerman sentences are presented in steady state speech-shaped noise at -2dB, -4dB or -6dB SNR and the subject is required to recall and process the information from the sentences by giving button-press responses to multiple-choice questions thereby assessing what the subject could infer from what was heard. AIST will be administered to 40 normal hearing subjects (29 to date) and performance related to speech reception threshold, working memory capacity and updating ability, as well as subjective rating of listening effort. Preliminary results show a greater SNR-related improvement in AIST scores at low SNRs than can be explained by improved audibility alone, consistent with release of memory resources due to reduced listening effort. There is also a trend towards a positive relationship between AIST scores and individual working memory capacity and updating ability.
|
|
| 31. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Testing listening effort for speech comprehension using the individuals’ cognitive spare capacity
- 2011
-
Ingår i: Audiology Research. - Pavia, Italy : PAGEPress. - 2039-4330 .- 2039-4349. ; 1:1, s. 82-85
-
Tidskriftsartikel (refereegranskat)abstract
- Most hearing aid fittings today are almost solely based on the patient’s audiogram. Although the loss of gain in the cochlea is important, for a more optimal fitting, more individual parameters of the patient’s cochlear loss together with the patient's cognitive abilities to process the auditory signal are required (Stenfelt & Rönnberg, 2009; Edwards, 2007). Moreover, the evaluation of the fitting is often based on a speech in noise task and the aim is to improve the individual patient’s signal to noise ratio (SNR) thresholds. As a consequence, hearing aid fitting may be seen as a process aimed to improve the patient’s SNR threshold rather than to improve communication ability. However, subsequent to a hearing aid fitting, there can be great differences in SNR improvement between patients that have identical hearing impairment in terms of threshold data (the audiogram). The reasons are certainly complex but one contributing factor may be the individual differences in cognitive capacity and associated listening effort. Another way to think about amplified hearing is to ease a subject’s listening effort (Sarampalis, et al., 2009). When the speech signal is degraded by noise or by a hearing impairment, more high-order cognitive or top-down processes are required to perceive and understand the signal, and listening is therefore more effortful. It is assumed that a hearing aid would ease the listening effort for a hearing impaired person. However, it is not clear how to measure the listening effort. We here present a test that will tap into the different cognitive aspects of listening effort, the Auditory Inference Span Test (AIST). The AIST is a dual task hearing in noise test, that combines auditory and memory processing and is well suited as a clinical test for listening effort.
|
|
| 32. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
The Auditory Inference Span Test – Developing a test for cognitive aspects of listening effort for speech comprehension
- 2010
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Most hearing aid fittings today are almost solely based on the patient’s audiogram. However, more individual parameters of the patient’s hearing thresholds together with the patient’s cognitive abilities to process the auditory signal are required. Hearing aid fitting may be seen as a process aimed to improve the patient’s hearing thresholds rather than to improve communication ability. Another way to think about hearing aid fitting is to ease the patient’s listening effort. However, it is not clear how to measure the listening effort. Therefore, we propose the Auditory Inference Span Test (AIST) as a clinical tool during hearing aid fitting to assess the patient’s effort to understand speech. AIST is a combined auditory, memory, and processing test. It relies on the idea that the more cognitive resources that are required to process and understand speech, less cognitive resources are available for storage of the speech information. In AIST, sentences are presented in noise and afterwards the patient is required to recall and process the information from the sentences. Correctness and reaction time is recorded as measurements of perceived listening effort. Data from piloting tests indicate that the AIST is well suited as a clinical test for listening effort. In a future study to verify that the AIST is sensitive to cognitive capacity, the test will be evaluated with measurements of the subject's cognitive capacity as well as the subject's hearing thresholds. For a clinical test the requirement is that it is fast and easily facilitated. The AIST takes no more than fifteen minutes to complete, and the aim is to further shorten the time and adapt the test for clinical use. This ensures the AIST to be a useable instrument for testing listening effort using the individuals' cognitive spare capacity.
|
|
| 33. |
- Rönnberg, Niklas, 1973-
(författare)
-
Towards Interactive Sonification in Monitoring of Dynamic Processes
- 2019
-
Ingår i: Proceedings of the Nordic Sound and Music Computing Conference 2019 (NSMC2019) and the Interactive Sonification Workshop 2019 (ISON2019). - Stockholm. ; , s. 92-99
-
Konferensbidrag (refereegranskat)abstract
- The modern control room is predominantly made up of visual displays, which might make monitoring different processes a cumbersome and visually cognitively demanding task. Sonification could be used to support the monitoring task. However, it is not clear how the most beneficial sonification should be designed. In this pilot study an experimental setup was developed to explore perception of different sonification approaches. A user experiment was performed to assess perception of the sonification when and where simulated data deviated most from the normal level. It was found that all sonification conditions were generally useful, regardless of the participant’s musical experience, shown both in terms of objective and subjective measurements. Stereo panning of the sound was also generally experienced as helpful, but the use of different pitch might not have been experienced to contribute as much for solving the task. The findings in this pilot study will be further used to create new research ideas about sonification for monitoring of dynamic processes.
|
|
| 34. |
- Rönnberg, Niklas, 1973-, et al.
(författare)
-
Traces Of Modal Synergy : Studying Interactive Musical Sonificationof Images in General-Audience Use
- 2019
-
Ingår i: The 25th International Conference on Auditory Display (ICAD 2019). - Newcastle upon Tyne, UK : Department of Computer and Information Sciences, Northumbria University. - 0967090466 - 9780967090467 ; , s. 199-206
-
Konferensbidrag (refereegranskat)abstract
- Photone is an interactive installation combining color images with musical sonification. The musical expression is generated based on the syntactic (as opposed to semantic) features of an image as it is explored by the userﾒs pointing device, intending to catalyze a holistic user experience we refer to as modal synergy where visual and auditory modalities multiply rather than add. We collected and analyzed two months' worth of data from visitorsﾒ interactions with Photone in a public exhibition at a science center. Our results show that a small proportion of visitors engaged in sustained interaction with Photone, as indicated by session times. Among the most deeply engaged visitors, a majority of the interaction was devoted to visually salient objects, i.e., semantic features of the images. However, the data also contains instances of interactive behavior that are best explained by exploration of the syntactic features of an image, and thus may suggest the emergence of modal synergy.
|
|
| 35. |
- Rönnberg, Niklas, 1973-
(författare)
-
Where Visualization Fails, Sonification Speaks
- 2024
-
Konferensbidrag (refereegranskat)abstract
- Traveling by public transport can be challenging for a visually impaired traveler. However, visual information can be supported by sonification, the use of non-speech sound to convey information about data. This research project aims to explore how sonification can be used to provide information to a traveler at a bus stop. Three situations are described together with different sonification design approaches that will later be further developed and evaluated.
|
|
| 36. |
|
|
| 37. |
|
|
| 38. |
- Svensson, Åsa, 1988-, et al.
(författare)
-
Automation, teamwork, and the feared loss of safety : Air traffic controllers’ experiences and expectations on current and future ATM systems
- 2021
-
Ingår i: 32nd European Conference on Cognitive Ergonomics: Designing Virtual and Physical Interactive Systems, ECCE 2021. - New York, NY, USA : ACM Digital Library. - 9781450387576 ; , s. -8
-
Konferensbidrag (refereegranskat)abstract
- In various control systems, automation is implemented to increase efficiency and safety. With increased automation, it becomes increasingly relevant to view the automation as a team member, rather than as a tool. In best cases, human-automation teamwork keeps workload within acceptable limits, increase situation awareness, and keeps the operator in the control loop. However, human-automation teamwork will only flourish if the automation is developed with the human operator in mind. Therefore, investigations of the current experiences and expectations regarding automation and teamwork are important for the development of automation. Through a questionnaire among Air Traffic Controllers (ATCOs), the present study aims to investigate how ATCOs perceive automation and safety in current and future air traffic control systems and the importance of different teamwork factors for human-human and human-automation collaboration. The results indicate that the ATCOs believe that safety will increase in the future along with increasing automation as long as the automation is working as expected. The ATCOs expressed a fear of deskilling and losing situation awareness with automation, a fear associated with a new ATCO role of monitoring the system and take over when the automation fails. The results suggest that design for human-automation teamwork aspects that ATCOs value, such as adaptability or mutual performance monitoring, could be a way forward.
|
|
| 39. |
- Svensson, Åsa, et al.
(författare)
-
Expected teamwork attributes between human operator and automation in air traffic control
- 2022
-
Konferensbidrag (refereegranskat)abstract
- To meet the increasing demands within the air traffic control (ATC) industry, automation is seen as one solution to increase efficiency, safety, and capacity. In ATC, teamwork between air traffic controllers (ATCO) is a key component to safely direct the aircraft. With increasing automation, the human-automation teamwork will become even more important to keep the human operator in the loop and contribute to the operator’s involvement in decision-making. However, operators can be cautious to new automation due to experience has taught them to be cautious and the willingness to accept automation can relate to the purpose of the automation. Therefore, this study presents how ATCOs want to work with an automated tool in the future, what qualities the automation should have, and what the human should be able to do when working with an automated system. Through an online questionnaire, 113 operational ATCOs answered questions about ATCOs’ experiences and expectations of current and future ATM systems, as well as teamwork between the ATCO and automated systems. For this paper, the teamwork aspects of the questionnaire are presented and discussed. The ATCOs believed that they should be able to trust the automation, know the reasoning behind the automation’s suggestions and decisions, and be able to always take control of the situation from the automation. The results will be used in early development processes of automation, in both the ATM domain but also in other control domains, that faces similar challenges.
|
|
| 40. |
- Svensson, Åsa, 1988-
(författare)
-
Human-automation teamwork : Current practices and future directions in air traffic control
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This dissertation explores the topic of human-automation teamwork in Air Traffic Control (ATC). ATC is a high stakes environment where complex automation is being introduced while the human operator has the legal responsibility. With increasing demands on productivity in various industries (as also in ATC), automation is introduced for efficiency, maintaining safety, and to keep the workload of the human operator within acceptable limits. However, previous research has shown that automation may cause negative effects on the human operator and performance, such as forcing the operator out of the control loop, which might lead to problems or confusion. Previous research suggests a need for strengthening human-automation collaboration where automation is seen as a team member to keep the operator in the loop. In order to achieve such teamwork, the design of the automation needs to be human-centred, i.e. that the automation is designed for the underlying need of the operator.The aim of this dissertation is to explore teamwork in ATC from several angles to understand how the air traffic controllers are working in current ATC environments and how automation could be designed to support human-automation teamwork. The included studies rely on interviews, simulations, and questionnaires, all with operational air traffic controllers as participants.The results indicate that for both human-human teamwork and human-automation teamwork, teamwork factors such as adaptability and mutual performance monitoring (knowing what the other team members are doing) are important for the work performance in current ATC environments, where mutual performance monitoring is especially important during stressful situations.When designing automation, lessons learned from human-human teamwork should be considered. The work within the scope of this dissertation identifies and concerns two human-automation teamwork aspects: boundary awareness and implicit communication. These are proposed to support the operator’s knowledge about the automation and the communication flow between the operator and the automation. Boundary awareness is the operator’s knowledge of the automation’s abilities, its boundaries (what it can or cannot manage), and about consequences if it would go outside of these boundaries. Implicit communication is the unspoken or implied small cues that the operator and the automation can use to communicate with each other. It is proposed that implicit communication can be based on the work patterns of the operator. The knowledge gained through the work in this dissertation can be used as a foundation for further research and design of automation regarding operator knowledge about the automation boundaries and the communication within the team.
|
|
| 41. |
- Svoronos-Kanavas, Iason, et al.
(författare)
-
An exploratory use of audiovisual displays on oceanographic data
- 2022
-
Konferensbidrag (refereegranskat)abstract
- The present study is an interdisciplinary endeavour that transmutes science, technology, and aesthetics into an audiovisual experience. The objective is to highlight the potential of combining sonification with visualisation in order to enhance the comprehension of extensive and complex sets of data. Moreover, this paper describes contemporary tools and methods for the implementation of the practice and suggests effective ways to monitor environmental changes. It can be regarded as an exploratory study for familiarisation with the potential of sonification and visualisation in the exploration of environmental data.
|
|
| 42. |
|
|
