| 1. |
|
|
| 2. |
|
|
| 3. |
|
|
| 4. |
|
|
| 5. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
A multimedia environment for interactive music performance
- 1997
-
Ingår i: TMH-QPSR. ; 38:2-3, s. 029-032
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- We propose a music performance tool based on the Java programming language. This software runs in any Java applet viewer (i.e. a WWW browser) and interacts with the local Midi equipment by mean of a multi-task software module for Midi applications (MidiShare). Two main ideas are at the base of our project: one is to realise an easy, intuitive, hardware and software independent tool for performance, and the other is to achieve an easier development of the tool itself. At the moment there are two projects under development: a system based only on a Java applet, called Japer (Java performer), and a hybrid system based on a Java user interface and a Lisp kernel for the development of the performance tools. In this paper, the first of the two projects is presented.
|
|
| 6. |
|
|
| 7. |
|
|
| 8. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Emotion rendering in music : Range and characteristic values of seven musical variables
- 2011
-
Ingår i: Cortex. - : Elsevier BV. - 0010-9452 .- 1973-8102. ; 47:9, s. 1068-1081
-
Tidskriftsartikel (refereegranskat)abstract
- Many studies on the synthesis of emotional expression in music performance have focused on the effect of individual performance variables on perceived emotional quality by making a systematical variation of variables. However, most of the studies have used a predetermined small number of levels for each variable, and the selection of these levels has often been done arbitrarily. The main aim of this research work is to improve upon existing methodologies by taking a synthesis approach. In a production experiment, 20 performers were asked to manipulate values of 7 musical variables simultaneously (tempo, sound level, articulation, phrasing, register, timbre, and attack speed) for communicating 5 different emotional expressions (neutral, happy, scary, peaceful, sad) for each of 4 scores. The scores were compositions communicating four different emotions (happiness, sadness, fear, calmness). Emotional expressions and music scores were presented in combination and in random order for each performer for a total of 5 x 4 stimuli. The experiment allowed for a systematic investigation of the interaction between emotion of each score and intended expressed emotions by performers. A two-way analysis of variance (ANOVA), repeated measures, with factors emotion and score was conducted on the participants' values separately for each of the seven musical factors. There are two main results. The first one is that musical variables were manipulated in the same direction as reported in previous research on emotional expressive music performance. The second one is the identification for each of the five emotions the mean values and ranges of the five musical variables tempo, sound level, articulation, register, and instrument. These values resulted to be independent from the particular score and its emotion. The results presented in this study therefore allow for both the design and control of emotionally expressive computerized musical stimuli that are more ecologically valid than stimuli without performance variations.
|
|
| 9. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Evaluation of computer systems for expressive music performance
- 2013
-
Ingår i: Guide to Computing for Expressive Music Performance. - London : Springer. - 9781447141235 - 9781447141228 ; , s. 181-203
-
Bokkapitel (refereegranskat)abstract
- In this chapter, we review and summarize different methods for the evaluation of CSEMPs. The main categories of evaluation methods are (1) comparisons with measurements from real performances, (2) listening experiments, and (3) production experiments. Listening experiments can be of different types. For example, in some experiments, subjects may be asked to rate a particular expressive characteristic (such as the emotion conveyed or the overall expression) or to rate the effect of a particular acoustic cue. In production experiments, subjects actively manipulate system parameters to achieve a target performance. Measures for estimating the difference between performances are discussed in relation to the objectives of the model and the objectives of the evaluation. There will be also a section with a presentation and discussion of the Rencon (Performance Rendering Contest). Rencon is a contest for comparing the expressive musical performances of the same score generated by different CSEMPs. Practical examples from previous works are presented, commented on, and analysed.
|
|
| 10. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Expressive musical icons
- 2001
-
Ingår i: Proceedings of the International Conference on Auditory Display - ICAD 2001. ; , s. 141-143
-
Konferensbidrag (refereegranskat)abstract
- Recent research on the analysis and synthesis of music performance has resulted in tools for the control of the expressive content in automatic music performance [1]. These results can be relevant for applications other than performance of music by a computer. In this work it is presented how the techniques for enhancing the expressive character in music performance can be used also in the design of sound logos, in the control of synthesis algorithms, and for achieving better ringing tones in mobile phones.
|
|
| 11. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Expressive sonification of footstep sounds
- 2010
-
Ingår i: Proceedings of ISon 2010. - Stockholm, Sweden : KTH Royal Institute of Technology. ; , s. 51-54
-
Konferensbidrag (refereegranskat)abstract
- In this study we present the evaluation of a model for the interactive sonification of footsteps. The sonification is achieved by means of specially designed sensored-shoes which control the expressive parameters of novel sound synthesis models capable of reproducing continuous auditory feedback for walking. In a previousstudy, sounds corresponding to different grounds were associated to different emotions and gender. In this study, we used an interactive sonification actuated by the sensored-shoes for providing auditory feedback to walkers. In an experiment we asked subjects to walk (using the sensored-shoes) with four different emotional intentions (happy, sad, aggressive, tender) and for each emotion we manipulated the ground texture sound four times (wood panels, linoleum, muddy ground, and iced snow). Preliminary results show that walkers used a more active walking style (faster pace) when the sound of the walking surface was characterized by an higher spectral centroid (e.g. iced snow), and a less active style (slower pace) when the spectral centroid was low (e.g. muddy ground). Harder texture sounds lead to more aggressive walking patters while softer ones to more tender and sad walking styles.
|
|
| 12. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Interactive sonification
- 2012
-
Ingår i: Journal on Multimodal User Interfaces. - : Springer Science and Business Media LLC. - 1783-7677 .- 1783-8738. ; 5:3-4, s. 85-86
-
Tidskriftsartikel (refereegranskat)abstract
- In October 2010, Roberto Bresin, Thomas Hermann and Andy Hunt launched a call for papers for a special issue on Interactive Sonification of the Journal on Multimodal User Interfaces (JMUI). The call was published in eight major mailing lists in the field of Sound and Music Computing and on related websites. Twenty manuscripts were submit- ted for review, and eleven of them have been accepted for publication after further improvements. Three of the papers are further developments of works presented at ISon 2010— Interactive Sonification workshop. Most of the papers went through a three-stage review process.The papers give an interesting overview of the field of Interactive Sonification as it is today. Their topics include the sonification of data exploration and of motion, a new sound synthesis model suitable for interactive sonification applications, a study on perception in the everyday periphery of attention, and the proposal of a conceptual framework for interactive sonification.
|
|
| 13. |
|
|
| 14. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Looking for the soundscape of the future : preliminary results applying the design fiction method
- 2020
-
Ingår i: Sound and Music Computing Conference 2020.
-
Konferensbidrag (refereegranskat)abstract
- The work presented in this paper is a preliminary study in a larger project that aims to design the sound of the future through our understanding of the soundscapes of the present, and through methods of documentary filmmaking, sound computing and HCI. This work is part of a project that will complement and run parallel to Erik Gandini’s research project ”The Future through the Present”, which explores how a documentary narrative can create a projection into the future, and develop a cinematic documentary aesthetics that releases documentary film from the constraints of dealing with the present or the past. The point of departure is our relationship to labour at a time when Robotics, VR/AR and AI applied to Big Data outweigh and augment our physical and cognitive capabilities, with automation expected to replace humans on a large scale within most professional fields. From an existential perspective this poses the question: what will we do when we don’t have to work? And challenges us to formulate a new idea of work beyond its historical role. If the concept of work ethics changes, how would that redefine soundscapes? Will new sounds develop? Will sounds from the past resurface? In the context of this paper we try to tackle these questions by first applying the Design Fiction method. In a workshop with twenty-three participants predicted both positive and negative future scenarios, including both lo-fi and hi-fi soundscapes, and in which people will be able to control and personalize soundscapes. Results are presented, summarized and discussed.
|
|
| 15. |
|
|
| 16. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Robust Non-Verbal Expression in Humanoid Robots: New Methods for Augmenting Expressive Movements with Sound
- 2021
-
Konferensbidrag (refereegranskat)abstract
- The aim of the SONAO project is to establish new methods basedon sonification of expressive movements for achieving a robust interaction between users and humanoid robots. We want to achievethis by combining competences of the research team members inthe fields of social robotics, sound and music computing, affective computing, and body motion analysis. We want to engineersound models for implementing effective mappings between stylized body movements and sound parameters that will enable anagent to express high-level body motion qualities through sound.These mappings are paramount for supporting feedback to andunderstanding robot body motion. The project will result in thedevelopment of new theories, guidelines, models, and tools forthe sonic representation of high-level body motion qualities in interactive applications. This work is part of the growing researchfield known as data sonification, in which we combine methodsand knowledge from the fields of interactive sonification, embodied cognition, multisensory perception, non-verbal and gesturalcommunication in robots.
|
|
| 17. |
|
|
| 18. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Software Tools for Musical Expression
- 2000
-
Ingår i: Proceedings of the 2000 International Computer Music Conference, ICMC 2000. - : International Computer Music Association.
-
Konferensbidrag (refereegranskat)abstract
- In this article software tools that model principles used in music performance are presented. They all add expressive variations to a score-based representation of the music. The two main tools are Director Musices, a Lisp program, and Japer, a Java applet. Some possible applications are illustrated: music production, teaching of music performance, and performance of mobile phone ringing tones.
|
|
| 19. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Sonification of the self vs. sonification of the other : Differences in the sonification of performed vs. observed simple hand movements
- 2020
-
Ingår i: International journal of human-computer studies. - : Elsevier BV. - 1071-5819 .- 1095-9300. ; 144
-
Tidskriftsartikel (refereegranskat)abstract
- Existing works on interactive sonification of movements, i.e., the translation of human movement qualities from the physical to the auditory domain, usually adopt a predetermined approach: the way in which movement features modulate the characteristics of sound is fixed. In our work we want to go one step further and demonstrate that the user role can influence the tuning of the mapping between movement cues and sound parameters. Here, we aim to verify if and how the mapping changes when the user is either the performer or the observer of a series of body movements (tracing a square or an infinite shape with the hand in the air). We asked participants to tune movement sonification while they were directly performing the sonified movement vs. while watching another person performing the movement and listening to its sonification. Results show that the tuning of the sonification chosen by participants is influenced by three variables: role of the user (performer vs observer), movement quality (the amount of Smoothness and Directness in the movement), and physical parameters of the movements (velocity and acceleration). Performers focused more on the quality of their movement, while observers focused more on the sonic rendering, making it more expressive and more connected to low-level physical features.
|
|
| 20. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
SOUND FOREST/LJUDSKOGEN: A LARGE-SCALE STRING-BASED INTERACTIVE MUSICAL INSTRUMENT
- 2016
-
Ingår i: Sound and Music Computing 2016. - : SMC Sound&Music Computing NETWORK. - 9783000537004 ; , s. 79-84
-
Konferensbidrag (refereegranskat)abstract
- In this paper we present a string-based, interactive, largescale installation for a new museum dedicated to performing arts, Scenkonstmuseet, which will be inaugurated in 2017 in Stockholm, Sweden. The installation will occupy an entire room that measures 10x5 meters. We aim to create a digital musical instrument (DMI) that facilitates intuitive musical interaction, thereby enabling visitors to quickly start creating music either alone or together. The interface should be able to serve as a pedagogical tool; visitors should be able to learn about concepts related to music and music making by interacting with the DMI. Since the lifespan of the installation will be approximately five years, one main concern is to create an experience that will encourage visitors to return to the museum for continued instrument exploration. In other words, the DMI should be designed to facilitate long-term engagement. Finally, an important aspect in the design of the installation is that the DMI should be accessible and provide a rich experience for all museum visitors, regardless of age or abilities.
|
|
| 21. |
- Burger, Birgitta, et al.
(författare)
-
Communication of Musical Expression by Means of Mobile Robot Gestures
- 2010
-
Ingår i: Journal on Multimodal User Interfaces. - Stockholm : Springer Science and Business Media LLC. - 1783-7677 .- 1783-8738. ; 3:1, s. 109-118
-
Tidskriftsartikel (refereegranskat)abstract
- We developed a robotic system that can behave in an emotional way. A 3-wheeled simple robot with limited degrees of freedom was designed. Our goal was to make the robot displaying emotions in music performance by performing expressive movements. These movements have been compiled and programmed based on literature about emotion in music, musicians’ movements in expressive performances, and object shapes that convey different emotional intentions. The emotions happiness, anger, and sadness have been implemented in this way. General results from behavioral experiments show that emotional intentions can be synthesized, displayed and communicated by an artificial creature, also in constrained circumstances.
|
|
| 22. |
|
|
| 23. |
- Dahl, S., et al.
(författare)
-
Gestures in Performance
- 2010
-
Ingår i: Musical Gestures. - : Taylor and Francis. ; , s. 36-68
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- On occasion, one can observe a whole orchestra section moving and playing in unison. In such an instance, all violinists play the melody using the same type of bowing movements and lean forward in unison at a given time during a specific passage in the music. Thus, not only do the musicians use very similar movements to produce the same notes but they sometimes also coordinate bodily sways or other movements with the other players. The musical gesture seems to manifest itself in both sound and movement. Whether we are watching as audience, or participating and interacting in the actual performance itself, we receive a considerable amount of gestural information. The aim of this chapter is to give examples of gestures that may be observed during performance, and to consider the kind of information they might convey, either to other performers or to the audience.
|
|
| 24. |
- Dubus, Gaël, et al.
(författare)
-
Exploration and evaluation of a system for interactive sonification of elite rowing
- 2015
-
Ingår i: Sports Engineering. - : Springer London. - 1369-7072 .- 1460-2687. ; 18:1, s. 29-41
-
Tidskriftsartikel (refereegranskat)abstract
- In recent years, many solutions based on interactive sonification have been introduced for enhancing sport training. Few of them have been assessed in terms of efficiency or design. In a previous study, we performed a quantitative evaluation of four models for the sonification of elite rowing in a non-interactive context. For the present article, we conducted on-water experiments to investigate the effects of some of these models on two kinematic quantities: stroke rate value and fluctuations in boat velocity. To this end, elite rowers interacted with discrete and continuous auditory displays in two experiments. A method for computing an average rowing cycle is introduced, together with a measure of velocity fluctuations. Participants answered to questionnaires and interviews to assess the degree of acceptance of the different models and to reveal common trends and individual preferences. No significant effect of sonification could be determined in either of the two experiments. The measure of velocity fluctuations was found to depend linearly on stroke rate. Participants provided feedback about their aesthetic preferences and functional needs during interviews, allowing us to improve the models for future experiments to be conducted over longer periods.
|
|
| 25. |
- Elblaus, Ludvig, 1981-
(författare)
-
Crafting Experience : Designing Digital Musical Instruments for Long-Term Use in Artistic Practice
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis collects a series of publications that all present work where digital musical instruments (DMIs) played a central role. While the papers focus on the individual projects where the DMIs where created, the first part of this thesis describes patterns and insights arrived at by comparing the projects and the DMIs, as well as discuss them in the context of other contemporary research.The projects described in the included papers all are quite different, but the role I performed in them was consistent in many ways. I position my- self as a craftsperson, and trace the practice of crafting digital musical instru- ments through the projects. As a working metaphor, I present the idea of the DMI-craftsperson as a translator between different domains. This requires a broader outlook than the mechanics of the instruments themselves, including some working understanding of the domains that the DMI interacts with.The relationship between DMIs and contemporary musical practice is a thread that runs through the work. I criticise the practice of exclusively performing laboratory based evaluations, and the concept of rigid and requirements based evaluations of artistic artefacts. Instead, I argue, relying on Sonic Interaction Design and embodied aesthetics, that the complexities and nuances of perfor- mance can only be fully explored by engaging in long-term artistic practice.
|
|
| 26. |
- Elblaus, Ludvig, 1981-, et al.
(författare)
-
Demo Hour
- 2015
-
Ingår i: interactions. - : Association for Computing Machinery (ACM). - 1072-5520 .- 1558-3449. ; 22:5, s. 6-9
-
Tidskriftsartikel (refereegranskat)abstract
- Interactivity is a unique forum of the ACM CHI Conference that showcases hands-on demonstrations, novel interactive technologies, and artistic installations. At CHI 2015 in Seoul we hosted more than 30 exhibits, including an invited digital interactive art exhibit. Interactivity highlights the diverse group of computer scientists, sociologists, designers, psychologists, artists, and many more who make up the CHI community.
|
|
| 27. |
- Elblaus, Ludvig, 1981-, et al.
(författare)
-
Modes of sonic interaction in circus : Three proofs of concept
- 2014
-
Ingår i: Proceedings of Sound and Music Computing Conference 2014. - Athens : The International Computer Music Association, The Sound and Music Computing Steering Committe, The Institute for Research on Music & Acoustics. ; , s. 1698-1706, s. 1698-1706-
-
Konferensbidrag (refereegranskat)abstract
- The art of circus is a vibrant and competitive culture that embraces new tools and technology. In this paper, a series of exploratory design processes resulting in proofs of concepts are presented, showing strategies for effective use of three different modes of sonic interaction in contemporary circus. Each design process is based on participatory studio work, involving professional circus artists. All of the proofs of concepts have been evaluated, both with studio studies and public circus performances, taking the work beyond theoretical laboratory projects and properly engaging the practice and culture of contemporary circus.The first exploration uses a contortionist’s extreme bodily manipulation as inspiration for sonic manipulations in an accompanying piece of music. The second exploration uses electric amplification of acoustic sounds as a transformative enhancement of existing elements of circus performance. Finally, a sensor based system of real-time sonification of body gestures is explored and ideas from the sonification of dance are translated into the realm of circus.
|
|
| 28. |
- Elblaus, Ludvig, 1981-, et al.
(författare)
-
Nebula: An Interactive Garment Designed for Functional Aesthetics
- 2015
-
Ingår i: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. - New York, NY, USA : ACM. - 9781450331463 ; , s. 275-278
-
Konferensbidrag (refereegranskat)abstract
- In this paper we present Nebula, a prototype for examining the properties of textiles, fashion accessories, and digital technologies to arrive at a garment design that brings these elements together in a cohesive manner. Bridging the gap between everyday performativity and enactment, we aim at discussing aspects of the making process, interaction and functional aesthetics that emerged. Nebula is part of the Sound Clothes project that aims at exploring the expressive potential of wearable technologies creating sound from motion.
|
|
| 29. |
- Elblaus, Ludvig, 1981-, et al.
(författare)
-
NIME Design and Contemporary Music Practice : Benefits and Challenges
- 2014
-
Konferensbidrag (refereegranskat)abstract
- This paper deals with the question of how the developmentof new musical artifacts can benet from deeply engagingwith contemporary musical practice. With the novel ideasproduced by the NIME community manifested in musicalinstruments in continuous use, new research questions canbe answered and new sources of knowledge can be explored.This can also be very helpful in evaluation, as it is possi-ble to evaluate the qualities of an instrument in a speciedcontext, rather than evaluating a prototyped instrument onthe basis of its unrealised potential. The information fromsuch evaluation can then be fed back into the developmentprocess, allowing researchers to probe musical practice itselfwith their designs.
|
|
| 30. |
|
|
| 31. |
- Falkenberg, Kjetil, Docent, 1972-, et al.
(författare)
-
Musikkommunikation och ljudinteraktion
- 2021
-
Ingår i: Introduktion till medieteknik. - Lund : Studentlitteratur AB. ; , s. 155-166
-
Bokkapitel (refereegranskat)
|
|
| 32. |
- Favero, Federico, et al.
(författare)
-
Study of the Effects of Daylighting and Artificial Lighting at 59° Latitude on Mental States, Behaviour and Perception
- 2023
-
Ingår i: Sustainability. - : MDPI AG. - 2071-1050. ; 15:2
-
Tidskriftsartikel (refereegranskat)abstract
- Although there is a documented preference for daylighting over artificial electric lighting indoors, there are comparatively few investigations of behaviour and perception in indoor day-lit spaces at high latitudes during winter. We report a pilot study designed to examine the effects of static artificial lighting conditions (ALC) and dynamic daylighting conditions (DLC) on the behaviour and perception of two groups of participants. Each group (n = 9 for ALC and n = 8 for DLC) experienced one of the two conditions for three consecutive days, from sunrise to sunset. The main results of this study show the following: indoor light exposure in February in Stockholm can be maintained over 1000 lx only with daylight for most of the working day, a value similar to outdoor workers’ exposure in Scandinavia; these values can be over the recommended Melanopic Equivalent Daylight Illuminance threshold; and this exposure reduces sleepiness and increases amount of activity compared to a static artificial lighting condition. Mood and feeling of time passing are also affected, but we do not exactly know by which variable, either personal or group dynamics, view or variation of the lighting exposure. The small sample size does not support inferential statistics; however, these significant effects might be large enough to be of importance in practice. From a sustainability point of view, daylighting can benefit energy saving strategies and well-being, even in the Scandinavian winter.
|
|
| 33. |
- Friberg, Anders, et al.
(författare)
-
Analysis by synthesis
- 2014
-
Ingår i: Music in the Social and Behavioral Sciences. - Los Angeles : Sage Publications. - 9781452283036
-
Bokkapitel (refereegranskat)
|
|
| 34. |
- Friberg, Anders, et al.
(författare)
-
Expressive timing
- 2014
-
Ingår i: Music in the Social and Behavioral Sciences. - Los Angeles : Sage Publications. - 9781452283036 ; , s. 440-442
-
Bokkapitel (refereegranskat)
|
|
| 35. |
- Frid, Emma, et al.
(författare)
-
An Exploratory Study On The Effect Of Auditory Feedback On Gaze Behavior In a Virtual Throwing Task With and Without Haptic Feedback
- 2017
-
Ingår i: Proceedings of the 14th Sound and Music Computing Conference. - Espoo, Finland : Aalto University. - 9789526037295 ; , s. 242-249
-
Konferensbidrag (refereegranskat)abstract
- This paper presents findings from an exploratory study on the effect of auditory feedback on gaze behavior. A total of 20 participants took part in an experiment where the task was to throw a virtual ball into a goal in different conditions: visual only, audiovisual, visuohaptic and audio- visuohaptic. Two different sound models were compared in the audio conditions. Analysis of eye tracking metrics indicated large inter-subject variability; difference between subjects was greater than difference between feedback conditions. No significant effect of condition could be observed, but clusters of similar behaviors were identified. Some of the participants’ gaze behaviors appeared to have been affected by the presence of auditory feedback, but the effect of sound model was not consistent across subjects. We discuss individual behaviors and illustrate gaze behavior through sonification of gaze trajectories. Findings from this study raise intriguing questions that motivate future large-scale studies on the effect of auditory feedback on gaze behavior.
|
|
| 36. |
- Frid, Emma, 1988-
(författare)
-
Diverse Sounds : Enabling Inclusive Sonic Interaction
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This compilation thesis collects a series of publications on designing sonic interactions for diversity and inclusion. The presented papers focus on case studies in which musical interfaces were either developed or reviewed. While the described studies are substantially different in their nature, they all contribute to the thesis by providing reflections on how musical interfaces could be designed to enable inclusion rather than exclusion. Building on this work, I introduce two terms: inclusive sonic interaction design and Accessible Digital Musical Instruments (ADMIs). I also define nine properties to consider in the design and evaluation of ADMIs: expressiveness, playability, longevity, customizability, pleasure, sonic quality, robustness, multimodality and causality. Inspired by the experience of playing an acoustic instrument, I propose to enable musical inclusion for under-represented groups (for example persons with visual- and hearing-impairments, as well as elderly people) through the design of Digital Musical Instruments (DMIs) in the form of rich multisensory experiences allowing for multiple modes of interaction. At the same time, it is important to enable customization to fit user needs, both in terms of gestural control and provided sonic output. I conclude that the computer music community has the potential to actively engage more people in music-making activities. In addition, I stress the importance of identifying challenges that people face in these contexts, thereby enabling initiatives towards changing practices.
|
|
| 37. |
- Frid, Emma, et al.
(författare)
-
Haptic feedback combined with movement sonification using a friction sound improves task performance in a virtual throwing task
- 2018
-
Ingår i: Journal on Multimodal User Interfaces. - : Springer. - 1783-7677 .- 1783-8738. ; 13:4, s. 279-290
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we present a study on the effects of auditory- and haptic feedback in a virtual throwing task performed with a point-based haptic device. The main research objective was to investigate if and how task performance and perceived intuitiveness is affected when interactive sonification and/or haptic feedback is used to provide real-time feedback about a movement performed in a 3D virtual environment. Emphasis was put on task solving efficiency and subjective accounts of participants’ experiences of the multimodal interaction in different conditions. The experiment used a within-subjects design in which the participants solved the same task in different conditions: visual-only, visuohaptic, audiovisual and audiovisuohaptic. Two different sound models were implemented and compared. Significantly lower error rates were obtained in the audiovisuohaptic condition involving movement sonification based on a physical model of friction, compared to the visual-only condition. Moreover, a significant increase in perceived intuitiveness was observed for most conditions involving haptic and/or auditory feedback, compared to the visual-only condition. The main finding of this study is that multimodal feedback can not only improve perceived intuitiveness of an interface but that certain combinations of haptic feedback and movement sonification can also contribute with performance-enhancing properties. This highlights the importance of carefully designing feedback combinations for interactive applications.
|
|
| 38. |
- Frid, Emma, et al.
(författare)
-
Interactive sonification of a fluid dance movement : an exploratory study
- 2019
-
Ingår i: Journal on Multimodal User Interfaces. - : Springer. - 1783-7677 .- 1783-8738. ; 13:3, s. 181-189
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we present three different experiments designed to explore sound properties associated with fluid movement: (1) an experiment in which participants adjusted parameters of a sonification model developed for a fluid dance movement, (2) a vocal sketching experiment in which participants sketched sounds portraying fluid versus nonfluid movements, and (3) a workshop in which participants discussed and selected fluid versus nonfluid sounds. Consistent findings from the three experiments indicated that sounds expressing fluidity generally occupy a lower register and has less high frequency content, as well as a lower bandwidth, than sounds expressing nonfluidity. The ideal sound to express fluidity is continuous, calm, slow, pitched, reminiscent of wind, water or an acoustic musical instrument. The ideal sound to express nonfluidity is harsh, non-continuous, abrupt, dissonant, conceptually associated with metal or wood, unhuman and robotic. Findings presented in this paper can be used as design guidelines for future applications in which the movement property fluidity is to be conveyed through sonification.
|
|
| 39. |
- Frid, Emma, et al.
(författare)
-
Interactive Sonification of Spontaneous Movement of Children : Cross-Modal Mapping and the Perception of Body Movement Qualities through Sound
- 2016
-
Ingår i: Frontiers in Neuroscience. - : Frontiers Media S.A.. - 1662-4548 .- 1662-453X. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we present three studies focusing on the effect of different sound models in interactive sonification of bodily movement. We hypothesized that a sound model characterized by continuous smooth sounds would be associated with other movement characteristics than a model characterized by abrupt variation in amplitude and that these associations could be reflected in spontaneous movement characteristics. Three subsequent studies were conducted to investigate the relationship between properties of bodily movement and sound: (1) a motion capture experiment involving interactive sonification of a group of children spontaneously moving in a room, (2) an experiment involving perceptual ratings of sonified movement data and (3) an experiment involving matching between sonified movements and their visualizations in the form of abstract drawings. In (1) we used a system constituting of 17 IR cameras tracking passive reflective markers. The head positions in the horizontal plane of 3-4 children were simultaneously tracked and sonified, producing 3-4 sound sources spatially displayed through an 8-channel loudspeaker system. We analyzed children’s spontaneous movement in terms of energy-, smoothness- and directness index. Despite large inter-participant variability and group-specific effects caused by interaction among children when engaging in the spontaneous movement task, we found a small but significant effect of sound model. Results from (2) indicate that different sound models can be rated differently on a set of motion-related perceptual scales (e.g. expressivity and fluidity). Also, results imply that audio-only stimuli can evoke stronger perceived properties of movement (e.g. energetic, impulsive) than stimuli involving both audio and video representations. Findings in (3) suggest that sounds portraying bodily movement can be represented using abstract drawings in a meaningful way. We argue that the results from these studies support the existence of a cross-modal mapping of body motion qualities from bodily movement to sounds. Sound can be translated and understood from bodily motion, conveyed through sound visualizations in the shape of drawings and translated back from sound visualizations to audio. The work underlines the potential of using interactive sonification to communicate high-level features of human movement data.
|
|
| 40. |
- Frid, Emma, et al.
(författare)
-
Perception of Mechanical Sounds Inherent to Expressive Gestures of a NAO Robot - Implications for Movement Sonification of Humanoids
- 2018
-
Ingår i: Proceedings of the 15th Sound and Music Computing Conference. - Limassol, Cyprus. - 9789963697304
-
Konferensbidrag (refereegranskat)abstract
- In this paper we present a pilot study carried out within the project SONAO. The SONAO project aims to compen- sate for limitations in robot communicative channels with an increased clarity of Non-Verbal Communication (NVC) through expressive gestures and non-verbal sounds. More specifically, the purpose of the project is to use move- ment sonification of expressive robot gestures to improve Human-Robot Interaction (HRI). The pilot study described in this paper focuses on mechanical robot sounds, i.e. sounds that have not been specifically designed for HRI but are inherent to robot movement. Results indicated a low correspondence between perceptual ratings of mechanical robot sounds and emotions communicated through ges- tures. In general, the mechanical sounds themselves ap- peared not to carry much emotional information compared to video stimuli of expressive gestures. However, some mechanical sounds did communicate certain emotions, e.g. frustration. In general, the sounds appeared to commu- nicate arousal more effectively than valence. We discuss potential issues and possibilities for the sonification of ex- pressive robot gestures and the role of mechanical sounds in such a context. Emphasis is put on the need to mask or alter sounds inherent to robot movement, using for exam- ple blended sonification.
|
|
| 41. |
- Frid, Emma, 1988-, et al.
(författare)
-
Perceptual Evaluation of Blended Sonification of Mechanical Robot Sounds Produced by Emotionally Expressive Gestures : Augmenting Consequential Sounds to Improve Non-verbal Robot Communication
- 2021
-
Ingår i: International Journal of Social Robotics. - : Springer Nature. - 1875-4791 .- 1875-4805.
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents two experiments focusing on perception of mechanical sounds produced by expressive robot movement and blended sonifications thereof. In the first experiment, 31 participants evaluated emotions conveyed by robot sounds through free-form text descriptions. The sounds were inherently produced by the movements of a NAO robot and were not specifically designed for communicative purposes. Results suggested no strong coupling between the emotional expression of gestures and how sounds inherent to these movements were perceived by listeners; joyful gestures did not necessarily result in joyful sounds. A word that reoccurred in text descriptions of all sounds, regardless of the nature of the expressive gesture, was “stress”. In the second experiment, blended sonification was used to enhance and further clarify the emotional expression of the robot sounds evaluated in the first experiment. Analysis of quantitative ratings of 30 participants revealed that the blended sonification successfully contributed to enhancement of the emotional message for sound models designed to convey frustration and joy. Our findings suggest that blended sonification guided by perceptual research on emotion in speech and music can successfully improve communication of emotions through robot sounds in auditory-only conditions.
|
|
| 42. |
- Frid, Emma, et al.
(författare)
-
Sonification of fluidity -
An exploration of perceptual connotations of a particular movement feature
- 2016
-
Ingår i: Proceedings of ISon 2016, 5th Interactive Sonification Workshop. - Bielefeld, Germany. ; , s. 11-17
-
Konferensbidrag (refereegranskat)abstract
- In this study we conducted two experiments in order to investigate potential strategies for sonification of the expressive movement quality “fluidity” in dance: one perceptual rating experiment (1) in which five different sound models were evaluated on their ability to express fluidity, and one interactive experiment (2) in which participants adjusted parameters for the most fluid sound model in (1) and performed vocal sketching to two video recordings of contemporary dance. Sounds generated in the fluid condition occupied a low register and had darker, more muffled, timbres compared to the non-fluid condition, in which sounds were characterized by a higher spectral centroid and contained more noise. These results were further supported by qualitative data from interviews. The participants conceptualized fluidity as a property related to water, pitched sounds, wind, and continuous flow; non-fluidity had connotations of friction, struggle and effort. The biggest conceptual distinction between fluidity and non-fluidity was the dichotomy of “nature” and “technology”, “natural” and “unnatural”, or even “human” and “unhuman”. We suggest that these distinct connotations should be taken into account in future research focusing on the fluidity quality and its corresponding sonification.
|
|
| 43. |
- Frid, Emma, et al.
(författare)
-
Sonification of haptic interaction in a virtual scene
- 2014
-
Ingår i: Sound and Music Computing Sweden 2014, Stockholm, December 4-5, 2014. - Stockholm : KTH Royal Institute of Technology. ; , s. 14-16, s. 14-16
-
Konferensbidrag (refereegranskat)abstract
- This paper presents a brief overview of work-in-progress for a study on correlations between visual and haptic spatial attention in a multimodal single-user application comparing different modalities. The aim is to gain insight into how auditory and haptic versus visual representations of temporal events may affect task performance and spatial attention. For this purpose, a 3D application involving one haptic model and two different sound models for interactive sonification are developed.
|
|
| 44. |
- Frid, Emma, 1988-, et al.
(författare)
-
Sound Forest - Evaluation of an Accessible Multisensory Music Installation
- 2019
-
Ingår i: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. - New York, NY, USA : ACM. - 9781450359702 ; , s. 1-12
-
Konferensbidrag (refereegranskat)abstract
- Sound Forest is a music installation consisting of a room with light-emitting interactive strings, vibrating platforms and speakers, situated at the Swedish Museum of Performing Arts. In this paper we present an exploratory study focusing on evaluation of Sound Forest based on picture cards and interviews. Since Sound Forest should be accessible for everyone, regardless age or abilities, we invited children, teens and adults with physical and intellectual disabilities to take part in the evaluation. The main contribution of this work lies in its fndings suggesting that multisensory platforms such as Sound Forest, providing whole-body vibrations, can be used to provide visitors of diferent ages and abilities with similar associations to musical experiences. Interviews also revealed positive responses to haptic feedback in this context. Participants of diferent ages used diferent strategies and bodily modes of interaction in Sound Forest, with activities ranging from running to synchronized music-making and collaborative play.
|
|
| 45. |
- Giordano, B.L., et al.
(författare)
-
The production and perception of emotionally expressive walking sounds : Similarities between musical performance and everyday motor activity
- 2014
-
Ingår i: PLOS ONE. - : Public Library of Science. - 1932-6203. ; 9:12, s. e115587-
-
Tidskriftsartikel (refereegranskat)abstract
- Several studies have investigated the encoding and perception of emotional expressivity in music performance. A relevant question concerns how the ability to communicate emotions in music performance is acquired. In accordance with recent theories on the embodiment of emotion, we suggest here that both the expression and recognition of emotion in music might at least in part rely on knowledge about the sounds of expressive body movements. We test this hypothesis by drawing parallels between musical expression of emotions and expression of emotions in sounds associated with a non-musical motor activity: walking. In a combined production-perception design, two experiments were conducted, and expressive acoustical features were compared across modalities. An initial performance experiment tested for similar feature use in walking sounds and music performance, and revealed that strong similarities exist. Features related to sound intensity, tempo and tempo regularity were identified as been used similarly in both domains. Participants in a subsequent perception experiment were able to recognize both non-emotional and emotional properties of the sound-generating walkers. An analysis of the acoustical correlates of behavioral data revealed that variations in sound intensity, tempo, and tempo regularity were likely used to recognize expressed emotions. Taken together, these results lend support the motor origin hypothesis for the musical expression of emotions.
|
|
| 46. |
- Goebl, Werner, et al.
(författare)
-
Perception of touch quality in piano tones
- 2014
-
Ingår i: Journal of the Acoustical Society of America. - : Acoustical Society of America (ASA). - 0001-4966 .- 1520-8524. ; 136:5, s. 2839-2850
-
Tidskriftsartikel (refereegranskat)abstract
- Both timbre and dynamics of isolated piano tones are determined exclusively by the speed with which the hammer hits the strings. This physical view has been challenged by pianists who emphasize the importance of the way the keyboard is touched. This article presents empirical evidence from two perception experiments showing that touch-dependent sound components make sounds with identical hammer velocities but produced with different touch forms clearly distinguishable. The first experiment focused on finger-key sounds: musicians could identify pressed and struck touches. When the finger-key sounds were removed from the sounds, the effect vanished, suggest- ing that these sounds were the primary identification cue. The second experiment looked at key- keyframe sounds that occur when the key reaches key-bottom. Key-bottom impact was identified from key motion measured by a computer-controlled piano. Musicians were able to discriminate between piano tones that contain a key-bottom sound from those that do not. However, this effect might be attributable to sounds associated with the mechanical components of the piano action. In addition to the demonstrated acoustical effects of different touch forms, visual and tactile modalities may play important roles during piano performance that influence the production and perception of musical expression on the piano.
|
|
| 47. |
- Goina, Maurizio, et al.
(författare)
-
Interactive sonification in circus performance at Uniarts and KTH : ongoing research
- 2014
-
Ingår i: Proceedings of the Sound and Music Computing Sweden Conference 2014. - : KTH Royal Institute of Technology. ; , s. 23-24
-
Konferensbidrag (refereegranskat)abstract
- Contemporary circus artists are beginning to use new tools and technologies. Aligned with this trend and with interest to applying interactive sonification to circus performance a collaboration is currently in progress between the Sound and Music Computing Team at KTH Royal Institute of Technology and the School of Dance and Circus, part of Stockholm University of the Arts. The collaboration includes the Gynoïdes Project1 and also works on a series of proofs of concept. This collaboration allowed us to gain experience and mature practical knowledge (see [1]) and a range of further activities are planned in the near future of which a description is given herewith.
|
|
| 48. |
- Goina, Maurizio, et al.
(författare)
-
Our Sound Space (OSS) : An Installation for Participatory AND Interactive Exploration of Soundscapes
- 2023
-
Ingår i: Proceedings of the Sound and Music Computing Conferences. - : Sound and Music Computing Network. - 9789152773727 ; , s. 255-260, s. 255-260
-
Konferensbidrag (refereegranskat)abstract
- This paper describes the development of an interactive tool which allows playing different soundscapes by mixing diverse environmental sounds on demand. This tool is titled Our Sound Space (OSS) and has been developed as part of an ongoing project where we test methods and tools for the participation of young people in spatial planning. As such OSS is meant to offer new opportunities to engage youth in talks about planning, placemaking and more sustainable living environments. In this paper, we describe an implementation of OSS that we are using as an interactive soundscape installation sited in a public place daily visited by people from a diversity of entities (e.g. university, a gymnasium, a restaurant, start-ups). The OSS installation is designed to allow simultaneous activation of several prerecorded sounds broadcast through four loudspeakers. The installation is interactive, meaning that it can be activated and operated by anyone via smartphones and is designed to allow interaction among multiple people at the same time and space.
|
|
| 49. |
- Han, Xu, et al.
(författare)
-
Performance of piano trills: effects of hands, fingers, notes and emotions
- 2019
-
Ingår i: Combined proceedings of the Nordic Sound and Music Computing Conference 2019 and the Interactive Sonification Workshop 2019. - Stockholm. ; , s. 9-15
-
Konferensbidrag (refereegranskat)abstract
- Trill is a type of musical ornament. In automatic playback of piano music scores, trills are usually synthesised as a sequence of repeated notes with equal duration and dynamic level. This is not how trills are performed by pianists. In this study, trills were performed by three pianists on a Yamaha Disklavier and recorded as both audio and MIDI files. Then note duration, inter-onset interval (IOI) and key velocity for each note were extracted from MIDI files and analyzed in relation to hands, notes and emotions. Four significant effects were found; 1) hand effect: trills on right hand were in average performed with a faster rate, shorter note duration, longer off duration and faster key velocity, 2) finger effect: within the two notes forming a trill, notes with lower fingering number were performed with shorter off duration, while keeping note duration and key velocity close, 3) emotion effect: emotion mainly contributed to dynamic level, 4) crescendo effect: when crescendo happened, note duration and off duration compensated with each other and kept IOI at a almost constant value.
|
|
| 50. |
- Hansen, Kjetil Falkenberg, 1972-, et al.
(författare)
-
Active Listening and Expressive Communication for Children with Hearing Loss Using Getatable Environments for Creativity
- 2012
-
Ingår i: Journal of New Music Research. - Stockholm : Taylor & Francis. - 0929-8215 .- 1744-5027. ; 41:4, s. 365-375
-
Tidskriftsartikel (refereegranskat)abstract
- This paper describes a system for accommodating active listening for persons with hearing aids or cochlear implants, with a special focus on children at an early stage of cognitive development and with additional physical disabilities. A system called the Soundscraper is proposed and consists of a software part in Pure data and a hardware part using an Arduino microcontroller with a combination of sensors. For both the software and hardware development it was important to always ensure that the system was flexible enough to cater for the very different conditions that are characteristic of the intended user group.The Soundscraper has been tested with 25 children with good results. An increased attention span was reported, as well as positively surprising reactions from children where the caregivers were unsure whether they could hear at all. The sound synthesis methods, the gesture sensors and the employed parameter mapping were all simple, but they provided a controllable and sufficiently complex sound environment even with limited interaction. A possible future outcome of the application is the adoption of long-time analysis of sound preferences as opposed to traditional audiological investigations.
|
|
