| 1. |
- Bresin, Roberto, 1963-, et al.
(författare)
-
Robust Non-Verbal Expression in Humanoid Robots: New Methods for Augmenting Expressive Movements with Sound
- 2021
-
Konferensbidrag (refereegranskat)abstract
- The aim of the SONAO project is to establish new methods basedon sonification of expressive movements for achieving a robust interaction between users and humanoid robots. We want to achievethis by combining competences of the research team members inthe fields of social robotics, sound and music computing, affective computing, and body motion analysis. We want to engineersound models for implementing effective mappings between stylized body movements and sound parameters that will enable anagent to express high-level body motion qualities through sound.These mappings are paramount for supporting feedback to andunderstanding robot body motion. The project will result in thedevelopment of new theories, guidelines, models, and tools forthe sonic representation of high-level body motion qualities in interactive applications. This work is part of the growing researchfield known as data sonification, in which we combine methodsand knowledge from the fields of interactive sonification, embodied cognition, multisensory perception, non-verbal and gesturalcommunication in robots.
|
|
| 2. |
- Falkenberg, Kjetil, Docent, 1972-, et al.
(författare)
-
Creating digital musical instruments with and for children: Including vocal sketching as a method for engaging in codesign
- 2020
-
Ingår i: Human Technology. - : Centre of Sociological Research, NGO. - 1795-6889. ; 16:3, s. 348-371
-
Tidskriftsartikel (refereegranskat)abstract
- A class of master of science students and a group of preschool children codesigned new digital musical instruments based on workshop interviews involving vocal sketching, a method for imitating and portraying sounds. The aim of the study was to explore how the students and children would approach vocal sketching as one of several design methods. The children described musical instruments to the students using vocal sketching and other modalities (verbal, drawing, gestures). The resulting instruments built by the students were showcased at the Swedish Museum of Performing Arts in Stockholm. Although all the children tried vocal sketching during preparatory tasks, few employed the method during the workshop. However, the instruments seemed to meet the children’s expectations. Consequently, even though the vocal sketching method alone provided few design directives in the given context, we suggest that vocal sketching, under favorable circumstances, can be an engaging component that complements other modalities in codesign involving children.
|
|
| 3. |
- Hansen, Kjetil Falkenberg, et al.
(författare)
-
Unproved methods from the frontier in the course curriculum : A bidirectional and mutually beneficial research challenge
- 2020
-
Ingår i: INTED2020 Proceedings. - : IATED. - 9788409179398 ; , s. 7033-7038
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we report the experiences of students and teachers in a master course in Musical Communication and Music Technology at KTH Royal Institute of Technology. The students were exposed to vocal sketching [1], a novel sound design method, both as their course material and for the examination. The results in terms of learning outcome and course experience were confirmed and more than convincing, while the results in terms of validating the efficacy of the method were meagre.As part of our research, we designed an experiment where the students first interviewed preschool children who were asked to describe a fantasy musical instrument and then built it. The course schedule included lectures on voice sketching, sound synthesis, sound quality, new musical instruments, parameter mapping, and music programming. The project work and idea was presented during the first lecture, eight weeks before meeting the children. The interview took place in a workshop at the Swedish Museum for Performing Arts who had an exhibition of new musical instruments. Student/child pairs visited the exhibition in order to 1) familiarize themselves, 2) establish communication, and 3) get a common point of reference in terms of the exhibited instruments. After this process, the pairs completed an interview session inspired by [2]. The parents and teacher could join in if desired. The students got two weeks to build the instruments and present these at the museum. The purpose was not to evaluate the instruments, but to explore the vocal sketch method. The design and building phase was a prototyping task which the students were comfortable with. All design decisions needed to be set in relation to the course literature. All the presented projects followed a scenario- and contextual-inspired design approach [3] where a target solution needed to be established quickly grounded on a basic understanding of the agent (the child), its goals, and its presumed actions [4], and where the child mainly acted as informant [5]. While all the children could voice sketch, few actually did so in the interview. Despite this, the finished instruments matched the expectations of the children, and the course work satisfied the intended learning outcomes. As a research outcome, we suggest that future studies should include training vocal sketch techniques to produce suitable sounds. As for the pedagogical outcome, we are convinced from both the high quality of the works and the unusually positive course evaluations compared to previous years that the unproved research method was appropriate as course material. The bidirectional challenge in the research where students know that the method is experimental is hypothesized to further boost student motivation.
|
|
| 4. |
- Latupeirissa, Adrian Benigno, et al.
(författare)
-
Exploring emotion perception in sonic HRI
- 2020
-
Ingår i: 17th Sound and Music Computing Conference. - Torino : Zenodo. ; , s. 434-441
-
Konferensbidrag (refereegranskat)abstract
- Despite the fact that sounds produced by robots can affect the interaction with humans, sound design is often an overlooked aspect in Human-Robot Interaction (HRI). This paper explores how different sets of sounds designed for expressive robot gestures of a humanoid Pepper robot can influence the perception of emotional intentions. In the pilot study presented in this paper, it has been asked to rate different stimuli in terms of perceived affective states. The stimuli were audio, audio-video and video only and contained either Pepper’s original servomotors noises, sawtooth, or more complex designed sounds. The preliminary results show a preference for the use of more complex sounds, thus confirming the necessity of further exploration in sonic HRI.
|
|
| 5. |
- Latupeirissa, Adrian Benigno
(författare)
-
From Motion Pictures to Robotic Features : Adopting film sound design practices to foster sonic expression in social robotics through interactive sonification
- 2024
-
Konstnärligt arbete (övrigt vetenskapligt/konstnärligt)abstract
- This dissertation investigates the role of sound design in social robotics, drawing inspiration from robot depictions in science-fiction films. It addresses the limitations of robots’ movements and expressive behavior by integrating principles from film sound design, seeking to improve human-robot interaction through expressive gestures and non-verbal sounds.The compiled works are structured into two parts. The first part focuses on perceptual studies, exploring how people perceive non-verbal sounds displayed by a Pepper robot related to its movement. These studies highlighted preferences for more refined sound models, subtle sounds that blend with ambient sounds, and sound characteristics matching the robot’s visual attributes. This part also resulted in a programming interface connecting the Pepper robot with sound production tools.The second part focuses on a structured analysis of robot sounds in films, revealing three narrative themes related to robot sounds in films with implications for social robotics. The first theme involves sounds associated with the physical attributes of robots, encompassing sub-themes of sound linked to robot size, exposed mechanisms, build quality, and anthropomorphic traits. The second theme delves into sounds accentuating robots’ internal workings, with sub-themes related to learning and decision-making processes. Lastly, the third theme revolves around sounds utilized in robots’ interactions with other characters within the film scenes.Based on these works, the dissertation discusses sound design recommendations for social robotics inspired by practices in film sound design. These recommendations encompass selecting the appropriate sound materials and sonic characteristics such as pitch and timbre, employing movement sound for effective communication and emotional expression, and integrating narrative and context into the interaction.
|
|
| 6. |
- Latupeirissa, Adrian Benigno, et al.
(författare)
-
Probing Aesthetics Strategies for Robot Sound: Complexity and Materiality in Movement Sonification
- 2023
-
Ingår i: ACM Transactions on Human-Robot Interaction. - : Association for Computing Machinery (ACM). - 2573-9522.
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents three studies where we probe aesthetics strategies of sound produced by movement sonification of a Pepper robot by mapping its movements to sound models.We developed two sets of sound models. The first set was made by two sound models, a sawtooth-based one and another based on feedback chains, for investigating how the perception of synthesized robot sounds would depend on their design complexity. We implemented the second set of sound models for probing the “materiality” of sound made by a robot in motion. This set consisted of a sound synthesis based on an engine highlighting the robot’s internal mechanisms, a metallic sound synthesis highlighting the robot’s typical appearance, and a whoosh sound synthesis highlighting the movement.We conducted three studies. The first study explores how the first set of sound models can influence the perception of expressive gestures of a Pepper robot through an online survey. In the second study, we carried out an experiment in a museum installation with a Pepper robot presented in two scenarios: (1) while welcoming patrons into a restaurant and (2) while providing information to visitors in a shopping center. Finally, in the third study, we conducted an online survey with stimuli similar to those used in the second study.Our findings suggest that participants preferred more complex sound models for the sonification of robot movements. Concerning the materiality, participants liked better subtle sounds that blend well with the ambient sound (i.e., less distracting) and soundscapes in which sound sources can be identified. Also, sound preferences varied depending on the context in which participants experienced the robot-generated sounds (e.g., as a live museum installation vs. an online display).
|
|
| 7. |
- Latupeirissa, Adrian Benigno, et al.
(författare)
-
Semiotic analysis of robot sounds in films: implications for sound design in social robotics
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- This paper investigates the sound design of robots in films and their potential influence on the field of social robotics. Cinematic robot portrayals have inspired researchers and practitioners in Human-Robot Interaction (HRI). While the non-verbal sounds of iconic film robots like R2-D2 and Wall-E have been explored, this study takes a more comprehensive approach. We explore a broader selection of 15 films featuring humanoid robots across decades through a semiotic analysis of their non-verbal communication sounds, including those related to movements and internal mechanisms. Our analysis, guided by Bateman and Schmidt’s multimodal film analysis framework following Saussure’s organization of signs through paradigmatic and syntagmatic relations, interprets the paradigmatic axis as the examination of the sound and the syntagmatic axis as the examination of the events surrounding the sound. The findings uncover two primary film robot sound materials: mechanical and synthetic. Additionally, contextual analysis reveals three narrative themes and several sub-themes related to the physical attributes of robots, their internal workings, and their interactions with other characters. The discussion section explores the implications of these findings for social robotics, including the importance of sound materials, the role of movement sounds in communication and emotional expression, and the significance of narrative and context in human-robot interaction. The paper also acknowledges the challenges in translating film sound design into practical applications in social robotics. This study provides valuable insights for HRI researchers, practitioners, and sound designers seeking to enhance non-verbal auditory expressions in social robots.
|
|
| 8. |
- Latupeirissa, Adrian Benigno, et al.
(författare)
-
Sonic characteristics of robots in films
- 2019
-
Ingår i: Proceedings of the 16th Sound and Music Computing Conference. - Malaga, Spain. ; , s. 1-6
-
Konferensbidrag (refereegranskat)abstract
- Robots are increasingly becoming an integral part of our everyday life. Expectations on robots could be influenced by how robots are represented in science fiction films. We hypothesize that sonic interaction design for real-world robots may find inspiration from sound design of fictional robots. In this paper, we present an exploratory study focusing on sonic characteristics of robot sounds in films. We believe that findings from the current study could be of relevance for future robotic applications involving the communication of internal states through sounds, as well for sonification of expressive robot movements. Excerpts from five films were annotated and analysed using Long Time Average Spectrum (LTAS). As an overall observation, we found that robot sonic presence is highly related to the physical appearance of robots. Preliminary results show that most of the robots analysed in this study have “metallic” voice qualities, matching the material of their physical form. Characteristics of robot voices show significant differences compared to voices of human characters; fundamental frequency of robotic voices is either shifted to higher or lower values, and the voices span over a broader frequency band.
|
|
| 9. |
- Latupeirissa, Adrian Benigno, et al.
(författare)
-
Understanding non-verbal sound of humanoid robots in films
- 2020
-
Konferensbidrag (refereegranskat)abstract
- People’s mental model of robots is of importance since it can influence their expectations of how a robot should appear and behave, which in turn will affect the interaction between human and robot.The current mental model of robots is influenced by the presence ofrobots in films. Thus, understanding the principles of the design ofrobots in film would benefit the design of robots in the real world.This extended abstract presents an ongoing investigation of the use of non-verbal sounds of robot in films. Specifically, the investigation looks into the purpose of the sounds, how they are designed, and how the sound design has changed throughout the history of films. Preliminary result suggests the presence of a categoriza-tion of robotic sounds in films: inner workings, communication of movement, and expression of emotion.While further sound design principles are still being formulated,we would argue that having this historical perspective would benefitthe understanding of the current expectation of how a robot shouldsound, thus laying the groundwork for further research in the useof sound in HRI.
|
|
| 10. |
- Rafi, Ayesha Kajol, et al.
(författare)
-
Investigating the Role of Robot Voices and Sounds in Shaping Perceived Intentions
- 2023
-
Ingår i: HAI 2023 - Proceedings of the 11th Conference on Human-Agent Interaction. - : Association for Computing Machinery (ACM). ; , s. 425-427
-
Konferensbidrag (refereegranskat)abstract
- This study explores if, and how, the choices made regarding a robot's speaking voice and characteristic body sounds influence viewers' perceptions of its intent i.e., whether the robot's intention is positive or negative. The analysis focuses on robot representations and sounds in three films: "Robots"(2005) [1], "NextGen"(2018) [2], and "Love, Death, and Robots - Three Robots"(2019) [3]. In eight qualitative interviews, five parameters (tonality, intonation, volume, pitch, and speed) were used to understand robot sounds and the participant's perception of a robot's attitude and intentions. The study culminates in a set of recommendations and considerations for human-robot interaction designers to consider while sound coding for body, physiology, and movement.
|
|
