| 1. |
- Bisig, Daniel, et al.
(författare)
-
Raw Music from Free Movements: Early Experiments in Using Machine Learning to Create Raw Audio from Dance Movements
- 2021
-
Ingår i: Proceedings of AI Music Creativity Conference 2021. - 9783200082724
-
Konferensbidrag (refereegranskat)abstract
- Raw Music from Free Movements is a deep learning architecture that translates pose sequences into audio waveforms. The architecture combines a sequence-to-sequence model generating audio encodings and an adversarial autoencoder that generates raw audio from audio encodings. Experiments have been conducted with two datasets: a dancer improvising freely to a given music, and music created through simple movement sonification. The paper presents preliminary results. These will hopefully lead closer towards a model which can learn from the creative decisions a dancer makes when translating music into movement and then follow these decisions reversely for the purpose of generating music from movement.
|
|
| 2. |
- Boersen, Ronald, et al.
(författare)
-
Chatterbox: an interactive system of gibberish agents
- 2020
-
Ingår i: Proceedings of 26th International Symposium of Electronic Arts (ISEA 2020). - 9782981641328
-
Konferensbidrag (refereegranskat)abstract
- We present the interactive multi-agent system Chatterbox, as part of the sound art installation Translanguaging, exploring the notion of translanguaging as a mediation of multilingual and intercultural communication. We discuss the act of languaging as a dual process comprising both semantic language communication, as well as paralanguage that relates to the affective, personal, and cultural aspects related to translanguaging. Through the creation of the Chatterbox agent, generating gibberish vocal streams devoid of semantic content, we aim at highlighting the paralinguistic dimension of languaging. The agent model comprises a kind of gradient map, clustering a segmented corpus of vocal sounds in the latent space of a self-organized map, according to its paralinguistic fingerprint. We utilize Factor Oracles for the creative generation of novel utterances of paralanguaging gibberish by the agent. Incorporating simple subsumption architecture inspired rules, we further moderate the interaction between the gibberish agents, creating rich and complex multi-agent behavior in “paralanguaging discussion”. We outline the artistic and technical considerations in developing our Chatterbox agent throughout the paper. We share several observations made throughout the process of creating the Chatterbox agent, highlighting some of the connections between the notion of (trans)languaging and the implementation of our model.
|
|
| 3. |
- Cotton, Kelsey, 1994, et al.
(författare)
-
Singing for the Missing: Bringing the Body Back to AI Voice and Speech Technologies
- 2024
-
Ingår i: 9th International Conference on Movement and Computing. - 9798400709944
-
Konferensbidrag (refereegranskat)abstract
- Technological advancements in deep learning for speech and voice have contributed to a recent expansion in applications for voice cloning, synthesis and generation. Invisibilised stakeholders in this expansion are numerous absent bodies, whose voices and voice data have been integral to the development and refinement of these speech technologies. This position paper probes current working practices for voice and speech in machine learning and AI, in which the bodies of voices are “invisibilised". We examine the facts and concerns about the voice-Body in applications of AI-voice technology. We do this through probing the wider connections between voice data and Schaefferian listening; speculating on the consequences of missing Bodies in AI-Voice; and by examining how vocalists and artists working with synthetic Bodies and AI-voices are ‘bringing the Body back’ in their own practices. We contribute with a series of considerations for how practitioners and researchers may help to ‘bring the Body back’ into AI-voice technologies.
|
|
| 4. |
- Diapoulis, Georgios, 1981, et al.
(författare)
-
Bottom-up live coding: Analysis of continuous interactions towards predicting programming behaviours
- 2022
-
Ingår i: NIME proceedings. - : PubPub. - 2220-4806.
-
Konferensbidrag (refereegranskat)abstract
- This paper explores a minimalist approach to live coding using a single input parameter to manipulate the graph structure of a finite state machine through a stream of bits. This constitutes an example of bottom-up live coding, which operates on a low level language to generate a high level structure output. Here we examine systematically how to apply mappings of continuous gestural interactions to develop a bottom-up system for predicting programming behaviours. We conducted a statistical analysis based on a controlled data generation procedure. The findings concur with the subjective experience of the behavior of the system when the user modulates the sampling frequency of a variable clock using a knob as an input device. This suggests that a sequential predictive model may be applied towards the development of a tactically predictive system according to Tanimoto’s hierarchy of liveness. The code is provided in a git repository.
|
|
| 5. |
- Obaid, Mohammad, 1982, et al.
(författare)
-
Social Drones for Health and Well-being
- 2022
-
Ingår i: ACM International Conference Proceeding Series. - New York, NY, USA : ACM.
-
Konferensbidrag (refereegranskat)abstract
- Drone activities in our daily lives are on the rise, so is the need to advance research on how to design drones that can interact with human users in an autonomous and socially acceptable way. The focus of this workshop is to discuss this emerging technology in the context of health and well-being applications. In particular, we take an interdisciplinary approach to understand how to empower drones with more autonomous and AI-driven social features, hence leveraging possible methods, behaviors, and solutions for AI driven social drones in the context of health and well-being. The main activities of this workshop will entail hands on activities and discussions in collaborative format. The expected outcomes aim at building up synergies between participants to lead future steps in the domain of autonomous social drones in health care and well-being.
|
|
| 6. |
- Prpa, Mirjana, et al.
(författare)
-
Attending to Breath: Exploring How the Cues in a Virtual Environment Guide the Attention to Breath and Shape the Quality of Experience to Support Mindfulness
- 2018
-
Ingår i: DIS 2018 - Proceedings of the 2018 Designing Interactive Systems Conference. - New York, NY, USA : ACM. - 9781450351980
-
Konferensbidrag (refereegranskat)abstract
- Busy daily lives and ongoing distractions often make people feel disconnected from their bodies and experiences. Guided attention to self can alleviate this disconnect as in focused-attention meditation, in which breathing often constitutes the primary object on which to focus attention. In this context, sustained breath awareness plays a crucial role in the emergence of the meditation experience. We designed an immersive virtual environment (iVE) with a generative soundtrack that supports sustained attention on breathing by employing the users' breathing in interaction. Both sounds and visuals are directly mapped to the user's breathing patterns, thus bringing the awareness researched. We conducted micro-phenomenology interviews to unfold the process in which breath awareness can be induced and sustained in this environment. The findings revealed the mechanisms by which audio and visual cues in VR can elicit and foster breath-awareness, and unfolded the nuances of this process through subjective experiences of the study participants. Finally, the results emphasize the important role that a sense of agency and control have in shaping the overall quality of the experience. This can in turn inform the design specifications of future mindfulness-based designs focused on breath awareness.
|
|
| 7. |
- Prpa, Mirjana, et al.
(författare)
-
Respire: a Breath Away from the Experience in Virtual Environment
- 2018
-
Ingår i: Conference on Human Factors in Computing Systems - Proceedings. - New York, NY, USA : ACM.
-
Konferensbidrag (refereegranskat)abstract
- Respire is a virtual environment presented on a head-mounted display with generative sound built upon our previous work Pulse Breath Water. The system follows the changes in user's breathing patterns upon which it generates changes in the audio and virtual environment. The piece is built upon mindfulness-based design principles with a focus on the breath as a primary object of the user's attention, and employs various approaches to augmenting breathing in the virtual environment.
|
|
| 8. |
- Prpa, Mirjana, et al.
(författare)
-
The Pulse Breath Water System: Exploring Breathing as an Embodied Interaction for Enhancing the Affective Potential of Virtual Reality
- 2017
-
Ingår i: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). - Cham : Springer International Publishing. - 1611-3349 .- 0302-9743. ; 10280
-
Konferensbidrag (refereegranskat)abstract
- We introduce Pulse Breath Water, an immersive virtual environment (VE) with affect estimation in sound. We employ embodied interaction between a user and the system through the user’s breathing frequencies mapped to the system’s behaviour. In this study we investigate how two different mappings (metaphoric, and “reverse”) of embodied interaction design might enhance the affective properties of the presented system. We build on previous work in embodied cognition, embodied interaction, and affect estimation in sound by examining the impact of affective audiovisuals and two kinds of interaction mapping on the user’s engagement, affective states, and overall experience. The insights gained through questionnaires and semi-structured interviews are discussed in the context of participants’ lived experience and the limitations of the system to be addressed in future work.
|
|
| 9. |
- Strauss, Lucy, et al.
(författare)
-
instance: Soma-based multi-user interaction design for the telematic sonic arts
- 2021
-
Ingår i: Organised Sound. - 1355-7718 .- 1469-8153. ; 26:3, s. 390-402
-
Tidskriftsartikel (refereegranskat)abstract
- The telematic work instance is a performance for viola and dance that digitally connects performers in Vancouver and Cape Town. The network interface enables a violist and a dancer to simultaneously play multi-user digital music-dance instruments over the internet with music and dance. The composition, design and performance interaction of instance draw from acoustic multi-user instrument paradigms and music-dance interactions in the African performing arts to explore the idiosyncrasies of the telematic performance space. The iterative design process implements soma-based research methods to inspire sonic compositional material with the body and to explore the performers' embodied experience of sonic aesthetics during their interaction.
|
|
| 10. |
- Tatar, Kivanc, 1988, et al.
(författare)
-
Latent Timbre Synthesis: Audio-based Variational Auto-Encoders for Music Composition Applications
- 2020
-
Ingår i: Neural Computing and Applications. - : Springer Science and Business Media LLC. - 0941-0643 .- 1433-3058. ; 33:The Special Issue of Neural Computing and Applications: “Networks in Art, Sound and Design.”, s. 67-84
-
Tidskriftsartikel (refereegranskat)abstract
- We present the Latent Timbre Synthesis, a new audio synthesis method using deep learning. The synthesis method allows composers and sound designers to interpolate and extrapolate between the timbre of multiple sounds using the latent space of audio frames. We provide the details of two Variational Autoencoder architectures for the Latent Timbre Synthesis and compare their advantages and drawbacks. The implementation includes a fully working application with a graphical user interface, called interpolate_two, which enables practitioners to generate timbres between two audio excerpts of their selection using interpolation and extrapolation in the latent space of audio frames. Our implementation is open source, and we aim to improve the accessibility of this technology by providing a guide for users with any technical background. Our study includes a qualitative analysis where nine composers evaluated the Latent Timbre Synthesis and the interpolate_two application within their practices. 2.14.0.0
|
|
| 11. |
- Tatar, Kivanc, 1988, et al.
(författare)
-
Respire: Virtual Reality Art with Musical Agent Guided by Respiratory Interaction
- 2019
-
Ingår i: Leonardo Music Journal. - : MIT Press - Journals. - 0961-1215 .- 1531-4812. ; 29, s. 19-24
-
Tidskriftsartikel (refereegranskat)abstract
- Respire is an immersive art piece that brings together three components: an immersive virtual reality (VR) environment, embodied interaction (via a breathing sensor) and a musical agent system to generate unique experiences of augmented breathing. The breathing sensor controls the user’s vertical elevation of the point of view under and over the virtual ocean. The frequency and patterns of breathing data guide the arousal of the musical agent, and the waviness of a virtual ocean in the environment. Respire proposes an intimate exploration of breathing through an intelligent mapping of breathing data to the parameters of visual and sonic environments.
|
|
| 12. |
- Tatar, Kivanc, 1988, et al.
(författare)
-
Sound Design Strategies for Latent Audio Space Explorations using Deep Learning Architectures
- 2023
-
Ingår i: Proceedings of the Sound and Music Computing Conferences. - 2518-3672. ; 2023-June, s. 239-246
-
Konferensbidrag (refereegranskat)abstract
- The research in Deep Learning applications in sound and music computing have gathered an interest in the recent years; however, there is still a missing link between these new technologies and on how they can be incorporated into real-world artistic practices. In this work, we explore a well-known Deep Learning architecture called Variational Autoencoders (VAEs). These architectures have been used in many areas for generating latent spaces where data points are organized so that similar data points locate closer to each other. Previously, VAEs have been used for generating latent timbre spaces or latent spaces of symbolic music excepts. Applying VAE to audio features of timbre requires a vocoder to transform the timbre generated by the network to an audio signal, which is computationally expensive. In this work, we apply VAEs to raw audio data directly while bypassing audio feature extraction. This approach allows the practitioners to use any audio recording while giving flexibility and control over the aesthetics through dataset curation. The lower computation time in audio signal generation allows the raw audio approach to be incorporated into real-time applications. In this work, we propose three strategies to explore latent spaces of audio and timbre for sound design applications. By doing so, our aim is to initiate a conversation on artistic approaches and strategies to utilize latent audio spaces in sound and music practices.
|
|
| 13. |
- Zappi, Victor, et al.
(författare)
-
The Neuralacoustics Project: Exploring Deep-Learnin for Lightweight Numerical Modeling Synthesis
- 2022
-
Ingår i: Embedded AI for NIME: Challenges and Opportunities Workshop at New Interfaces for Musical Expression.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- Workshop, Session 1. We present a newly started project called Neuralacoustics, aimed at using embedded AI to reduce the computational requirements of numerical modeling synthesis. To achieve this goal, we propose a novel design pipeline, in which neural networks are first trained off-line to approximate the behavior of target interactive acoustic systems, and then deployed on embedded and mobile platforms as real-time synthesis engines.
|
|
