| 1. |
- Daněček, Radek, et al.
(författare)
-
Emotional Speech-Driven Animation with Content-Emotion Disentanglement
- 2023
-
Ingår i: Proceedings - SIGGRAPH Asia 2023 Conference Papers, SA 2023. - : Association for Computing Machinery (ACM).
-
Konferensbidrag (refereegranskat)abstract
- To be widely adopted, 3D facial avatars must be animated easily, realistically, and directly from speech signals. While the best recent methods generate 3D animations that are synchronized with the input audio, they largely ignore the impact of emotions on facial expressions. Realistic facial animation requires lip-sync together with the natural expression of emotion. To that end, we propose EMOTE (Expressive Model Optimized for Talking with Emotion), which generates 3D talking-head avatars that maintain lip-sync from speech while enabling explicit control over the expression of emotion. To achieve this, we supervise EMOTE with decoupled losses for speech (i.e., lip-sync) and emotion. These losses are based on two key observations: (1) deformations of the face due to speech are spatially localized around the mouth and have high temporal frequency, whereas (2) facial expressions may deform the whole face and occur over longer intervals. Thus we train EMOTE with a per-frame lip-reading loss to preserve the speech-dependent content, while supervising emotion at the sequence level. Furthermore, we employ a content-emotion exchange mechanism in order to supervise different emotions on the same audio, while maintaining the lip motion synchronized with the speech. To employ deep perceptual losses without getting undesirable artifacts, we devise a motion prior in the form of a temporal VAE. Due to the absence of high-quality aligned emotional 3D face datasets with speech, EMOTE is trained with 3D pseudo-ground-truth extracted from an emotional video dataset (i.e., MEAD). Extensive qualitative and perceptual evaluations demonstrate that EMOTE produces speech-driven facial animations with better lip-sync than state-of-the-art methods trained on the same data, while offering additional, high-quality emotional control.
|
|
| 2. |
- Stojanovski, Todor, et al.
(författare)
-
Rethinking Computer-Aided Architectural Design (CAAD) - From Generative Algorithms and Architectural Intelligence to Environmental Design and Ambient Intelligence
- 2022
-
Ingår i: Computer-Aided Architectural Design. - Singapore : Springer Nature. ; , s. 62-83
-
Konferensbidrag (refereegranskat)abstract
- Computer-Aided Architectural Design (CAAD) finds its historical precedents in technological enthusiasm for generative algorithms and architectural intelligence. Current developments in Artificial Intelligence (AI) and paradigms in Machine Learning (ML) bring new opportunities for creating innovative digital architectural tools, but in practice this is not happening. CAAD enthusiasts revisit generative algorithms, while professional architects and urban designers remain reluctant to use software that automatically generates architecture and cities. This paper looks at the history of CAAD and digital tools for Computer Aided Design (CAD), Building Information Modeling (BIM) and Geographic Information Systems (GIS) in order to reflect on the role of AI in future digital tools and professional practices. Architects and urban designers have diagrammatic knowledge and work with design problems on symbolic level. The digital tools gradually evolved from CAD to BIM software with symbolical architectural elements. The BIM software works like CAAD (CAD systems for Architects) or digital board for drawing and delivers plans, sections and elevations, but without AI. AI has the capability to process data and interact with designers. The AI in future digital tools for CAAD and Computer-Aided Urban Design (CAUD) can link to big data and develop ambient intelligence. Architects and urban designers can harness the benefits of analytical ambient intelligent AIs in creating environmental designs, not only for shaping buildings in isolated virtual cubicles. However there is a need to prepare frameworks for communication between AIs and professional designers. If the cities of the future integrate spatially analytical AI, are to be made smart or even ambient intelligent, AI should be applied to improving the lives of inhabitants and help with their daily living and sustainability.
|
|
