Search: onr:"swepub:oai:DiVA.org:kth-341039" >
Generation of speec...
Generation of speech and facial animation with controllable articulatory effort for amusing conversational characters
-
- Gustafsson, Joakim, Professor, 1966- (author)
- KTH,Tal, musik och hörsel, TMH
-
- Székely, Éva (author)
- KTH,Tal, musik och hörsel, TMH
-
- Beskow, Jonas (author)
- KTH,Tal, musik och hörsel, TMH
-
(creator_code:org_t)
- Institute of Electrical and Electronics Engineers (IEEE), 2023
- 2023
- English.
-
In: 23rd ACM International Conference on Interlligent Virtual Agent (IVA 2023). - : Institute of Electrical and Electronics Engineers (IEEE).
- Related links:
-
https://doi.org/10.1...
-
show more...
-
https://kth.diva-por... (primary) (Raw object)
-
https://urn.kb.se/re...
-
https://doi.org/10.1...
-
show less...
Abstract
Subject headings
Close
- Engaging embodied conversational agents need to generate expressive behavior in order to be believable insocializing interactions. We present a system that can generate spontaneous speech with supporting lip movements. The neural conversational TTSvoice is trained on a multi-style speech corpus that has been prosodically tagged (pitch and speaking rate) and transcribed (including tokens for breathing, fillers and laughter). We introduce a speech animation algorithm where articulatory effort can be adjusted. The facial animation is driven by time-stamped phonemes and prominence estimates from the synthesised speech waveform to modulate the lip and jaw movements accordingly. In objective evaluations we show that the system is able to generate speech and facial animation that vary in articulation effort. In subjective evaluations we compare our conversational TTS system’s capability to deliver jokes with a commercial TTS. Both systems succeeded equally good.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Språkteknologi (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Language Technology (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Robotteknik och automation (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Robotics (hsv//eng)
Publication and Content Type
- ref (subject category)
- kon (subject category)
To the university's database