Spontaneous conversational speech synthesis from found data

QC 20200310
Synthesising spontaneous speech is a difficult task due to disfluencies, high variability and syntactic conventions different from those of written language. Using found data, as opposed to lab-recorded conversations, for speech synthesis adds to these challenges because of overlapping speech and the lack of control over recording conditions. In this paper we address these challenges by using a speaker-dependent CNN-LSTM breath detector to separate continuous recordings into utterances, which we here apply to extract nine hours of clean single-speaker breath groups from a conversational podcast. The resulting corpus is transcribed automatically (both lexical items and filler tokens) and used to build several voices on a Tacotron 2 architecture. Listening tests show: i) pronunciation accuracy improved with phonetic input and transfer learning; ii) it is possible to create a more fluent conversational voice by training on data without filled pauses; and iii) the presence of filled pauses improved perceived speaker authenticity. Another listening test showed the found podcast voice to be more appropriate for prompts from both public speeches and casual conversations, compared to synthesis from found read speech and from a manually transcribed lab-recorded spontaneous conversation.

Henter, Gustav Eje,Assistant ProfessorKTH,Tal, musik och hörsel, TMH(Swepub:kth)u1cj22n0 (författare)
Beskow, JonasKTH,Tal, musik och hörsel, TMH(Swepub:kth)u1g7i6t9 (författare)
Gustafson, Joakim,professor,1966-KTH,Tal, musik och hörsel, TMH(Swepub:kth)u183famk (författare)
KTHTal, musik och hörsel, TMH (creator_code:org_t)

Ingår i:Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH: ISCA, s. 4435-4439

Av författaren/redakt...: Székely, Éva; Henter, Gustav E ...; Beskow, Jonas; Gustafson, Joaki ...

Om ämnet

NATURVETENSKAP: NATURVETENSKAP; och Data och informa ...; och Språkteknologi

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.