| 1. |
- Svensson Lundmark, Malin, et al.
(författare)
-
Segmental articulations across prosodic levels
- 2023
-
Ingår i: Proceedings of the 13th International Conference Nordic Prosody Conference. - 9788366675728 ; , s. 255-261
-
Bokkapitel (refereegranskat)abstract
- This study deals with segmental articulations based on acceleration signals from EMA data. We test how well the one-to-one relationship between acceleration peaks and segment boundaries (as described within the framework of the Descriptive Approach to Segmental Articulations) works across different prosodic levels in Swedish, here: focused and unfocused position, with the assumption of a positive correlation of prominence level with segmental lengthening. Furthermore, we use forced alignment (the Montreal ForcedAligner), instead of manual segmentation, to collect landmarks on the acceleration signals. The result shows that the correlation between acceleration peaks and segment boundaries is even stronger in focus position, and that the pattern holds for both lips at /m/, tongue body at /a/, and tongue tip data at /n/ and /l/(although the tongue tip results were affected by analyzing them together). The study shows that the acceleration signal works well for forced alignment in Swedish, which creates strong arguments for the one-to-one relationship between acceleration peaks and acoustic segment boundaries, and in addition that it holds across different levels of prominence. Furthermore, the study prompts a deeper discussion of the relationship between acceleration and prominence as a result of articulatory effort.
|
|
| 2. |
- Frid, Johan, et al.
(författare)
-
EMA-based head movements and phrasing : a preliminary study
- 2018
-
Ingår i: - Gothenburg : University of Gothenburg. ; , s. 17-20, s. 17-20
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- In this paper we describe present work on multimodal prosody by means of simultaneous recordings of articulation and head movements. Earlier work has explored patterning, usage and machine-learning based detection of focal pitch accents, head beats and eyebrow beats through audiovisual recordings. Kinematic data obtained through articulography allows for more comparable and accurate measurements, as well as three-dimensional data. Therefore, our current approach involves examining speech and body movements concurrently, using electromagnetic articulography (EMA). We have recorded large amounts of this kind of data previously, but for other purposes. In this paper, we present results from a preliminary study on the interplay between head movements and phrasing and find tendencies for upward movements occuring before and downward movements occuring after prosodic boundaries.
|
|
| 3. |
- Frid, Johan, et al.
(författare)
-
EMA-based head movements, word accent and vowel length
- 2019
-
Ingår i: Book of Abstracts MMSYM 2019. - : University of Leuven. ; , s. 11-11
-
Konferensbidrag (refereegranskat)abstract
- This paper describes on-going work in the field of multimodal prosody carried out by means of simultaneous recordings of speech acoustics, articulation and head movements. People naturally move their heads when they speak, and head movements have been found both to correlate strongly with the pitch and amplitude of the speaker's voices and to convey linguistic information. Here, we report on a study that explores how head movement patterns vary and co-occur with lexical pitch accents (and their acoustic correlates F0 and intensity) and vowel length. The study uses data from Swedish, where there are both two lexical pitch accents and two vowel lengths that differ phonologically.We use EMA (Electromagnetic articulography), which allows for high sample rates, accurate synchronisation of kinematic and acoustic recordings, as well as three-dimensional movement data. Kinematic data is obtained by gluing small sensors on the speakers’ articulators (tongue, lips, jaw). Head movement data is obtained by similar sensors on the nose ridge and behind the ears, which allows us to capture the angle of the tilt of the head.Articulatory data was collected from 18 South Swedish speakers (12 female) using a Carstens AG501. Each speaker read leading questions + sentences containing a target word from a prompter (presented eight times in random order), an arrangement employed to put a contrastive focus onto the last element in the target sentence. This left the target word in a low-prominence inducing context, hence controlling for possible effects of sentence intonation.For this study we used eight target words where pitch accent and vowel length were cross- matched so that there were two cases of each combination of word accent category and vowel length category. All words shared the similar word-initial C /m/, followed by a vowel that was either /a/ or /ɑ:/. The target words were segmented and time-normalized between 0 to 1 and the head tilt angle (sagAng) was normalized for each speaker by z-transforming the angles per speaker. Spatial movements were analysed using Generalized Additive Models, which we used to test if there were effects of segmental position (C versus V in the first syllable), word accent (1 or 2) and vowel length (short or long) on sagAng. Models were fit using the maximum likelihood (ML) estimation method.The Chi-Square test on the ML scores indicates that a model with the word accent distinction is significantly better than a model without it (X2(4.00)=632.796, p<2e-16***). Similarly, a model with vowel length distinction is significantly better than a model without it (X2(4.00)=820.997, p<2e-16***). Finally, a model with segmental position is significantly better than a model without it (X2(8.00)= 173.316, p<2e-16***).The results indicate that head nod patterns that occur in synchronisation with the stressed syllable of spoken words differ with respect to word accent, vowel length and segmental position. This could possibly point to an effect of F0 and intensity on the head nod movements.
|
|
| 4. |
- Frid, Johan, et al.
(författare)
-
EMA-based head movements, word accents, vowel length and segments : a preliminary study
- 2019
-
Ingår i: Proceedings from FONETIK 2019 Stockholm, June 10-12, 2019. - Stockholm : Stockholm University. - 9789177979845 - 9789177979852 ; , s. 125-126
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- This study describes on-going work in the field of multimodal prosody carried out by means of simultaneous recordings of speech acoustics, articulation and head movements.
|
|
| 5. |
- Frid, Johan, et al.
(författare)
-
Investigating visual prosody using articulography
- 2019
-
Ingår i: Proceedings of the Digital Humanities in the Nordic Countries 4th Conference. - : CEUR-WS.org. ; 2364
-
Konferensbidrag (refereegranskat)abstract
- In this paper we describe present work on multimodal prosody by means of simultaneous recordings of articulation and head movements. Earlier work has explored patterning, usage and machine-learning based detection of focal pitch accents, head beats and eyebrow beats through audiovisual recordings. Kinematic data obtained through articulography allows for more compa- rable and accurate measurements, as well as three-dimensional data. Therefore, our current approach involves examining speech and body movements concurrently, using electromagnetic articulography (EMA). We have recorded large amounts of this kind of data previously, but for other purposes. In this paper, we present results from a study on the interplay between head movements and phrasing and find tendencies for upward movements occuring before and downward movements occuring after prosodic boundaries.
|
|
| 6. |
|
|
| 7. |
|
|
| 8. |
- Frid, Johan, et al.
(författare)
-
Towards classification of head movements in audiovisual recordings of read news
- 2017
-
Ingår i: Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016). - Linköping : Linköping University Electronic Press. - 1650-3740 .- 1650-3686. - 9789176854235 ; , s. 4-9
-
Konferensbidrag (refereegranskat)abstract
- In this paper we develop a system for detection of word-related head movements in audiovisu-al recordings of read news. Our materials consist of Swedish television news broadcasts and comprise audiovisual recordings of five news readers (two female, three male). The corpus was manually labelled for head movement, applying a simplistic annotation scheme consisting of a binary decision about absence/presence of a movement in relation to a word. We use OpenCV for frontal face detection and based on this we calculate velocity and acceleration features. Then we train a machine learning system to predict absence or presence of head movement and achieve an accuracy of 0.892, which is better than the baseline. The system may thus be helpful for head movement labelling.
|
|
| 9. |
|
|
| 10. |
|
|
