Multimodal depth in film : A proposal for a multimodal and sound-oriented approach to intersemiotic analysis

• Film is an inherently multimodal medium. Despite this, film studies and film musicology alike have a tradition of approaching the discussion of formal interaction in film as a question of how well the image track and the music relate to each other, rather than as a question of complex multimodal intersemiosis (for example in epitomic works such as: Carroll (1996); Eisenstein and Leyda (1969); Gorbman (1987)). Even though this reductive theorisation has been questioned and challenged from media and (film)musicology scholars (e.g. Chion, 1994; Cook, 1998; Langkjær, 2008) and from within the field of multimodality (e.g. Bateman and Schmidt (2013); Tseng (2013)), the notion that film = audiovisual = moving image + music, is still largely dominant. Following Walter Murch and Iben Have’s ideas about an audiovisual dimension (Chion, 1994; Have, 2008), I want to propose that the formal interrelations in film can be analysed according to a multimodal dimension and that the depth of this dimension is decided by the level of dialogue in the multimodal complex. Rather than seeing film as an addition of images and music, I propose that film consists of a number of visual modes (e.g. lightning, viewing perspective, facial gestures) and a number of auditory modes (e.g. dialogue, instrumentation, and musical harmonics), which are orchestrated as different ‘voices’ (drawing on Bakhtin’s ideas of dialogue and polyphony (Bakhtin, 1981)). The intersemiosis is a result of a dialogic interrelationship of all of these voices, or modes. It is my working hypothesis that a high level of dialogue equals a more complex meaning potential and thus a deeper multimodal dimension, whereas a low level of dialogue equals a more straight-forward meaning potential and a flatter multimodal dimension.

Ämnesord

HUMANIORA  -- Annan humaniora -- Övrig annan humaniora (hsv//swe)

HUMANITIES  -- Other Humanities -- Other Humanities not elsewhere specified (hsv//eng)

Nyckelord

intersemiosis

multimodality

music

film

animation

Humaniora

Humanities

Publikations- och innehållstyp

vet (ämneskategori)

kon (ämneskategori)