Haptic feedback combined with movement sonification using a friction sound improves task performance in a virtual throwing task

• In this paper we present a study on the effects of auditory- and haptic feedback in a virtual throwing task performed with a point-based haptic device. The main research objective was to investigate if and how task performance and perceived intuitiveness is affected when interactive sonification and/or haptic feedback is used to provide real-time feedback about a movement performed in a 3D virtual environment. Emphasis was put on task solving efficiency and subjective accounts of participants’ experiences of the multimodal interaction in different conditions. The experiment used a within-subjects design in which the participants solved the same task in different conditions: visual-only, visuohaptic, audiovisual and audiovisuohaptic. Two different sound models were implemented and compared. Significantly lower error rates were obtained in the audiovisuohaptic condition involving movement sonification based on a physical model of friction, compared to the visual-only condition. Moreover, a significant increase in perceived intuitiveness was observed for most conditions involving haptic and/or auditory feedback, compared to the visual-only condition. The main finding of this study is that multimodal feedback can not only improve perceived intuitiveness of an interface but that certain combinations of haptic feedback and movement sonification can also contribute with performance-enhancing properties. This highlights the importance of carefully designing feedback combinations for interactive applications.

Subject headings

TEKNIK OCH TEKNOLOGIER  -- Annan teknik -- Mediateknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies -- Media Engineering (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Annan teknik -- Interaktionsteknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies -- Interaction Technologies (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Signalbehandling (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Signal Processing (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap -- Människa-datorinteraktion (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Human Computer Interaction (hsv//eng)

Keyword

Sonification

Haptic feedback

Multimodal interaction

Task performance

Media Technology

Medieteknik

Human-computer Interaction

Människa-datorinteraktion

Informations- och kommunikationsteknik

Information and Communication Technology

Publication and Content Type

ref (subject category)

art (subject category)