Towards Designing Better Speech Agent Interaction : Using Eye Gaze for Interaction

• This research is about addressing the need to better understand interaction with conversational user interfaces (CUIs) and how human-technology `conversations' can be improved by drawing on the lessons learned from human-human interaction. It focuses on incorporating abstractions of complex human behaviour, specifically gaze, to enhance interactions with speech agents in conversations. Across four empirical studies, a mix of methods is used to look closely at the interaction between the user and the system.I offer empirical and conceptual contributions for interaction designers and researchers. First, I present a novel speech interface, Tama, which is a gaze-aware speech agent designed to explore the use of gaze in conversational interactions with smart speakers. Second, I present the empirical contributions, that is, the studies that document the interactions with and around speech interfaces, including ongoing, non-system-directed speech. A moment-by-moment analysis of these interactions highlights the opportunities that the gaze offers as a modality to enhance the interaction with the speech agent, as well as the problems and limitations when such a modality is used. The third contribution is a conceptual contribution made by providing perspective on minimal anthropomorphic design. This produces interactions that are not human-like in terms of behaviour but do take advantage of the skills used in human interaction as a key to advancing interactions with speech agents.Based on my research work and contributions, I reflect upon advancing interactions with speech interfaces, focusing on what different technologies can offer and the possibility of taking the next step in designing CUIs. I then discuss the need to bridge the work of different fields (i.e. conversation analysis (CA), human-computer interaction (HCI), and human-robot interaction (HRI)) to combine models and approaches from all these fields in order to guide designers building speech systems. I see three competing yet complementary interaction paradigms across CUIs. I call these paradigms Direct Speech Interaction, Agent-Mediated Interaction, and Para-Speech Interaction. Each of these paradigms has specific challenges and opportunities for interaction. 

Ämnesord

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Robotteknik och automation (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Robotics (hsv//eng)

NATURVETENSKAP  -- Data- och informationsvetenskap (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Annan teknik -- Interaktionsteknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies -- Interaction Technologies (hsv//eng)

Nyckelord

Human-computer interaction

Conversational User Interfaces

Smart Speaker

Gaze Interaction

User Studies

Interaction Design

Gaze Detection

Agent Interaction

Wizard-of-Oz

data- och systemvetenskap

Computer and Systems Sciences

Publikations- och innehållstyp

vet (ämneskategori)

dok (ämneskategori)