Opportunities and obligations to take turns in collaborative multi-party human-robot interaction

• In this paper we present a data-driven model for detecting opportunities and obligations for a robot to take turns in multi-party discussions about objects. The data used for the model was collected in a public setting, where the robot head Furhat played a collaborative card sorting game together with two users. The model makes a combined detection of addressee and turn-yielding cues, using multi-modal data from voice activity, syntax, prosody, head pose, movement of cards, and dialogue context. The best result for a binary decision is achieved when several modalities are combined, giving a weighted F1 score of 0.876 on data from a previously unseen interaction, using only automatically extractable features.

Subject headings

TEKNIK OCH TEKNOLOGIER  -- Annan teknik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Other Engineering and Technologies (hsv//eng)

Keyword

Human computer interaction

Modal analysis

Robots

Binary decision

Card-sorting

Combined detections

Data-driven model

Head pose

Multi-modal data

Robot head

Voice activity

Human robot interaction

Publication and Content Type

ref (subject category)

kon (subject category)