Adaptive real-time similar repetitive manual procedure prediction and robotic procedure generation for human-robot collaboration

• Manual procedure recognition and prediction are essential for practical human-robot collaboration in industrial tasks, such as collaborative assembly. However, current research mostly focuses on diverse human motions, while the similar repetitive manual procedures that are prevalent in real production tasks are often overlooked. Furthermore, the dynamic uncertainty caused by human-robot interferences and the generalisation of individuals, scenarios, and multiple sensor deployments pose challenges for implementing manual procedure prediction and robotic procedure generation. To address these issues, this paper proposes a real-time, similar repetitive procedure-oriented human skeleton processing system that employs the human skeleton as a robust modality. It utilises an improved deep spatial-temporal graph convolutional network and a FIFO queue-based discriminator for real-time data processing, procedure prediction, and generation. The proposed method is validated on multiple datasets with tens of individuals engaged in a real dynamic and uncertain human-robot collaborative assembly cell and able to run on entry-level hardware. The results demonstrate competitive performance of handcraft feature-free, early prediction and generalisation on individual variance, environment background, camera position, lighting conditions, and stochastic interference in human-robot collaboration.

Ämnesord

NATURVETENSKAP  -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)

NATURAL SCIENCES  -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Robotteknik och automation (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Robotics (hsv//eng)

Nyckelord

Collaborative assembly

Human-robot collaboration

Manual procedure prediction

Robotic procedure generation

Similar repetitive manual procedure

Publikations- och innehållstyp

ref (ämneskategori)

art (ämneskategori)