Unimore logo AImageLab
Back to the research area

Human-Robot Interaction

Collaborative robots, or cobots, have entered the automation market for several years now, achieving a rather rapid and wide diffusion. The Human-Robot Interaction (HRI) and Human-Robot Collaboration (HRC) paradigms still have unexplored potential and challenges not yet fully investigated and solved.

Among others, the knowledge of the instantaneous pose of robots and humans in a shared workspace is a key element to set up an effective and fruitful collaboration. The pose estimation problem opens up to new applications, ranging from solutions for the safety of the interaction to behavior analysis.

Despite robots usually provide their encoder status through dedicated communication channels, enabling the use of forward kinematics, an external method is desirable in certain cases where the robot controller is not accessible. In particular, aiming to have an end-to-end system which predicts the pose of both robots and humans, a method based on an external camera view of a collaborative scene is more suitable.



Keywordshuman-robot interaction, human-robot collaboration, 3D pose estimation, RGB-D perception


Dataset

SimBa

The SimBa dataset is composed of both synthetic and real sequences in which the Rethink Baxter robot moves respectively to a set of random pick-n-place locations on a table, assuming realistic poses.

Dataset page


Paper

Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation from Depth Maps

Under review

This work proposes a non-invasive and light-invariant framework based on depth devices and deep neural networks to estimate the 3D pose of robots from an external camera.  We introduce a novel representation of the predicted pose, namely Semi-Perspective Decoupled Heatmaps (SPDH), to efficiently and accurately predict 3D joint locations in world coordinates using common deep neural networks designed for the 2D Human Pose Estimation.

Further details and code will be released soon!