3D Human Pose Estimation from Depth Maps

Human Pose Estimation is a fundamental task for many applications in the Computer Vision community. Although it is a widely developed area in the 2D domain, i.e. color images, few literature works aim to produce precise skeletons using different types of data. Our research acitivity is based on the use of depth image, i.e. depth-maps, to exploit the depth values, i.e. the distance from the camera, to obtain accurate 3D skeletons in the camera coordinate system.

---

Human Pose Estimation

We studied how to exploit the depth information to improve the human pose estimation and to obtain 3D skeletons (using the depth information and the camera parameters).
The main competitor for this task is the Shotton algorithm, which is implemented in the well-known device Microsoft Kinect. This algorithm is able to obtain body pose estimation, only on some strictly-constrained situations, like fixed frontal position of the subject with respect to the camera. Thus, we adapted neural network architectures developed for the intensity domain and developed a method which is able to obtain body joints position starting from depth frames only.
Moreover, we manually annotated a subset of the widely-used dataset Watch-N-Patch in order to correctly assess the quality of our approach compared to the traditional Shotton algorithm. The manually-refined annotations are available as the Watch-R-Patch dataset.

*Code of the Human Pose Annotation Tool used to collect Watch-R-Patch, the manually-refined version of a subset of Watch-N-Patch.

---

3D Human Pose Refinement

We proceed studying how depth information can be exploited, represeting it as depth maps, surface normals, voxels and point clouds. We developed a new architecture, namely RefiNet, which is able to exploit such 3D data to refine a 2D skeleton input and estimate an accurate 3D human pose. While the previous work addresses the feasibility of obtaining a pose from depth maps only, the aim of this project is to obtain a precise 3D body pose, where all joints are correctly located inside the body of the subject. This is crucial step in order to use the skeleton data to subsequent tasks, for instance the estimation of precise human measurements like the anthropometric ones.

---

Anthropometric Measurement Estimation

Depth sensors provides real world 3D coordinates that we exploited, along with an estimated human pose, to estimate limbs length, subject height and other meaningful anthropometric measurements.

We acquired a dataset, namely Baracca, which contains multiple anthropometric measures and soft-biometric traits for 30 subjects. We then developed and evaluated some geometrical, machine learning, and deep learning-based baselines that predict such values starting from an image or an estimated 2D or 3D human pose.

Publications

1	D'Eusanio, Andrea; Pini, Stefano; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "RefiNet: 3D Human Pose Refinement with Depth Maps" Proceedings of the 25th International Conference of Pattern Recognition, Milan, pp. 2320 -2327 , 10-15 January 2021, 2021 | DOI: 10.1109/ICPR48806.2021.9412451 Conference
2	Pini, Stefano; D'Eusanio, Andrea; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "Baracca: a Multimodal Dataset for Anthropometric Measurements in Automotive" Proceedings of the International Joint Conference on Biometrics (IJCB), Houston, September 28 - October 1, 2020, 2020 | DOI: 10.1109/IJCB48548.2020.9304903 Conference
3	D'Eusanio, Andrea; Pini, Stefano; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "Manual Annotations on Depth Maps for Human Pose Estimation" Proceedings of the 20th International Conference on Image Analysis and Processing, Trento, Italia, 9-13 September 2019, 2019 | DOI: 10.1007/978-3-030-30642-7_21 Conference