Driver Attention through Head Localization and Pose Estimation

Automatic recognition of the driver's attention level is a problem not yet solved in research.
This project investigates new non-invasive systems for real-time monitoring of the state of attention of drivers and aims at developing a low-cost multi-sensory system that can be installed on circulating vehicles. Computer vision and machine learning techniques as well as multi-physical technologies will be explored.

Monitoring the attention of a car driver is a complex but extremely interesting problem, given the several practical applications both in the short and long term period. For example, the Italian National Institute of Statistics (Istat) has identified the driver distraction as the principal cause of road accidents in 2014.
The driver’s attention may be reduced due to risky behaviors, such as the use of electronic devices and infotainment systems while driving, or to fatigue; in these situations the monitoring of the distraction level is therefore of great impact and relevance. These studies are also behind the development of new solutions for semi-autonomous driving, a strategic goal for car manufacturers. Indeed, the development and the early introduction of co-piloting devices require studies to build accurate risk profiles.

In collaboration with the research group headed by Prof. Annalisa Bargellini (Department of Biomedical, Metabolic and Neural Sciences), we are working on the identification and characterization of the major causes of inattention. A set of corresponding physical indicators will be also selected and measured using a multisensory solution, which should meet important requirements such as ease of installation, modularity, low cost, and simplicity of configuration. Prototype solutions will be tested in a laboratory demonstrator.

In particular, head pose estimation is one of the key elements for attention monitoring and driver behavior analysis. Different weather conditions, like shining sun and clouds, in addition to sunrises, sunsets, nights that could dramatically change the quality and the visual performance of acquired images. Depth and IR cameras are mainly adopted to mitigate the impact of these issues.

Learn more on our work accepted at CVPR 2017!

PANDORA dataset (CVPR 2017)

POSEIDON framework (CVPR 2017)

POSEidonPlus framework (TPAMI 2018)

pamiheadpose

International collaborations

The research activity is carried out in collaboration with the Center for Automotive Research at Stanford (CARS).
CARS brings together researchers, students, industry, government and the community to enable a future of human-centered mobility. Understanding how people and machines work together has never been so important than when building vehicles of the future. CARS supports educational experiences for students, infrastructure for research and events that bring students and campus researchers together with industry professionals and the broader community.

Publications

1	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "Deep Head Pose Estimation from Depth Data for In-car Automotive Applications" Proceedings of the 2nd International Workshop on Understanding Human Activities through 3D Sensors, vol. 10188, Cancun (Mexico), pp. 74 -85 , Dec 4 , 2016, 2018 \| DOI: 10.1007/978-3-319-91863-1_6 Conference
2	Borghi, Guido; Venturelli, Marco; Vezzani, Roberto; Cucchiara, Rita "POSEidon: Face-from-Depth for Driver Pose Estimation" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2017-, Honolulu, Hawaii, pp. 5494 -5503 , July, 22-25, 2017, 2017 \| DOI: 10.1109/CVPR.2017.583 Conference
3	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "From Depth Data to Head Pose Estimation: a Siamese approach" Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), vol. 5, Porto, Portugal, pp. 194 -201 , 27 february - 1 march, 2017, 2017 \| DOI: 10.5220/0006104501940201 Conference
4	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "From Depth Data to Head Pose Estimation: a Siamese approach" Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), Porto, Portugal, 27 february - 1 march, 2017, 2016 Conference

Video Demo

Research Activity Info

1	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "Deep Head Pose Estimation from Depth Data for In-car Automotive Applications" Proceedings of the 2nd International Workshop on Understanding Human Activities through 3D Sensors, vol. 10188, Cancun (Mexico), pp. 74 -85 , Dec 4 , 2016, 2018 \| DOI: 10.1007/978-3-319-91863-1_6 Conference
2	Borghi, Guido; Venturelli, Marco; Vezzani, Roberto; Cucchiara, Rita "POSEidon: Face-from-Depth for Driver Pose Estimation" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2017-, Honolulu, Hawaii, pp. 5494 -5503 , July, 22-25, 2017, 2017 \| DOI: 10.1109/CVPR.2017.583 Conference
3	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "From Depth Data to Head Pose Estimation: a Siamese approach" Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), vol. 5, Porto, Portugal, pp. 194 -201 , 27 february - 1 march, 2017, 2017 \| DOI: 10.5220/0006104501940201 Conference
4	Venturelli, Marco; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "From Depth Data to Head Pose Estimation: a Siamese approach" Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), Porto, Portugal, 27 february - 1 march, 2017, 2016 Conference