Unimore logo AImageLab

Turms Dataset

Features

  • Data: the dataset consists of 14k frames, acquired in a stereo mode, divided into sequences. The spatial resolution is 640x240. For each sequence, different actions have been performed by subjects;

  • Variety: a great variety is guaranteed due to the presence of 7 subjects (5 males, 2 females), each recorded twice;

  • Annotations: since the original SDK of Leap Motion does not work in our particular setup due to the presence of occlusions by the wheel, we manually annotated the position (x,y coordinates) of both left and right hands in each frame. We also provide the initial position of the steering wheel (5 points of the ellipse fitted in the external contour of the wheel)

To acquire a dataset in an automotive context, the choice of the most appropriate acquisition device is fundamental: an infrared camera has to been exploited, to overcome the traditional limit of RGB sensors that produce bad quality images with limited or even missing light sources.
Thanks to this particular position, we suppose to have a small number of hand occlusions produced by driver's body during driving activity.
We decided to use the Leap Motion device to acquire the Turms dataset.
Leap Motion is a device originally designed to interact with digital contents in virtual and augmented reality, based on natural body language: this device is able to track both hand and finger movements, with a low latency, converting them into 3D input. Usually it is placed at the same plane of a laptop keyboard, on the desktop, in front of the user and with its cameras pointing upwards.
We collect the Turms dataset in a real car context, placing the acquisition device at the back of the steering wheel.
Leap Motion device fits automotive requirements, for these reasons:

  • Infrared camera: infrared light allows to develop vision-based systems that are able to work in different light conditions, for example during the night or during bad weather events;

  • Stereo camera: depth or 3D information could be retrieved due to the presence of stereo images;

  • Short range: thanks to its fish-eye lens, Leap Motion is able to capture a 150 degree scene; in this way, a single frame can include the steering wheel, right and left hands and the upper part of the driver body;

  • Small size: a fundamental requirement because of the limited dimensions of the car instrument panel, for an eventual future integration in the cockpit.

  • Real time performance: high frame rate (more than 200 fps) allows to capture every fast hand motion;

Download

Click here to download a copy of the dataset. If you use this data, please cite our work.
Click here to dowload the dataset annotations [work in progress].

 

We believe in open research and we are happy if you find this data useful. If you use it, please cite our work.

@inproceedings{borghi17landmark,
title = {Hands on the wheel: a Dataset for Driver Hand Detection and Tracking},
author = {Borghi, Guido and Frigieri, Elia and Vezzani, Roberto and Cucchiara, Rita},
booktitle = {Proceedings of the 8th International Workshop on Human Behavior Understanding (HBU) },
year = {2018}
}

 

Acknowledgments

We sincerely thank all the people who participated in the experiments that led to the creation of this dataset.