Unimore logo AImageLab

Briareo Dataset

About the Dataset

Natural User interfaces in which the interaction is not carried through physical devices, are becoming more and more important in many computer vision fields, since they are extremely user-friendly and intuitive. The use of this type of technology can be an effective way to reduce and monitor driver’s inattention during the driving activity. To this end we propose a new dataset, called Briareo, specifically collected for the hand gesture recognition task in the automotive context. We focus on the acquisition of dynamic hand gestures, where each gesture is a combination of motion and one or more hand poses. Images have been collected from an innovative point of view: the acquisition devices are placed in the central tunnel between the driver and the passenger seats, orientated towards the car ceiling. In this way, visual occlusions produced by driver’s body can be mitigated. Three different sensors are used in order to acquire the dataset:

  • Traditional RGB camera
  • Time-of-Flight (ToF) depth sensor (pmdtec Pico Flexx)
  • Stereo Camera (Leap Motion)
The collected data has a great variability: a high number of subjects and gestures from different sources has been recorded.

Details

  • RGB camera
    • Traditional Camera, high availability for different resolution and form factors
    • Frame rate: 30 fps
  • ToF sensor
    • Pico Flexx
    • Resolution: 224 × 171
    • Frame rate: 45 fps
    • Form factor: 68mm × 17mm × 7.35mm - it can be easily integrated in a car cockpit!
    • Range: 0.1 - 1 meters
  • Stereo Camera
    • Leap Motion
    • Resolution: 640 × 240 raw images, 400 x 400 rectified images
    • Framerate: 200 fps
    • Form factor: 70mm × 12mm × 3mm - it can be easily integrated in a car cockpit!
The dataset is composed of 12 gesture classes:
  1. Fist
  2. Pinch
  3. Flip-over
  4. Telephone
  5. Right swipe
  6. Left swipe
  7. Top-down swipe
  8. Bottom-up swipe
  9. Thumb
  10. Index
  11. Clockwise rotation
  12. Counter-clockwise rotation
A total of 40 subjects (33 males and 7 females) have taken part to the data collection. Every subject has performed each gesture 3 times, leading to a total of 120 collected sequences. Each sequence lasts at least 40 frames. At the end of this procedure, we record an additional sequence including all hand gestures in a single recording. The three cameras have been synchronized so that the frames at a certain instant depict the same scene. The following data are released within the dataset: RGB images, depth maps and infrared intensities (Pico Flexx), raw and rectified infrared images (Leap Motion), 3D hand joints (Leap Motion SDK).

Download

To download the dataset, we require an email address where we'll send a download link to. This will help us to keep in touch in case errors are found or as updates become available.

By downloading the dataset, you declare that you will use it for research and educational purposes only, any commercial use is prohibited.



Acknowledgements

We sincerely thank all the people who participated in the experiments that led to the creation of this dataset.

Citation

We believe in open research and we are happy if you find this data useful. If you use it, please cite our paper.

@article{briareo2019,
  title={Hand Gestures for the Human-Car Interaction: the {B}riareo dataset},
  author={Manganaro, Fabio and Pini, Stefano and Borghi, Guido and Vezzani, Roberto and Cucchiara, Rita},
  journal={20th International Conference on Image Analysis and Processing (ICIAP)},
  year={2019}
}