A Transformer-Based Network for Dynamic Hand Gesture Recognition
Abstract: Transformer-based neural networks represent a successful self-attention mechanism that achieves state-of-the-art results in language understanding and sequence modeling. However, their application to visual data and, in particular, to the dynamic hand gesture recognition task has not yet been deeply investigated. In this paper, we propose a transformer-based architecture for the dynamic hand gesture recognition task. We show that the employment of a single active depth sensor, specifically the usage of depth maps and the surface normals estimated from them, achieves state-of-the-art results, overcoming all the methods available in the literature on two automotive datasets, namely NVidia Dynamic Hand Gesture and Briareo. Moreover, we test the method with other data types available with common RGB-D devices, such as infrared and color data. We also assess the performance in terms of inference time and number of parameters, showing that the proposed framework is suitable for an online in-car infotainment system.
Citation:
D'Eusanio, Andrea; Simoni, Alessandro; Pini, Stefano; Borghi, Guido; Vezzani, Roberto; Cucchiara, Rita "A Transformer-Based Network for Dynamic Hand Gesture Recognition" 2020 International Conference on 3D Vision (3DV 2020), Online, pp. 623 -632 , 25-28 November 2020, 2020 DOI: 10.1109/3DV50981.2020.00072
not available
Paper download:
- Author version:
- DOI: 10.1109/3DV50981.2020.00072