End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization

Abstract: Here we introduce an approximated differentiable renderer to refine a 6-DoF pose prediction using only 2D alignment information. To this end, a two-branched convolutional encoder network is employed to jointly estimate the object class and its 6-DoF pose in the scene. We then propose a new formulation of an approximated differentiable renderer to re-project the 3D object on the image according to its predicted pose; in this way the alignment error between the observed and the re-projected object silhouette can be measured. Since the renderer is differentiable, it is possible to back-propagate through it to correct the estimated pose at test time in an online learning fashion. Eventually we show how to leverage the classification branch to profitably re-project a representative model of the predicted class (i.e. a medoid) instead. Each object in the scene is processed independently and novel viewpoints in which both objects arrangement and mutual pose are preserved can be rendered. Differentiable renderer code is available at:https://github.com/ndrplz/tensorflow-mesh-renderer.

Citation:

Palazzi, Andrea; Bergamini, Luca; Calderara, Simone; Cucchiara, Rita "End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization" Proceedings of European Conference on Computer Vision Workshops, vol. 11131, Munich, Germany, pp. 702 -715 , 8 - 14 September 2018, 2019 DOI: 10.1007/978-3-030-11015-4_53

 not available

Paper download:

• Author version:
• DOI: 10.1007/978-3-030-11015-4_53