System for bin picking using 3D and 2D video feeds

Research activity in the development of a complete robotic system for the picking of articulated objects from bins. Objects are mostly transparent (flowpacks) and overlapped each other. Segmentation and pick-position calculation will be based on 2D standard camera segmentation joint with 3D sensors for identifying the top objects in the bin.

This 2-year project had the objective of developing a complete system based on computer vision to guide a robot for picking objects at random disposal on bins or conveyor belts. The specific challenges include the transparency of typical objects (from the pharmaceutical and cosmetic fields), their mutual occlusions and the highly-demanding time constraints. All these challenges called for a specific solution based on an innovative SIFT-based detection of multiple instances of objects, through the use of object models from different faces of the objects and in different illumination conditions. The matching of SIFT descriptors between the current image and the object models is performed on a GPU to improve the speed and then used with a new clustering technique which allows to detect objects also if partially visible (due to both occlusions and reflexes due to transparency of flowpacks).