Face Recognition in News Streams

The goal of the project is to identify the foreign ministers of the G7/G20 group into the images of the daily news. The problem has been divided in two parts: on the one hand the daily main news and the relative images retrieval, on the other hand the face detection and the face recognition.

The GDELT project is used to retrieve the links of the main news of every day in the form of a daily CSV file (this file has between 125.000-250.000 rows with not-unique links in the days of the test). Then, every image from every page bigger than a threshold is downloaded and stored by the system. Wrong links and unavailable images are ignored. In the first tests of the system, only a subset of the links are used.

Then, a face detection algorithm based on a Multi-task Cascaded CNN runs over all the downloaded images to find and save faces bigger than a threshold.

The last step uses an open-source NN code based on the structure of FaceNet and pre-trained with the MS-Celeb-1M dataset. This network takes as input an image and returns a 128-d embedded vector that is the image representation in a Euclidean space of 128 dimensions. In this space images of the same person are near while images of different people are far so it is possible to find if two images represents the same person evaluating the Euclidean distance between the two images representations. Any clustering algorithm can be used in the resulting space to find same or similar people.
The first implementation of this project uses a trivial threshold on the distance measure to evaluate if two images refer to the same person.

A training phase is needed to find the region in the space where the face of every foreign minister lies. For every minister, his images are downloaded from Google Images by a crawler and filtered by a human, then his faces are extracted and the embedded representation is calculated for every extracted face. These resulting embedded vectors are used as comparison data for the unknown faces.

During the recognition phase, the images of faces saved in the second step are given as input to the FaceNet-like network and the resulting 128-d embedded vectors are compared with the embeddings of the foreign ministers previously calculated. If the mean of the distances between an unknown face and the faces of a minister is lower than a threshold, the face is recognized as the face of that minister.

Demo interface

A preliminary demo interface has been publicly released. Check it out!