FREE SUrveillance in a pRivacy respectFul way

Free surf is meant to be a paradigm for the new generation of video surveillance systems, free from controls by human operators, and completely respectful of the privacy. The technological support will be given by emerging solutions of computer engineering both in system architectures for real-time video processing and in innovative techniques of Computer Vision and Pattern Recognition. The majority of the commercial systems only focus on video acquisition and on their visualization in control rooms. The most innovative systems already exploit simple computer vision techniques for motion detection, but with many structural and technological constraints, such as the installation of fixed cameras only, with manual calibration, very simple target models, and, in particular, the lack of inferential capabilities and scene understanding that makes necessary the constant presence of human operators. The "free surveillance" systems aim at overcome these technological constraints, by creating new automatic systems also socially acceptable, since they will be perfectly coherent with the current laws on privacy.

This project aims at developing innovative solutions for detecting people in an automatic way by processing videos in real-time. Original and robust techniques will be applied to installations free from structural constraints, and, in particular, to multicamera distributed systems (with fixed, PTZ and mobile cameras) coordinated with sensors networks. The visual data about people will be used to recognize situations modelled with statistical and machine learning techniques in order to generate alarms in the case of dangerous situations. Information retrieval techniques based on visual content will be employed to detect occurrences of the same people or similar situations in videos acquired in different time or different cameras. This will provide effective tools for the support to investigations and posterity analysis. Face detection techniques will be used to extract and remove biometric data from the videos in real time. By doing this, these systems could be also used to transmit on the web information on the environment without transmitting data protected by privacy laws. They could be installed in public places in a socially-acceptable way, without the "Big Brother" syndrome.

These extermely-innovative proposals will not lead only on theoretical research activity, but also to the concrete development of working prototypes in the two years of the project. This is guaranteed by the composition of the group: the three RUs are involved in computer vision and pattern recognition researches since many years, and they are leading groups in the national and international context, and have active collaborations with the most important international research centres. The project has a strong national relevance because it will allow to develop also in Italy, in a coordinated and synergic way, research on computer vision, research that must be done with a sufficient number of people involved to maintain international excellence. Additionally, the created consortium also includes extra-university groups that work on this field, such as the ITC-iRST of Trento, that, even if not funded, are interested in collaborating to the project and, even more important, the project has the support and the interest of many companies and public entities that want to exploit directly the results of such activity. The project has strong implications of innovation and social impact, and can provide an immediate technological transfer to the companies that already collaborate with the RUs of the project or that have awaked their interest for new solutions to automatic video surveillance.

Partners in FreeSURF

The Goals in FreeSURF

The FREE SURF project aims at proposing new technologies for the next generations of video surveillance systems oriented to the automatic real-time control of the presence and actions undertaken by people in the environment, without the direct control of a human operator. The FREE SURF project is born with a twofold aim: first, innovative scientific research in the field of Computer Vision and Pattern Recognition, second, innovative applied research for the development of new generations of video surveillance systems, both effective and socially acceptable with respect to privacy concerns.

The first objective is to conduct a thoughtful research activity in the field of Computer Engineering for video surveillance of people in "structural constraint FREE" systems, that is in systems free from structural and environmental constraints. The automatic visual control of human presence and actions in a given environment is, indeed, one of the most studied problems in the last decade. Nowadays, a very large literature exists, which presents algorithms and robust implementations for the recognition of single persons, in structured environments: closed environments with controlled illumination, open environments with large field of view (in order to consider people as small rigid moving objects), with few people, with only partially occluded fields of view, controlled by fixed cameras (to segment objects as different from the background), and installed with a precise manual calibration (for an exact 3D reconstruction).

The final objective of the project is to study innovative methodologies and techniques for going further on: the final targets are environments free from structural constraints, in scenes with more people that live together and interact each other, as in parks or tourist areas. The foreseen activities are devoted to the study of new ways to extract visual data, from distributed camera systems, from hybrid systems with active cameras, capable to automatically move toward a target, from moving cameras, and coordinated with networks of sensors. New algorithms will be studied and working prototypes developed for people segmentation and tracking in videos acquired by multiple auto-calibrated cameras, by exploiting geometrical information and appearance (color and texture). Approaches for active camera control and mosaicing of the scene from moving cameras will be studied. Moreover, mobile agents systems will be studied to coordinate cameras and sensor networks in large scenes like archaeological sites. These techniques will all implemented in separated modules by each RU, but they will be coordinated in a single architecture to provide a common interface for the reasoning modules.

All the previous modules have the common objective to extract visual data on the people in the scene. In particular, trajectory computation with invariants independent of the point of view, people posture analysis and soft biometries are the main data that will be extracted. Differently from projects dealing with biometric analysis, the FREE SURF project is oriented to the automatic visual analysis of the presence and behavior of people independently of their identities, which are not easy to assess in noisy, low-resolution videos with large filed of view, like those typical of distributed video surveillance systems. As a further support, hybrid system with PTZ and mobile cameras can provide, if needed, information with more details, which can be used in "posterity logging" by the experts. The visual data are provided to modules for dual activities: to monitor dangerous situations in real time, and to annotate interesting situations for future off-line queries. The first is a strategic tool to help the human operator in the prevention and fast responsiveness to facts regarding security, the second provides a valid support to investigations and a-posteriori analysis. These solutions may enable the many existing surveillance systems to provide effective support in a predictive way.

The second objective of the project is to define new solutions to answer to the social requirements of safety and security both for places and people with solutions made "in a privacy-respectful way". In Italy, as well as in the whole world, cameras are spreading everywhere. This is not a social problem by itself if the cameras were only intelligent sensors, such as fire sensors, capable to simply process videos and generate alarms. The social awkwardness comes from the fact that behind a camera there is a human operator that controls and watches, prejudicing the individual's privacy. While it is acceptable to use cameras by the public officers for security reasons in public places, the diffusion of private closed-circuit TV systems in department stores, shopping centers or even the diffusion of web-accessible webcams in public tourist places are not. The current laws on privacy are (rightly) very restrictive, but currently there is no technological mean to guarantee that such laws are respected: many installed systems, without video processing, are either not compliant with these laws or simply unusable and used only as a deterrent for criminality since they declare to have a such low resolution or a such deep field of view to prevent people identification. The aim of this project is, instead, to provide automatic tools for extracting visual information on people that can be, on the one side, used by authorized systems to automatically monitor the scene, and, on the other side, used to guarantee the removal of biometric data and, thus, the compatibility with privacy laws. In particular, robust algorithms for people tracking and face and head detection will be useful to obscure biometric data in real time.

The reported objectives of the project are for sure ambitious, but they are also feasible in the timeframe of the project. Besides the limited funding requests to MIUR, the RUs have made available many man-months of people with already a great experience on the field and can benefit of the resources, both hardware and software, in their laboratories. The consortium is composed of three very active RUs at national level, in particular in the GIRPR (www.girpr.it): all the three RUs have organized both the International Conf. of GIRPR (ICIAP), and the school for PhD students (VISMAC) that has been/will be held in Firenze, Palermo and Modena, in year 1999, 2001 and 2007 (ICIAP) and 1992, 2006 and 2000 (VISMAC). They are, moreover, very active at international level, with frequent PhD students and researchers with research centers in USA and other countries (such as the University of Technology, Sydney, Australia), and they have many active projects in the field.

A very important aspect of the project is that it already contributes to the creation of a network of expressions of interest and collaborations. In particular, ITC-iRST of Trento, one of the most important non-university research centers in video surveillance, will participate to the project; big national companies such as SIRTI and ALCATEL have indicated their interest. Public entities such as Regione Emilia-Romagna and Parco Valle dei Templi di Agrigento have declared their interest in the project, as final users.

Related Works of the research groups

• Moses, Mobile Streaming for Video Surveillance.
  • Demo on Moses Mobility.
  • Demo on Moses Low Latency.
  • Demo on Moses Remote Video Surveillance.
• Bagdanov, A. D., Del Bimbo, A., Nunziati, W., and Pernici, F., "Learning Foveal Sensing Strategies in Unconstrained Surveillance Environments". In Proceedings of the IEEE international Conference on Video and Signal Based Surveillance (November 22 - 24, 2006). AVSS. IEEE Computer Society, Washington, DC, 40.
• G. Gualdi, R. Cucchiara, A. Prati, "Low-latency Live Video Streaming over Low-Capacity Networks" in Proceedings of the 8th IEEE International Symposium on Multimedia (ISM2006), San Diego, CA - USA, Dec. 11-13, 2006
• A. Del Bimbo, F. Pernici, "Towards On-Line Saccade Planning for High-Resolution Image Sensing". Pattern Recogn. Lett. 27, 15 (Nov. 2006), 1826-1834
• Nunziati, W. Alon, J. Sclaroff, S. Del Bimbo, A., "View Registration Using Interesting Segments of Planar Trajectories". Advanced Video and Signal Based Surveillance, 2005. AVSS 2005. IEEE Conference on, Publication Date: 15-16 Sept. 2005, On page(s): 75- 80