European Training Network on PErsonalized Robotics as SErvice Oriented applications

Personal robots are used in domestic and public environments to perform interaction tasks in support and collaboration with people. However, the foreseen market success of such a technology is still far to be realized. There are several research challenges for personal robotics, which are mainly related to the need of a high degree of personalization of the robot behavior with respect to the specific user’s needs and preferences. The user’s experience of personalized and adaptive interaction with the robot is key to its large-scale adoption. Companies need to build solutions rooted in a deeper analysis of humans’ specificities before developing products for people. Policymakers need to base their regulations on the view of possible long-term interaction, and not only on possible contingent needs. At the same time, academia needs to nurture the development of an extended research community with a set of interdisciplinary skills to investigate different robot’s capabilities for understanding and modeling the interaction with human beings, for decision mechanisms to appropriately adapt the robot’s behavior to the context, and software integration mechanisms that allow an easy, personalized configuration approach to limit the hard and costly processes for customizing novel robotic system.

ABOUT PERSEO

The European Training Network on PErsonalized Robotics as SErvice Oriented applications – PERSEO – aims at training and creating a new generation of interdisciplinary researchers and professionals with an entrepreneurship ethos and transferrable skillset to face both societal and technological research challenges of this forthcoming market of personal robots. These consist respectively in the development of personalized and acceptable robot’s perceptual and interaction capabilities to be distributed as Cloud Services personal robot applications as companions, in rehabilitation, and for edutainment. The PERSEO research and training program is organized into three Research Themes aimed at investigating personalization of robot capabilities at different levels of possible Human-Robot Interaction (HRI), namely “Physical, “Cognitive”, and “Social”. This requires a set of research skills ranging from computer science and AI to automation, ethics, and psychology, and the use of a collaborative design approach. Integration Milestones, will provide co-working societal problems where collaborative design/implementation is fostered and supported by partner organizations.

ESR at UNIMORE

• Early-Stage Researcher (ESR) n. 7 - Integration of vision and language for human-robot interaction

  OBJECTIVES: To develop new approaches that bridge together perception, language and action in robotic scenarios, fostering a natural interaction between humans and robots. Objectives of the ESR include:

  - the development of language and vision-based navigation algorithms, in which a mobile agent is trained to perform actions or reach a target destination via natural language instructions;

  - the investigation of solutions for interacting with robotic agents in natural language, by endowing the robot with the capability of describing its current state, and understanding inputs in natural language;

  - the training of navigation and interaction algorithms on simulated environments and their deployment on real robots.

  EXPECTED RESULTS: State-of-the-art algorithms for navigation and visual-semantic tasks which can bring the interaction between human and robots feasible in natural language, and which can effectively connect vision and language on robotic systems. Novel and state of the art approaches for real-time HRI in natural language, with a specific focus on semantically challenging domains. Deployment of such algorithms on real robots.

• Early-Stage Researcher (ESR) n. 2 - Object and People Recognition for Personalized Interaction

OBJECTIVES: The aim of the project is to research new algorithms for people recognition, that effectively scale to real-world data. This project will research and design novel algorithms for human body pose estimation during interactions by effectively mix convolutional networks for body part location with regression network for 3D pose estimation in conjunction with differentiable rendering techniques capable of projecting parametric body model on the scene and optimizing the pose estimation process through plain back-propagation. The objective will be to provide real time pose estimation algorithms and exploiting the extracted 2D and 3D body poses for classifying and eventually predicting actions. A possible research line will be the exploitation for the latter task of generative recurrent models trained for both action classification and intention prediction. This task has been effectively applied to pedestrian trajectories, but a very limited number of pioneering works deal with complex full-body human actions and are limited to surveillance scenarios.

EXPECTED RESULTS: State of the art algorithms for real time body pose estimation that can be used by the robot for planning and deciding the proper strategy for the HRI. Generative models for action classification and intention prediction in order to potentially generate multiple futures and provide the robots several hypotheses on which the planning and control can be based. In synergy with concurrent European projects (MOVECARE, WISER), we will test the recognition of typical home objects (glasses, remote controllers, keys, etc.) which exhibit different point of manipulation and interaction modalities.

Recruitment/research points of contact

ESR 2

Rita Cucchiara: rita.cucchiara@unimore.it

Simone Calderara: simone.calderara@unimore.it

ESR 7

Rita Cucchiara: rita.cucchiara@unimore.it

Lorenzo Baraldi: lorenzo.baraldi@unimore.it

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 955778.

​

BENEFICIARIES

UNIVERSITÀ DEGLI STUDI DI NAPOLI FEDERICO II (UNINA) ICAROS Center

Principal Investigator: Silvia Rossi, Key Person: Francesco Cutugno, Project Manager: Mariacarla Staffa;

UNIVERSITY OF MANCHESTER (UNIMAN)​

Principal Investigator: Angelo Cangelosi, Key Person: Daniel Hernadez, Gi Hyun Lim;

SHEFFIELD HALLAM UNIVERSITY (SHU)

Principal Investigator: Alessandro Di Nuovo, Key Person: Daniela Conti, Sergio Davies, Richard Moore;

TECHNISCHE UNIVERISTAT MUNCHEN (TUM)

Principal Investigator: Dongheui Lee, Key Person: Alejandro Agostini;

UNIVERSITY OF MODENA AND REGGIO EMILIA (UNIMORE)

Principal Investigator: Rita Cucchiara, Key Person: Simone Calderara, Lorenzo Baraldi;

UNIVERSITY OF VIENNA (UNIVIE) 

Principal Investigator: Mark Coeckelbergh, Key Person: Peter Rantasa;

UNIVERSITAT BIELEFELD (UNIBI)

Principal Investigator: Friederike Eyssel, Key Person: Ricarda Wullenkord;

SCUOLA SUPERIORE SANT’ANNA (SSSA) 

Principal Investigator: Andrea Bertolini, Key Person: Erica Palmerini;

PAL ROBOTICS (PAL) 

Principal Investigator: Francesco Ferro, Key Person: Jordi Pagès, Alessandro Di Fava;

ORTELIO LTD (ORTELIO) 

Principal Investigator: Stratos Arampatzis, Key Person: Sascha Griffiths.

PARTNERS

IBM UNITED KINGDOM LTD (IBM), United Kingdom;

OMITECH S.R.L.(OMITECH), Italy;

FUNDING BOX RESEARCH APS (FBR), Denmark;

CONSIGLIO NAZIONALE DELLE RICERCHE (CNR), Italy;

UNIVERSITY OF TWENTE (UT), Netherlands;

UNIVERSITÀ DELLA CAMPANIA VANVITELLI (VANV), Italy;

ASSOCIAZIONE NOVILUNIO ONLUS (NOVILUNIO), Italy;

VILLA DELLE GINESTRE (VDG), Italy;

HACKIDEMIA, USA.

Publications

1	Bigazzi, Roberto; Baraldi, Lorenzo; Kousik, Shreyas; Cucchiara, Rita; Pavone, Marco "Mapping High-level Semantic Regions in Indoor Environments without Object Recognition" Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), vol. 2024, Yokohama, pp. 7686 -7693 , May 13th-17th, 2024, 2024 | DOI: 10.1109/ICRA57147.2024.10610897 Conference
2	Rawal, Niyati; Bigazzi, Roberto; Baraldi, Lorenzo; Cucchiara, Rita "AIGeN: An Adversarial Approach for Instruction Generation in VLN" Proceedings of the IEEE/CVF Conference on Computer Vision Workshops, Seattle, 17th-21st June, 2024 Conference
3	Poppi, Samuele; Rawal, Niyati; Bigazzi, Roberto; Cornia, Marcella; Cascianelli, Silvia; Baraldi, Lorenzo; Cucchiara, Rita "Towards Explainable Navigation and Recounting" IMAGE ANALYSIS AND PROCESSING, ICIAP 2023, PT I, vol. 14233, Udine, Italy, pp. 171 -183 , September 11-15, 2023, 2023 | DOI: 10.1007/978-3-031-43148-7_15 Conference