Future Artificial Intelligence Research

The Future Artificial Intelligence Research (FAIR) project aims to help address the research questions, methodologies, models, technologies, and even ethical and legal rules for building Artificial Intelligence systems capable of interacting and collaborating with humans, perceiving and acting within changing contexts, being aware of their limitations and able to adapt to new situations, being aware of the perimeters of safety and trust, and being mindful of the environmental and social impacts their implementation and execution may entail. Research activities are carried out within 10 spokes that will involve 350 + researchers. Each spoke is characterized by a specific thematic area and its own set of research challenges with the aim of addressing FAIR challenges from different vantage points. The spokes are: Human-centred AI, Integrative AI, Resilient AI, Adaptive AI, High-quality AI, Symbiotic AI, Edge-exascale AI, Pervasive AI, Green-aware AI, Sustainable Bio-socio-cognitive AI.

While these 10 themes will be the topics of 10 large spoke-projects, a number of fundamental scientific and technological challenges will be tackled with coordinated interspoke actions, called transversal projects - TP, where each involved spoke will contribute from its own specific perspective:

• (TP1) Legal and Ethical Design of Trustworthy AI Systems: how to create responsible, trustworthy AI “bydesign”, “in-design” and “for designers”;
• (TP2) Vision, Language and Multimodal Challenges: how to create AI agents capable of perception in real, complex environments with multiple combined modalities (text, speech, images, video, …);
• (TP3) Learning and Reasoning from Individual to Communities to Society: how to create AI agents that integrate learning and reasoning to assist decision making at multiple scales (individual, societal);
• (TP4) Adjustable Autonomy and Physical Embodied Intelligence: how to create autonomous AI systems capable to understand the limits of their autonomy, asking for human supervision when appropriate;
• (TP5, TP6) Frontiers of Machine Learning: how to gear the methods of mathematics and physics to understand why and when machine learning works, and how to expand the frontiers of “lifelong”, continual, incremental learning and meta learning (learning to learn);
• (TP7) Data Centric AI and Infrastructures: how to manage, prepare and curate large, highquality data for AI development.

Spokes act as catalysts for both the development of innovative AI technologies and new AI services in strategic sectors for the country system, involving the industrial sector, both at the level of large companies and innovative small and medium-sized enterprises.

AImageLab is involved in the Spoke 8 of the FAIR project working actively on a subproject relying PERVASIVE-AI. In addition, AImageLab is actively engaged on the trasversal project on VISION, LANGUAGE and MULTIMODAL Challenges.

The notion of Pervasive AI is not only about intelligent objects, but refers to the possibility of introducing intelligent models and algorithms into the (socio-technical) contexts in which we live: from work to cities, from policy decisions to public services, from business processes to the production of artifacts in industry and the arts. The notion of pervasiveness implies that we need to model, predict, and decide on large-scale and complex socio-technical systems in which each component is itself intelligent, possibly self-interested, and we need to take into account artificial infrastructures, natural phenomena, economic aspects, environmental impacts, social dynamics, and human behavior. To make AI pervasive we need to holistically address challenges related to robustness and accuracy of models, efficiency of implementations, cultural acceptability, and sustainability of AI: thus algorithmic and modeling, scientific and technological, educational, legal, socio-economic, ethical, and cultural challenges emerge in this context.

In conclusion, the FAIR approach follows a holistic, multidisciplinary approach, aimed at a profound rethinking of the foundations of AI, that goes hand in hand with investigating the social impact of the new forms of AI. Depending on the course that the AI revolution takes, AI will either empower our ability to make more informed choices or reduce human agency; expand the human experience or replace it; create new forms of human activity or reduce jobs; help distribute well-being for many or increase the concentration of power and wealth in the hands of a few; expand or endanger democracy in our societies; help to fight the climate change or increase emissions.

Publications

1	Barsellotti, Luca; Amoroso, Roberto; Baraldi, Lorenzo; Cucchiara, Rita "FOSSIL: Free Open-Vocabulary Semantic Segmentation through Synthetic References Retrieval" Proceedings - 2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024, Waikoloa, Hawaii, pp. 1453 -1462 , Jan 4-8, 2024, 2024 | DOI: 10.1109/WACV57701.2024.00149 Conference
2	Bernhard, Maximilian; Amoroso, Roberto; Kindermann, Yannic; Baraldi, Lorenzo; Cucchiara, Rita; Tresp, Volker; Schubert, Matthias "What’s Outside the Intersection? Fine-grained Error Analysis for Semantic Segmentation Beyond IoU" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii, Jan 4-8, 2024 Conference
3	Cornia, Marcella; Baraldi, Lorenzo; Fiameni, Giuseppe; Cucchiara, Rita "Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets" INTERNATIONAL JOURNAL OF COMPUTER VISION, vol. 132, pp. 1701 -1720 , 2024 | DOI: 10.1007/s11263-023-01949-w Journal
4	Barsellotti, Luca; Amoroso, Roberto; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation" Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 17th-21st June, 2024 Conference
5	Moratelli, Nicholas; Barraco, Manuele; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Are Learnable Prompts the Right Way of Prompting? Adapting Vision-and-Language Models with Memory Optimization" IEEE INTELLIGENT SYSTEMS, vol. 39, pp. 26 -34 , 2024 | DOI: 10.1109/MIS.2024.3386099 Journal
6	Caffagni, Davide; Cocchi, Federico; Moratelli, Nicholas; Sarto, Sara; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, Jun 17-21, 2024 Conference
7	Caffagni, Davide; Cocchi, Federico; Barsellotti, Luca; Moratelli, Nicholas; Sarto, Sara; Baraldi, Lorenzo; Baraldi, Lorenzo; Cornia, Marcella; Cucchiara, Rita "The Revolution of Multimodal Large Language Models: A Survey" Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, August 11–16, 2024, 2024 Conference
8	Sarto, Sara; Barraco, Manuele; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation" Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, vol. 2023, Vancouver, can, pp. 6914 -6924 , Jun 18-22 2023, 2023 | DOI: 10.1109/CVPR52729.2023.00668 Conference
9	Cartella, Giuseppe; Baldrati, Alberto; Morelli, Davide; Cornia, Marcella; Bertini, Marco; Cucchiara, Rita "OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data" Proceedings of the 22nd International Conference on Image Analysis and Processing, vol. 14233, Udine, Italy, pp. 245 -256 , September 11-15, 2023, 2023 | DOI: 10.1007/978-3-031-43148-7_21 Conference
10	Caffagni, Davide; Barraco, Manuele; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "SynthCap: Augmenting Transformers with Synthetic Data for Image Captioning" Proceedings of the 22nd International Conference on Image Analysis and Processing, vol. 14233, Udine, Italy, pp. 112 -123 , September 11-15, 2023, 2023 | DOI: 10.1007/978-3-031-43148-7_10 Conference
11	Cocchi, Federico; Baraldi, Lorenzo; Poppi, Samuele; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Unveiling the Impact of Image Transformations on Deepfake Detection: An Experimental Analysis" IMAGE ANALYSIS AND PROCESSING, ICIAP 2023, PT II, vol. 14234, Udine, Italy, pp. 345 -356 , September 11-15, 2023, 2023 | DOI: 10.1007/978-3-031-43153-1_29 Conference
12	Barraco, Manuele; Sarto, Sara; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning" Proceedings of the IEEE International Conference on Computer Vision, ICCV 2023, Paris, France, pp. 3009 -3019 , October 2-6, 2023, 2023 | DOI: 10.1109/ICCV51070.2023.00282 Conference
13	Mancusi, Gianluca; Panariello, Aniello; Porrello, Angelo; Fabbri, Matteo; Calderara, Simone; Cucchiara, Rita "TrackFlow: Multi-Object Tracking with Normalizing Flows" Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, Paris, fra, pp. 9497 -9509 , OCT 02-06, 2023, 2023 | DOI: 10.1109/ICCV51070.2023.00874 Conference