Unimore logo AImageLab
Back to the project list

CultMEDIA

Reuse of existing high quality digital material and storytelling are important facilities in cultural heritage industry, respectively as a new business model in participatory production of cultural multimedia material and to boost diffusion of knowledge through new qualitative products at low production cost. CultMEDIA project aims at facilitating the definition of transmedia storytelling and at the same time to optimize costs and complexity of cultural media production by exploiting the new advances in machine learning and content-based automatic annotation and search to appropriately harvest large multimedia archives, looking for high quality and suited material, to produce new high quality media. The project focuses on video, as basic element of any transmedia experience and intends to explore new ways of producing video of multimedia storytelling.

In terms of innovation, the project will provide a disruptive improvement in the processes and services related to the cultural heritage content production, through the extended usage of Artificial Intelligence and Machine Learning algorithms as well as vision and audio captioning processing. They will effectively support the process of video and storytelling production, supporting reuse of existing material, creation of new graphic components, development of stories and ultimately providing large cost savings. In particular providing tools to mix 3D virtual clips with real data automatically; extracting knowledge from visual and auditory data understanding their content automatically so to extract meaningful clips for the purpose of the new document; capturing sentiment and mood of text, visual and auditory data so to select components that are coherent with the mood expected in the final production. All these techniques have never been developed to support new productions earlier.

The cultural and creative industries (CCIs) are important sectors that employs in 2016 6.7 million people in Europe, around the 3.5% of EU product and services. It is in a constant grows in Europe and Italy too. In Italy, this occupational rate has chance for a dramatic improvement. As Italy has the largest collection of cultural heritage in the world, CCIs seems limited in its growing capacity and potential and it looks no not uniformly distributed among the country (see picture). From the ISTAT 2015, the joint presence of significant CCIs and relevant exploitable cultural heritage are collapsed in only the 18.2% of Italian municipalities, while in 25% there is a mere concentration of CCIs only and in another 49.7% there is a potential not economically exploited cultural heritage. All these statements underline the urgency of a national initiative as CultMEDIA, which could collect and distributed economic best practices, new technology solutions and services. As the platform will allow a cheaper way to produce content for cultural heritage, and considering that the final users are expected to be content media producers and designers too, it is definitely expected a significant occupational growth in the domain.



Partners:

  • Università degli Studi Suor Orsola Benincasa (UNISOB)
  • Università degli Studi di Modena e Reggio Emilia (UNIMORE)
  • Università degli Studi di Firenze (UNIFI)
  • Consiglio Nazionale delle Ricerche (CNR)
  • Progetti di Impresa SRL a socio unico (PdI)
  • ETT S.p.A. (ETT)

Materials:

  • Kick-off (PDF)
  • Data collection and annotation pipeline (PDF)

Publications

1 Carraggi, Angelo; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita "Visual-Semantic Alignment Across Domains Using a Semi-Supervised Approach" Computer Vision – ECCV 2018 Workshops, vol. 11134, Munich, Germany, pp. 625 -640 , 8-14 September 2018, 2019 | DOI: 10.1007/978-3-030-11024-6_47 Conference
2 Tomei, Matteo; Baraldi, Lorenzo; Cornia, Marcella; Cucchiara, Rita "What was Monet seeing while painting? Translating artworks to photo-realistic images" Computer Vision – ECCV 2018 Workshops, Munich, Germany, 8-14 September 2018, 2019 | DOI: 10.1007/978-3-030-11012-3_46 Conference
3 Baraldi, Lorenzo; Cornia, Marcella; Grana, Costantino; Cucchiara, Rita "Aligning Text and Document Illustrations: towards Visually Explainable Digital Humanities" Proceedings of the 24th International Conference on Pattern Recognition, Beijing, China, pp. 1097 -1102 , August 20th-24th, 2018, 2018 | DOI: 10.1109/ICPR.2018.8545064 Conference
4 Cornia, Marcella; Baraldi, Lorenzo; Serra, Giuseppe; Cucchiara, Rita "SAM: Pushing the Limits of Saliency Prediction Models" 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, pp. 1971 -1973 , June 18-22 2018, 2018 | DOI: 10.1109/CVPRW.2018.00250 Conference
5 Cornia, Marcella; Baraldi, Lorenzo; Serra, Giuseppe; Cucchiara, Rita "Visual Saliency for Image Captioning in New Multimedia Services" Multimedia & Expo Workshops (ICMEW), 2017 IEEE International Conference on, Hong Kong, pp. 309 -314 , July 10-14, 2017, 2017 | DOI: 10.1109/ICMEW.2017.8026277 Conference
6 Cornia, Marcella; Abati, Davide; Baraldi, Lorenzo; Palazzi, Andrea; Calderara, Simone; Cucchiara, Rita "Attentive Models in Vision: Computing Saliency Maps in the Deep Learning Era" AI*IA 2017 Advances in Artificial Intelligence, vol. 10640, Bari, Italy, pp. 387 -399 , November 14-17, 2017, 2017 | DOI: 10.1007/978-3-319-70169-1_29 Conference

Project Info

Staff:

Duration:

01/12/2017 -

Funded by:

MIUR

Project type:

Cluster Nazionale