OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data

Abstract: The inexorable growth of online shopping and e-commerce demands scalable and robust machine learning-based solutions to accommodate customer requirements. In the context of automatic tagging classification and multimodal retrieval, prior works either defined a low generalizable supervised learning approach or more reusable CLIP-based techniques while, however, training on closed source data. In this work, we propose OpenFashionCLIP, a vision-and-language contrastive learning method that only adopts open-source fashion data stemming from diverse domains, and characterized by varying degrees of specificity. Our approach is extensively validated across several tasks and benchmarks, and experimental results highlight a significant out-of-domain generalization capability and consistent improvements over state-of-the-art methods both in terms of accuracy and recall. Source code and trained models are publicly available at: https://github.com/aimagelab/open-fashion-clip.

Citation:

Cartella, Giuseppe; Baldrati, Alberto; Morelli, Davide; Cornia, Marcella; Bertini, Marco; Cucchiara, Rita "OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data" Proceedings of the 22nd International Conference on Image Analysis and Processing, vol. 14233, Udine, Italy, pp. 245 -256 , September 11-15, 2023, 2023 DOI: 10.1007/978-3-031-43148-7_21

 not available

Paper download:

• Author version:
• DOI: 10.1007/978-3-031-43148-7_21