Artwork recommendations guided by foundation models: survey and novel approach
Description
This repository contains the materials presented in the journal paper 'Artwork recommendations guided by foundation models: survey and novel approach'. The code of notebook 'Retrieve_combined_features.ipynb' illustrates the combination of recommendations with the same weights of genre, style and artist criteria, it can be easily modified for using with different weights. The code for fine-tuning a classification model is based on the example.
Tetiana Yemelianenko, Iuliia Tkachenko, Tess Masclef, Mihaela Scuturici, Serge Miguet
Table of content
Dataset
The WikiArt dataset used for fine-tuning could be found here
Steps
To reproduce the steps first you need to finetune models for genre, style and artist classification on WikiArt dataset or used fine-tuned adapters. Then, using fine-tuned models you need to calculate embeddings for the images from WikiArt dataset and create ANNOY indexes. These files are available upon a request.
Citation
@InProceedings{Yemelianenko_2024_MTA,
author = {Yemelianenko, Tetiana and Tkachenko, Iuliia and Masclef, Tess and Scuturici, Mihaela and Miguet, Serge},
title = {Artwork recommendations guided by foundation models: survey and novel approach},
booktitle = {},
month = {},
year = {},
pages = {}
}Acknowledgments
This work was funded by french national research agency with grant ANR-20-CE38-0017. We would like to thank the PAUSE ANR-Program: Ukrainian scientists support to support the scientific stay of T. Yemelianenko in LIRIS laboratory.
