Detection of Areas of Interest in Artworks using Saliency Methods

Human visual attention is explored to find semantically important elementsfor audio descriptions designed for an inclusive public with visually impairedand sighted people. The purpose of the research is to associate audiodescription with tactile exploration via a force feedback tablet (F2T) showingthe trajectory of exploration. To detect areas of interest, we used an Eyetrackerto analyze the behavior of 48 volunteer participants who observedimages of Bayeux Tapestry. The scenes were presented with or without shortaudio descriptions. A small dataset of the images of the Tapestry is preparedto train a deep learning model. Three models are used to predict the saliencyin the images of Bayeux Tapestry: Resnet50, TransalNet and SAM-LSTMResnet.After training on our dataset, we observe that the closest results to theground truth are with SAM-LSTM-Resnet. This model gives bettercorrelation with the areas of saliency and the location of observer’s attention.Three other models are compared for scan path prediction: a bio-inspiredmodel called G-Eymol, SPGen, and a deep reinforcement model generalizedfor a free-viewing task. The deep reinforcement model generalizes better onthe Bayeux Tapestry dataset. The results are presented and discussed.