FENG Haifeng

PhD graduated
Departure date : 09/30/2012

Supervision : Bernadette BOUCHON-MEUNIER

Co-supervision : LESOT Marie-Jeanne, DETYNIECKI Marcin

Studies of the relationships between colours and emotions from spontaneous annotations

This thesis aims at the identification and characterization of relationships between colors and emotions on the basis of annotated images. In particular, we are interested in the automatic discovery of relationships between the following three concepts: “objective colors” given by the chromatic content of images, “subjective colors” appearing as labels provided by human spontaneous annotation of images, and “emotions” as well expressed spontaneously by users through labels. In this study, we assemble an adapted experimental data set using the FlickR website. This online photo management and sharing application has search engine built around "social tagging", which allows photographers to annotate their images by labels of their choice. We collected approximately 22000 images annotated with at least one (subjective) color label and one emotional label, i.e. images for which a user considered, spontaneously, the emotional and chromatic content as important. First, using a tree algorithm decision, we study the relations between objective colors and emotions. We then characterize the emotions based on subjective colors, again using a decision tree algorithm, so as to allow the comparison with the results obtained in the previous study. Moreover, in order to further explore the latter characterization we use an approach based on association rules. Finally, to complement these studies, we analyze the relationships between objective and subjective colors. Characterizations of the relations between the 3 concepts are based on various chromatic representations, including a classical objective coding, a subjective coding, and an intermediate level consisting of an interpreted objective coding. For the latter, we designed, assembled and analyzed a visual dictionary based, not on discriminative hyperplane separations between colors, but on sets of representative points for each color. In terms of characterization of emotions, our approach revealed, for the subjective colors and emotions couple, not only known relationships in the psychological literature (e.g. between black and fear), but also less expected relations, for example between pink and surprise. If the aim is emotion prediction, which is difficult in nature, the subjective and interpreted coding give similar results. This means that both are of equal interest – from an application point of view – for emotion detection in images, based on color description. But from an analytic point of view, the subjective coding seems closer to the emotions, since its format is disadvantaged in the experiments carried out: not only, the prediction method is more favorable to the nature of the objective coding (numerical vs. binary coding), but also, the objective coding is richer, containing de facto all colors vs. one or two subjective colors spontaneously tagged. The relationship between objective and subjective colors is stronger than the one between colors and emotions, which is expected since these relations are more immediate. The experimental results show, however, that their identification remains a difficult problem, pointing out that the color annotations chosen by users are not in a simple relationship with the dominant color of the images or combinations of colors proportions.

Defence : 12/15/2011 - 09h30 - Site Jussieu 25-26/105

Jury members :

AKDAG Herman (LIASD - Université Paris 8) [Rapporteur]
BOUCHARD Carole (LCPI - Arts et Métiers ParisTech) [Rapporteur]
AIT YOUNES Amine (CResTIC - IUT de Reims-Châlons-Charleville)
LESOT Marie-Jeanne (LIP6-UPMC)
OMHOVER Jean-François (LCPI - Arts et Métiers ParisTech)

2010-2011 Publications