Abstract

This thesis describes a fully automated computer vision system for detection and classification of the seven basic facial expressions using Multi-Class Support Vector Machines (SVM). Facial expressions are communicated by subtle changes in one or more discrete features such as tightening the lips, raising the eyebrows, opening and closing of eyes or certain combination of them, which can be identified through monitoring the changes in muscles movements (Action Units), located around the regions of mouth, eyes and eyebrows. An analytic representation of face with fifteen feature points describing the geometric and physical (muscle) model of facial expression structure has been used that represents and identifies the principal muscle actions and also provides visual observation (sensing) of the discrete features responsible for each of the seven basic human emotions. Feature points from the region of mouth have been detected by segmenting the lip contour applying a newly introduced variational formulation of the existing level set method. In addition, a multi-detector approach of facial feature point detection has been utilized for identifying the points of interest from the regions of eyes, eyebrows and nose. The feature vector composed of fifteen features is then obtained with respect to the average representation of neutral face by calculating the degree of displacement of five different pairs of points, and measuring the deviations of ten points from a non-changeable rigid point. Finally, the obtained feature sets are used to train a Multi-Class SVM classifier. The proposed automated facial expressions classification system has been tested extensively on two publicly available facial expression databases and 92.04% and 86.33% of average successful classification rates have been achieved. Besides, satisfactory results have been obtained by comparing the proposed method with other previous methods of facial expression classification.