Facial expressions

Facial expressions are a form of non-verbal communication that convey emotions and intentions through the movement of facial muscles. These expressions can be both voluntary and involuntary, reflecting a wide range of emotions such as happiness, sadness, anger, surprise, fear, and disgust. The study of facial expressions is crucial in understanding human behavior, as they provide insights into a person’s emotional state and reactions.

Importance of facial expressions

Facial expressions play a vital role in psychological research to help in understanding emotional responses and social interactions. In behavioral studies facial expressions provide insights into non-verbal communication and human behavior in response to specific stimuli and contexts.

Automated facial expression analysis in NoldusHub

In NoldusHub, for the automatic analysis of facial expressions , a neural network is used to classify the six basic facial expressions described by Ekman and coworkers. Facial expressions are analyzed in three steps:

1. Face finding: The position of the face in an image is found using a deep learning based face finding algorithm.
2. Face modeling: Deep neural networks synthesize an artificial face model, which describes the location of almost 500 key points in the face. The predicted key points of such a model are learned from a database of annotated images.
3. Facial expression analysis: Classification of the facial expressions is done by a trained deep artificial neural network to recognize patterns in the face. Over 20,000 images of people faces from all over the world, that were manually annotated, were used to train the artificial neural network. The network was trained to classify the six basic or universal emotions described by Ekman: happy, sad, angry, surprised, scared, disgusted and a neutral state.

_{The six basic facial expression + neutral and their reliability}

Based on the underlying Action Units (i.e., muscles/muscle groups), affective facial expressions, such as Interest and Boredom, can be estimated.

Conclusion

Facial expressions are a powerful tool for understanding human emotions and behavior. Through NoldusHub, researchers can leverage facial expression data alongside other behavioral and physiological signals to gain a comprehensive understanding of human behavior, enhancing the quality and depth of their studies.