Changes in facial recognition and facial expressions with age

The ability to express and recognize emotion via facial expressions is well known to change with age. The present study investigated the differences in the facial recognition and facial expression of the elderly (n = 57) and the young (n = 115) and measure how each group uses different facial muscles for each emotion with Facial Action Coding System (FACS). In facial recognition task, the elderly did not recognize facial expressions better than young people and reported stronger feelings of fear and sad from photographs. In making facial expression task, the elderly rated all their facial expressions as stronger than the younger, but in fact, they expressed strong expressions in fear and anger. Furthermore, the elderly used more muscles in the lower face when making facial expressions than younger people. These results help to understand better how the facial recognition and expression of the elderly change, and show that the elderly do not effectively execute the topdown processing concerning facial expression.


Introduction
Expression and recognition of emotion through facial expression are a fundamental stage of basic communication. Facial expression is a critical way to communicate with the surroundings in terms of its role to convey the primary meaning of social information [1,2]. Hence, there has been a detailed examination of the understanding of the facial expression of basic emotions, including happiness, surprise, fear, anger, disgust, and sadness [3]. It is well known that emotional expression and recognition skills through facial expression increase or decrease with age [4,5]. Ebner and Johnson [6] presented a variety of facial expressions to the old and young people and confirmed how they recognized them. Both young and old were well aware of the expression of happiness, while older people are less aware of negative facial expressions such as anger and sadness. Besides, not only declined in sadness in facial expression recognition but also improvement in recognition of disgust facial expression is reported in the older group [7 -9]. Also, the elderly were more inclined to think that they felt happy when they showed smiles even if the reason for the smile is unclear [10].
The above phenomenon explained as the socioemotional selectivity theory (SST) [11] that the elderly tend to interpret certain events in an emotionally balanced and peaceful way due to their relatively short remaining time before death than young people [12]. This difference in recognition of facial expression between young and old is interpreted as the way of controlling and regulating emotion changes with age. From a neuropathology point of view, besides, negative emotions such as fear are linked to amygdala [13,14], and disgust is related to insula and basal ganglia [15,16]. Therefore, various changes in negative emotion in the older people can be explained by the natural aging process of the brain, accompanied by damage to medial temporal lobe and basal ganglia [17 -19].
The above studies clearly show differences in the elderly and younger groups in the process of recognizing and evaluating emotions. However, research on how the facial expressions differ in the elderly and younger groups is insufficient. Most of the past studies had focused on facial recognition, and studies about facial expression are little due to the lack of research methodology. Facial expressions in older people are more difficult to recognize since they are wrinkled and have poor control of facial muscle as they get older [20]. Permanent changes such as atrophy of the facial skeleton, malposition of fatty muscles and loss of soft tissue happen most commonly in the areas of the maxilla, the mandible, and the anterior nasal spine [21]. These changes, especially with aging at the lower face, will affect the mechanism by which facial expression is made. Furthermore, the lower part of the face is prominently used for voluntary facial expression, i.e., the posed facial expression in a social context, which is handled by the left hemisphere while the upper face is treated mainly through the right hemisphere [22 -24]. Therefore, many changes in the face of the elderly due to aging mentioned above will affect voluntary facial expression due to the inability to perform top-down commands from the left hemisphere. Otherwise, Fear of the future due to physical aging and disconnection from society might weaken the emotional expression itself by the elderly. Therefore, it is necessary to examine how facial expressions change with age analyzing how the elderly use facial muscles when making a facial expression, and find a correlation between the intensity of facial expression that the elderly intend to express and the intensity of emotion that is actually expressed and compare them with the younger group. It allows us to be able to understand better the characteristics of the facial expression of both the elderly and the young.
In this study, we will evaluate both facial expression and recognition in the healthy elderly and younger group by adopting a facial action coding system (FACS) to interpret voluntarily posed facial expression [25,26]. For this purpose, experiments on facial expression were conducted by recruiting elderly people over 62 years and young people under 39 years. Through the experiments, we will 1) examine how well the elderly and young people recognize facial expressions and compare how they evaluate the intensity of emotion they feel in a given facial expression, 2) examine the differences in the muscles of the upper and lower parts of the face when the elderly and young people make facial expression, and compare the strength of the facial expressions that they intended with the intensity of the expression they actually made 3) examine the correlation between the intensity of expressed emotion and facial action unit (AU) in each group.

Participants
61 elderly and 115 healthy young were recruited in this study. The elderly were between 62 to 84 years old and recruited from the Alzheimer's disease research center of SMG-SNU Borame hospital. Healthy young participants were recruited from the university student participant pool and aged between 18 to 39. None of them had a history of psychiatric disorder ever. Major medical diseases, severe head injury, and visual impairment were excluded in all groups. Especially, all the elderly participants were free from the diagnosis criteria of AD and depression with DSM-IV [27]. However, four elderly subjects were excluded from further analysis due to insufficient data during the experiment. Table 1 summarizes the demographic and clinical characteristics of the population. This study was approved by the Institutional Review Board of the Seoul National University Hospital (IRB No. 30-2017-63), and all participants submitted written consent for participating in the study.

Procedures
A series of photos containing six basic emotions and neutral on facial expression was presented to participants (figure 1). Participants performed facial emotion recognition tasks and posed for six basic emotions. To be more specific, they were presented with a photograph of a person representing a certain facial emotion, asked to identify the emotion the person was conveying (facial recognition task) and required to make the same face with photo towards the camera for 15 seconds (facial expression task). The facial expression responses were recorded via a Canon EOS 70D DSLR Camera with a 50mm prime lens, 720p resolution, and 60fps frame rate. The camera was positioned on a fixed stand approximately 120-140 above the floor to capture the entire face of the participants correctly. Then, participants were self-reported the perceived emotional intensity of a photo-graph and emotional intensity that tried to express when they posed emotion range from 0 to 10. They followed the same procedures for the rest of the six basic emotions and neutral.

Facial expression acquisition analysis
Photos of participants' posed facial expressions underwent a preliminary investigation via OpenFace 2.0, an open-source toolkit for facial behavior analysis, which consists of four pipeline: 1) facial landmark detection and tracking, 2) head pose estimation, 3) eye gaze estimation, and 4) facial expression recognition [28]. For analyzing facial expression, OpenFace 2.0 recognizes facial expressions by detecting AU intensity and presence according to FACS [25]. Without using all the AUs listed in FACS, OpenFace 2.0 offers 18 a subset of AUs by cross-dataset learning; specifically, 1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26, 28, and 45. The occurrences and intensities in AUs are estimated by using machine learning algorithms. The methods for AU estimation and analysis are described in more detail elsewhere [29]. In the present study, six basic emotions were created according to emotional FACS (EMFACS) [30]. EMFACS based on the FACS that was proved to have great reliability for the assessment of human facial movement [31,32]. Examples of each emotion are shown in Table 2.

Statistical analysis
Analysis of covariance was performed to verify statistically significant differences between the groups controlling for gender, botulinum toxin (botox) use and handedness. Chi-squared tests were used to compare dichotomous outcomes. The correlation between facial expression and the intended intensity was investigated. As there was no direct comparison of facial expression in the past, the p-value was set to 0.05 or less for the explanatory analysis. Table 3 shows whether the elderly and young have the right answer for the emotion in the presented photos. While young people have a high percentage of correct answers near 100% in six basic emotions except for fear, the elderly have about half the correct answer for all emotions except for fear. Both young and elderly had a low percentage of the correct answer in fear.  Table 4A shows the intensity of the emotion in the presented photograph containing six basic emotions by groups. Subjects evaluated emotion intensity as a value between 0 and 10. An analysis of covariance was conducted to compare emotional intensity between the young and old. In fear condition, for the young (mean = 6.67, SD = 1.77) and the old (mean = 7.32, SD = 2.02) conditions; F (1,167) = 4.498, p = 0.035. As a covariate, sex was significant with F (1,167) = 6.538, p = 0.011 and botox significant with F (1,167) = 4.549, p = 0.034. In sad condition, for the young (mean = 5.94, SD = 1.69) and the old (mean = 7.07, SD = 2.05) conditions; F (1,167) = 13.809, p < 0.001. These results suggest that the elderly felt stronger feelings in fear and sad than young people.  Table 5 represent the AU value of the elderly and young for the six basic emotions. All subjects looked at the photograph representing each emotion and then made their facial expression according to the emotions indicated by the photography. AU values were obtained from each expression, and responses to the intensity of the expressions made by the elderly and young were also obtained. An analysis of covariance was conducted to compare AU values between the young and old. In fear condition, for the young (mean = 4.16, SD = 1.48) and the old (mean = 6.19, SD = 2.11) conditions; F (1, 167) = 54.056, p < 0.001. In angry condition, for the young (mean = 4.28, SD = 1.83) and the old (mean = 4.94, SD = 1.67) conditions; F (1,167) = 5.766, p = 0.017. These results suggest that the elderly group had significantly higher AU value for Fear and Angry than the young group.   Table 4B indicates group differences by intensity assessment of the facial expression containing six basic emotions that subjects expressed. Subjects evaluated their emotional facial expression intensity as a value between 0 and 10. An analysis of covariance was conducted to compare emotional intensity between the young and old. In happy condition, for the young (mean = 5.41, SD = 1.91) and the old (mean = 6.47, SD = 1.88) conditions; F (1,167) = 11.802, p = 0.001. In fear condition, for the young (mean = 5.27, SD = 1.81) and the old (mean = 5.98, SD = 2.05) conditions; F (1,167) = 20.546, p = 0.019. In surprise condition, for the young (mean = 5.81, SD = 1.83) and the old (mean = 6.51, SD = 2.11) conditions; F (1, 167) = 6.091, p = 0.015. As an adjusted variable, handedness was significant with F (1,167) = 4.912, p = 0.028. In angry condition, for the young (mean = 5.48, SD = 2.01) and the old (mean = 6.54, SD = 2.08) conditions; F (1,167) = 11.108, p = 0.001. In sad condition, for the young (mean = 4.65, SD = 1.85) and the old (mean = 6.11, SD = 2.32) conditions; F (1, 167) = 21.243, p < 0.001. In disgust condition, for the young (mean = 5.41, SD = 1.54) and the old (mean = 6.25, SD = 1.98) conditions; F (1,167) = 11.381, p = 0.001. As a covariate, handedness was significant with F (1,167) = 6.422, p = 0.012. These results suggest that the elderly evaluates their facial expression more strongly than the young group in all six basic emotions. Table 6 shows individual AU that indicates differences between the elderly and young groups for each emotion including neutral. While there were no differences between the groups in the neutral expression, the rest of the basic emotion represented differences between the groups. The elderly group used AU located in the lower face as follows. AU 15, 26 in both fear and disgust expression, AU 14 in both surprise and angry expression. The elderly group used AU located in the upper face as follows. AU 6, 7 in fear expression. The young group used AU located in a lower face as follows. AU25 in fear expression, AU 15, 17 in sad expression and AU 25 in disgust expression. The young group used AU located in the upper face as follows. AU 45 in both happy and sad expressions. Fig 3 shows where the AU in each group is located on the face.   Table 7 represents a correlation between AU and intensity of the facial expression that the elderly expressed. AU of surprise, angry and sad showed a positive correlation with the intensity of the expression, AU of happy, fear and disgust showed a negative correlation, but none of them reached a significant level.

Discussion and Conclusions
The purpose of the present study was to investigate the differences in facial expression recognition and facial expression of the elderly and young people and to examine how the facial muscle of each basic emotion was used through AUs. Older people were less likely to recognize facial expressions than young people, and they felt stronger emotions than young people after seeing photographs of fear and sad expressions. In the direct facial expression task, the elderly had a stronger expression of fear and anger than younger people, but they felt that they expressed a stronger expression on all facial expressions. As a result of analyzing the AU values for six basic facial expressions, the elderly used more lower facial muscles than the young, and there was no correlation between the evaluation of facial expression strength and AU values. Next, we will discuss the significance of results to our understanding of the relationship between facial expression and emotion in the elderly and young.

Facial expression recognition
In the current study, like other studies, showed that older people were less likely to recognize facial expression than young people in all six basic emotions. However, our study did not reveal age-related improvement in disgust recognition in other studies [7,5]. Also, after looking at the pictures of six emotions, the older people evaluate that they felt stronger emotion about fear and sad than young. Previous studies report mixed results on the emotional intensity of the elderly. Many studies indicate that there is no difference in emotional intensity between the elderly and young people [33,34], but some studies report that the elderly feel stronger emotion than the young [35]. the elderly felt much sadder than young people, especially when they are exposed to stimuli that are closely related to them about negative emotions [36]. Our results show that older people fail to accurately recognize facial expressions but still feel strong negative emotions. It is contrary to the existing SST theory that older people positively interpret events with a vague context. However, this supports the evolutionary characteristics of human beings, which is the conservative interpretation of the inaccurate context, and suggests that our results reflect the problems of the elderly in modern society. The elderly were more defensive in dealing with emotions, with stronger perception of negative emotions that could worsen their relationships or situations, rather than positive feeling that were harmless to them. This did not appear in the younger group, and it could be explained that the self-reduction and lower self-esteem experienced by the elderly were affected.  Table 5 showed that the elderly have higher AU values than young people for negative emotion such as fear and angry. In addition, the elderly valued their expressed facial emotions more strongly than the young group for all six emotions (Table  4B). This indicates that the elderly tried to build their facial expression stronger, but their intention was not reflected in actual muscle movements, resulting in weaker facial expression. In order for the elderly to make facial expressions as intended, therefore, it is necessary to utilize their brain in the top-down processing format to ensure that the commands from the brain are correctly delivered to the facial muscles.

Degenerative changes in facial expression differences with age
Regarding the expression of strong negative emotions in the elderly representing our result, age differences reported between the elderly and the young when they discriminate negative emotion. Mienaltowski et al. [37] observed that the elderly had more difficult to distinguish negative emotion when they have low intensity. As our facial recognition results, older people evaluate the negative expression of others more negatively, and they also have strong facial expression of negative emotions. This may tend to make the facial expression excessively because the elderly themselves may not be able to identify negative emotions of weak intensity.
As a result of comparing all AU values for each emotion, next, there were differences between groups in all six basic emotions while there was no difference in the neutral emotion. Although the sagging of the mouth tail due to aging may cause a difference in facial expression of the elderly and young people, there is no difference in the neutral condition between groups. Thus, this supports our finding that the elderly and young use facial muscles in different ways when they are expressing emotions. The individual AU values of the elderly group were mostly different in the lower face, suggesting that the elderly communicate mainly with muscles related to the mouth and jaw. Since the lower face has a unilateral innervation, it can be explained that the lower face of the elderly that used to the conscious command of the top-down process contributed more to the facial expression. Murphe et al. [38] and Sullivan et al. [39] reports that young subjects stare primarily at the upper part of the face when looking at facial expression, while older people stare at the lower part of the face. In particular, when discriminating negative emotion such as fear, angry and sad, the longer the time to look at the upper part of the face, the better the emotion recognition, while the longer time to look at the lower part of the face, the less discriminating the face emotion is. In general, people look at eyes in the upper part of the face when they interpret emotion regardless of their age. Where the upper facial features mostly recognize fear, angry and sad, while information from the lower face recognizes happy and disgust [40]. The lower facial expression is associated with the involvement of the left hemisphere and happy-pleasant types of expression, while the upper facial expression is associated with the involvement of the right hemisphere and surprise-fear types of expression [23]. Our result shows that the elderly used more of the lower face when they made facial expressions, which indicates that older people are not communicating effectively through facial expression. Table 7 indicates that there was no relation between the intensity of the facial expression that the elderly expressed and AU values. This means that the emotion is not sufficiently revealed in the actual expression as the elderly intends to express. In other words, the elderly cannot express the strength they want to make facial expressions since the command from the brain is not properly delivered to the facial muscle, and this may suggest that the top-down process associated with facial expressions may be impaired in the elderly. Further research is needed to examine the relationship between emotional expression and top-down mechanisms in the elderly.

Limitation
Three issues should be discussed for the interpretation of our results. First. There is a report that the age of the observer and the subject in the photograph affect the evaluation of emotion in the photograph. The consequences are different between the elderly looking at a young person's picture and matching his facial expression, and young people looking at a young person's picture and matching his facial expression [41]. The age of the face regards to play a significant role in recognizing facial expressions. Since we presented photographs of young people as stimuli to both the elderly and younger groups, however, this should be considered for interpretation, and further study is needed. Next, subjects in our study looked at the photograph and judged their emotions first, and followed the facial expressions in the photograph. If the subject misidentifies the emotion of the photographs, it is possible that the subject has a different facial expression than the photograph, which may not match the analyzed AU. Last, both young and old groups had a low rate of correct answers in the facial expression recognition task. Previous studies have reported that young people perform better fear emotion recognition than older people [42], and it is contrary to our result.

Conclusion
Overall, the present study is the first to examine facial expression recognition and facial expression at the same time. This study identified the differences in facial recognition between the elderly and young people, and the differences of each group by comparing the intensity of facial expression. Also, this study provides evidence for the difference in facial expression between the elderly and the young. When the elderly and the young make a facial expression, the intensity of facial expressions is different, and the muscles used are also different. The elderly expressed more negative emotions than young people and used more muscles in the lower face when they are making facials expressions. It proves that the elderly cannot communicate efficiently through their facial expressions according to the fact that expressions appeared on upper and lower parts of the face are different, and the areas of the brain involved are also different [23]. Therefore, we hope that the various features of the posed emotion of the elderly in this study can be a significant contribution to diverse clinical research related emotions of the elderly.  Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: The data presented in this study are available on request from the corresponding author.