Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Feb 4:12:800657.
doi: 10.3389/fpsyg.2021.800657. eCollection 2021.

An Android for Emotional Interaction: Spatiotemporal Validation of Its Facial Expressions

Wataru Sato  1   2 , Shushi Namba  1 , Dongsheng Yang  3 , Shin'ya Nishida  3   4 , Carlos Ishi  5 , Takashi Minato  5
Affiliations

An Android for Emotional Interaction: Spatiotemporal Validation of Its Facial Expressions

Wataru Sato et al. Front Psychol. .

Abstract

Android robots capable of emotional interactions with humans have considerable potential for application to research. While several studies developed androids that can exhibit human-like emotional facial expressions, few have empirically validated androids' facial expressions. To investigate this issue, we developed an android head called Nikola based on human psychology and conducted three studies to test the validity of its facial expressions. In Study 1, Nikola produced single facial actions, which were evaluated in accordance with the Facial Action Coding System. The results showed that 17 action units were appropriately produced. In Study 2, Nikola produced the prototypical facial expressions for six basic emotions (anger, disgust, fear, happiness, sadness, and surprise), and naïve participants labeled photographs of the expressions. The recognition accuracy of all emotions was higher than chance level. In Study 3, Nikola produced dynamic facial expressions for six basic emotions at four different speeds, and naïve participants evaluated the naturalness of the speed of each expression. The effect of speed differed across emotions, as in previous studies of human expressions. These data validate the spatial and temporal patterns of Nikola's emotional facial expressions, and suggest that it may be useful for future psychological studies and real-life applications.

Keywords: Facial Action Coding System; android; dynamic facial expression; emotional facial expression; robot.

PubMed Disclaimer

Conflict of interest statement

SNi was employed by the company Nippon Telegraph and Telephone Corporation. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

FIGURE 1
FIGURE 1
Illustrations of the facial action units (AUs) produced by the android Nikola. For AU 25, AU 25 + 26 is shown.
FIGURE 2
FIGURE 2
Illustrations of the facial expressions of six basic emotions produced by the android Nikola.
FIGURE 3
FIGURE 3
Mean (±SE) accuracy percentages for the recognition of six emotions in facial expressions in Study 2.
FIGURE 4
FIGURE 4
Illustration of the dynamic facial expression stimuli used in Study 3. (Left) Nikola’s face changed from a neutral expression to one of six emotional expressions. (Right) Schematic illustration of the four speed conditions.
FIGURE 5
FIGURE 5
Mean (±SE) naturalness ratings for facial expressions of six emotions under the four speed conditions in Study 3.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Adams A., Robinson P. (2011). “An android head for social-emotional intervention for children with autism spectrum conditions,” in Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction, ACII 2011, Memphis, TN, 10.1007/978-3-642-24571-8_19 - DOI
    1. Ahn H. S., Lee D. W., Choi D., Lee D. Y., Hur M., Lee H. (2012). “Appropriate emotions for facial expressions of 33-DOFs android head EveR-4 H33,” in Proceedings of the 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, 10.1109/ROMAN.2012.6343898 - DOI
    1. Allison B., Nejat G., Kao E. (2009). The design of an expressive humanlike socially assistive robot. J. Mech. Robot. 1:011001. 10.1115/1.2959097 - DOI
    1. Asheber W. T., Lin C.-Y., Yen S. H. (2016). Humanoid head face mechanism with expandable facial expressions. Intern. J. Adv. Robot. Syst. 13:29. 10.5772/62181 - DOI
    1. Bartneck C. (2003). “Interacting with an embodied emotional character,” in Proceedings of the 2003 International Conference on Designing Pleasurable Products and Interfaces (DPPI2003), Pittsburgh, PA, 10.1145/782896.782911 - DOI

LinkOut - more resources