Skip to content
/ S2D Public

[TAFFC 2024] The official implementation of paper: From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos

License

Notifications You must be signed in to change notification settings

MSA-LMC/S2D

Repository files navigation

From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos

image

PWC
PWC
PWC

PWC
PWC

From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
Yin Chen$^{†}$, Jia Li$^{†∗}$, Shiguang Shan, Meng Wang, and Richang Hong

📰 News

[2024.9.5] The fine-tuned checkpoints are available.

[2024.9.2] The code and pre-trained models are available.

[2024.8.28] The paper is accepted by IEEE Transactions on Affective Computing.

[2023.12.5] Code and pre-trained models will be released here.

🚀 Main Results

Dynamic Facial Expression Recognition

image image

Static Facial Expression Recognition

image

Visualization

image

Fine-tune with pre-trained weights

1、 Download the pre-trained weights from baidu drive or google drive, and move it to the ckpts directory.

2、 Run the following command to fine-tune the model on the target dataset.

conda create -n s2d python=3.9
conda activate s2d
pip install -r requirements.txt
bash run.sh

📋 Reported Results and Fine-tuned Weights

The fine-tuned checkpoints can be downloaded from here.

Datasets w/o oversampling w/ oversampling
UAR WAR UAR WAR
FERV39K
FERV39K 41.28 52.56 43.97 46.21
DFEW
DFEW01 61.56 76.16 64.80 75.35
DFEW02 59.93 73.99 62.54 72.53
DFEW03 61.33 76.41 66.47 75.87
DFEW04 62.75 76.31 66.03 74.48
DFEW05 63.51 77.27 67.43 76.80
DFEW 61.82 76.03 65.45 74.81
MAFW
MAFW01 32.78 46.76 36.16 44.21
MAFW02 40.43 55.96 41.94 51.22
MAFW03 47.01 62.08 48.08 61.48
MAFW04 45.66 62.61 47.67 60.64
MAFW05 43.45 59.42 43.16 58.55
MAFW 41.86 57.37 43.40 55.22

✏️ Citation

If you find this work helpful, please consider citing:

@ARTICLE{10663980,
  author={Chen, Yin and Li, Jia and Shan, Shiguang and Wang, Meng and Hong, Richang},
  journal={IEEE Transactions on Affective Computing}, 
  title={From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos}, 
  year={2024},
  volume={},
  number={},
  pages={1-15},
  keywords={Adaptation models;Videos;Computational modeling;Feature extraction;Transformers;Task analysis;Face recognition;Dynamic facial expression recognition;emotion ambiguity;model adaptation;transfer learning},
  doi={10.1109/TAFFC.2024.3453443}}

About

[TAFFC 2024] The official implementation of paper: From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published