Master's Thesis by Jakob Kruse
This repository serves as a framework for emotion classification. It implements several classifiers for different data types/modalities like face images, text/sentences, voice/speech, physiological signals (heartrate, acceleration from a smartwatch) and more sensors. The classifiers can be trained and then used for inference. For training, recommended datasets and how to set them up is described in data/DESCRIPTION.md.
Emotions: All classifiers used in this work use 7 emotion classes. The classes are the six basic emotions defined by Paul Ekman: happiness, sadness, anger, fear, surprise, disgust; and an additional neutral class. Existing emotion classification models have been retrained on these seven classes in order to make the models comparable and have a single interface for all models.
Classifiers: All classifiers use the same interface for emotion classification.
I recommend setting up a new virtual environment in the venv folder.
How to set up a virtual environment in Python 3
sudo apt install python3-pip python3-venv
python -m venv venv
source venv/bin/activate
Then, install the required packages:
pip install -r requirements.txtTo download the datasets, please refer to data/DESCRIPTION.md. Some of the datasets are publicly available, to download them you can use:
bash data/prepare_all.shPrepare all the classification models:
bash models/install_all.shWindows Installation Instructions
All installation scripts are made for Linux and don't work out of the box for Windows. These steps are required (from root folder):- cd .\models\speech; git clone https://github.com/GasserElbanna/serab-byols.git; python -m pip install -e ./serab-byols
- python -m textblob.download_corpora
- In the file
emotion-recognition\models\speech\serab-byols\byol_a\common.pyin line 20, change the sound backend from "sox_io" to "soundfile" - To download the datasets, manual steps are required, as the download scripts are not made for Windows.
To download the pretrained models described in my thesis, use the script:
bash models/download_model.sh --data <modality> --model <model_name>Currently, only the best model for each modality is available. For details on
the available options for --data and --model, please refer to the help:
bash models/download_model.sh -hRun the tests using:
python -m pytest testsRun the tests with coverage:
python -m pytest --cov=src/ tests/📂 .github |
Folder containing Github CI configurations. |
📂 data |
Folder that contains all the training and evaluation datasets. |
📂 docs |
Folder that contains the documentation wiht sphinx. |
📂 experiments |
Folder with experiment configurations and scripts for training models. |
📂 models |
Folder that contains trained models for emotion classification. |
📂 src |
Folder with source code for this project. |
📃 .gitignore |
Files that are ignored by git. |
📃 .pre-commit-config.yaml |
Linter configuration file. |
📃 LICENSE |
MIT License file. |
📃 pyproject.toml |
Linter and Test configurations. |
📃 README.md |
Explanation (You are here). |
📃 requirements.txt |
Python Package requirements for the project. |
📃 setup.cfg |
Another linter configuration file. |
Install the pre-commit hooks for linting:
pre-commit installTo run all linters manually, use:
pre-commitNote: only changes added to git are included in linting when using the pre-commit command.
You can also run the single linters one at a time to apply the linter to unstaged files:
black --check .
flake8 .
isort --check-only .