A list of useful resources in the sign language recognition, translation or generation using different languages. Created during the hearai.pl project.
Feel free to add issue with short description of new publication or create a pull request - add the new resource to the table or fill missing description.
- Hear AI project: hearai/hearai
| Dataset | Language | Classes | Size | Data type | Adnotation | Language level | Licence | Link |
|---|---|---|---|---|---|---|---|---|
| Year | Paper | Dataset | Language | Task | Algorithms | Results | Code |
|---|---|---|---|---|---|---|---|
| 2021 | ANONYSIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisation | SMILE | German SL | Sign Language Production for Sign Language Video Anonymisation | AnonySign architecture | LPIPS ~ 0.243, FID ~ 49.48 | ❌ |
| 2021 | Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives | Pre-processed Phoenix14T | German SL | Sign Language Production | Mixture of Motion Primitives architecture | BLEU-4 ~ 12.67 | ❌ |
| 2021 | On-device Real-time Hand Gesture Recognition | ❌ | American SL | Hand Gesture Recognition | Hand Tracking + NN | Recall=88% | ❌ |
| 2021 | Development of a software module for recognizing the fingerspelling of the Russian Sign Language based on LSTM | ❌ | Russian SL | Sign Alphabet Recognition | LSTM Neural Network | Precision=91%, Recall=91% | ❌ |
| 2021 | Artificial Intelligence Technologies for Sign Language | - | - | Sign Language Recognition & Translation | - | - | - |
| 2021 | A Deep Convolutional Neural Network Approach to Sign Alphabet Recognition | Sign Language MNIST | American Sign Language | Sign Alphabet Recognition | CNN | Accuracy=~94% | Kaggle |
| 2021 | Efficient sign language recognition system and dataset creation method based on deep learning and image processing | ❌ | Brazilian Sign Language | Sign Language Recognition | XCeption | Accuracy=~80% | ❌ |
| 2021 | Multi-Modal Zero-Shot Sign Language Recognition | RKS-PERSIAN, ASLVID, isoGD | Persian Sign Language, American Sign Language | Sign Language Recognition | C3D, LSTM, BERT | Accuracy=~68% | ❌ |
| 2021 | Application of Transfer Learning to Sign Language Recognition using an Inflated 3D Deep Convolutional Neural Network | SIGNUM, MS-ASL | German Sign Language, American Sign Language | Sign Language Recognition | Inception-v3 | Accurracy=49% | Github |
| 2021 | Skeleton Aware Multi-modal Sign Language Recognition | AUTSL | Turkish Sign Language | Sign Language Recognition | SAM-SLR | Top-1Accuracy=~95%, Top-2Accuracy=~99.7% | Github |
| 2021 | Word-level Sign Language Recognition with Multi-stream Neural Networks Focusing on Local Regions | WLASL, ML-ASL | American Sign Lnaguage | Sign Language Recognition | YOLO3, I3D, ST-GCN | Top-10Accuracy=92.94% | ❌ |
| 2021 | Automatic Segmentation of Sign Language into Subtitle-Units | MEDIAPI-SKEL | French Sign Language | Sign Language Segmentation | ST-GCN, BiLSTM | Precision=~56%, Recal=~75% | ❌ |
| 2021 | SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition | MS-ASL, WLASL, NMFs-CSL, SLR500 | American Sign Language, Chinese Sign Language | Sign Language Recognition | SignBERT, Transformers | WLASL2000: top-1Accuracy=~54%, top-5Accuracy=~87% | ❌ |
| 2021 | PiSLTRc: Position-informed Sign Language Transformer with Content-aware Convolution | PHOENIX-2014, PHOENIX-2014-T, CSL | German Sign Language, Chinese Sign Language | Sign Language Recognition, Sign Language Translation | CNN, Sign Language Transformers, Self Attention Mechanism | PHOENIX2014T: WER=~23%, BLEU-4=~23% | Github |
| 2020 | Progressive Transformers for End-to-End Sign Language Production | Pre-processed Phoenix14T | German SL | Sign Language Production | Progressive Transformer | BLEU-4 ~ 9.94 | Github |
| 2020 | HamNoSyS2SiGML: Translating HamNoSys Into SiGML | ❌ | ❌ | Translating HamNoSys Into SiGML | Convert HamNoSys symbols to their Unicode codes | ❌ | Github |
| 2020 | Video-to-HamNoSys Automated Annotation System | DGS Corspus | Multiple | Convert Pose to HamNoSys | Tree-like-structure | Accuracy=~22% | ❌ |
| 2020 | Combining Feature Selection with Neural Networks for Polish Sign Alphabet Recognition | ❌ | Polish Sign Language | Sign Alphabet Recognition | VGG16 | ❌ | ❌ |
| 2020 | Independent sign language recognition with 3D body, hands, and face reconstruction | GSLL | Greek Sign Language | Sign Language Recognition | I3D, SMPL-X | ❌ | ❌ |
| 2020 | Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation | RWTH-Phoenix | German Sign Language | Sign Language Recognition, Sign Language Translation | SLRR, SLTT (Transformers) | WER=24%, BLEU-4=22% | Github |
| 2020 | Phonologically-Meaningful Subunits for Deep Learning-Based Sign Language Recognition | RWTH-Phoenix | German Sign Language | Sign Language Recognition | Trajectory Space Factorization, RNN | WER=~27%, Accuracy=~73% | ❌ |
| 2020 | Real-Time Sign Language Detection using Human Pose Estimation | DGS Corpus | German Sign Language | Sign Language Detection | LSTM | Accuracy=~92% | Github |
| 2020 | Pose-based Sign Language Recognition using GCN and BERT | WLASL | American Sign Lnaguage | Sign Language Recognition | GCN, BERT | Top-1Accuracy~60%, Top-5Accuracy=~84% | ❌ |
| 2020 | Spatial-Temporal Graph Convolutional Networks for Sign Language Recognition | ASLLVD | American Sign Language | Sign Language Recognition | ST-GCN | Accuracy=~61% | Github |
| 2019 | Improving American Sign Language Recognition with Synthetic Data | SYN1...10 | American SL | Sign Language Recognition with Synthetic Data | DeepHand model, K-means Clustering | 58.7% Acc < 71.1% | Github |
| 2019 | Exploiting Spatial-temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks | Human3.6M, STB | 3D Pose Estimation | GCN | distanceError=~39mm | Github | |
| 2018 | Approach to the Sign Language Gesture Recognition Framework Based on HamNoSys Analysis | sEMG, ACC, GYRO | Russian Sign Language | Sign Language Gesture Recognition | ❌ | ❌ | ❌ |
| 2018 | OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields | MPII, COCO | Pose Estimation | CNN, Affinity Fields | AP=~70% | Github | |
| 2018 | Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition | Kinetics, NTU-RGBD | Action Recognition | ST-GCN | Top-5Accuracy=~53% | Github |