A Deep Learning Approach for Gloss Sign Language Translation using Transformer | ||
| Journal of Computing and Communication | ||
| Article 1, Volume 1, Issue 2, August 2022, Pages 1-8 PDF (660.32 K) | ||
| Document Type: Original Article | ||
| DOI: 10.21608/jocc.2022.254979 | ||
| Authors | ||
| Ammar Mohamed* 1; Hesham Hefny2; mohamed Amin3 | ||
| 1Faculty of Graduate Studies for Statistical Research Cairo University | ||
| 2Department of Computer Science, Faculty of Graduate Studies for Statistical Research, Cairo University | ||
| 3Department of Computer Science, Faculty of Graduate Studies for Statistical Research, Cairo University, Cairo, Egypt | ||
| Abstract | ||
| One of the most recent applications of machine learning is to translate sign language into natural language. Many studies have attempted to classify sign language based on whether it is gesture or facial expression. These efforts, however, ignore genuine sentences' linguistic structure and context. The quality of traditional translation methods is poor, and their underlying models are not salable. They also take a long time to complete. The contribution of this paper is that it suggests utilizing a transformer to perform bidirectional translation using a deep learning approach. The proposed models experiment on the ASLG-PC12 corpus. The experimental results reveal that the proposed models outperform other approaches to the same corpus in both directions of translation, with ROUGE and BLEU scores of 98.78% and 96.89%, respectively, when translating from text to gloss. Additionally, the results indicate that the model with two layers achieves the best result with ROUGE and BLEU scores of 96.90% and 84.82% when translating from gloss to text. | ||
| Keywords | ||
| Neural Machine Translation; Sequence to Sequence Model; Sign Language; Deep learning; Transformer | ||
|
Statistics Article View: 423 PDF Download: 1,125 |
||
