Please use this identifier to cite or link to this item:
https://hdl.handle.net/20.500.13091/5717
Title: | Isolated sign language recognition through integrating pose data and motion history images | Authors: | Akdağ, Ali Baykan, Ömer Kaan |
Keywords: | Sign language recognition Deep learning Motion history image Feature fusion |
Publisher: | Peerj Inc | Abstract: | This article presents an innovative approach for the task of isolated sign language recognition (SLR); this approach centers on the integration of pose data with motion history images (MHIs) derived from these data. Our research combines spatial information obtained from body, hand, and face poses with the comprehensive details provided by three-channel MHI data concerning the temporal dynamics of the sign. Particularly, our developed finger pose-based MHI (FP-MHI) feature significantly enhances the recognition success, capturing the nuances of finger movements and gestures, unlike existing approaches in SLR. This feature improves the accuracy and reliability of SLR systems by more accurately capturing the fine details and richness of sign language. Additionally, we enhance the overall model accuracy by predicting missing pose data through linear interpolation. Our study, based on the randomized leaky rectified linear unit (RReLU) enhanced ResNet-18 model, successfully handles the interaction between manual and non-manual features through the fusion of extracted features and classification with a support vector machine (SVM). This innovative integration demonstrates competitive and superior results compared to current methodologies in the field of SLR across various datasets, including BosphorusSign22k-general, BosphorusSign22k, LSA64, and GSL, in our experiments. | URI: | https://doi.org/10.7717/peerj-cs.2054 https://hdl.handle.net/20.500.13091/5717 |
ISSN: | 2376-5992 |
Appears in Collections: | Scopus İndeksli Yayınlar Koleksiyonu / Scopus Indexed Publications Collections WoS İndeksli Yayınlar Koleksiyonu / WoS Indexed Publications Collections |
Show full item record
CORE Recommender
Items in GCRIS Repository are protected by copyright, with all rights reserved, unless otherwise indicated.