Bilgisayar ve Bilişim Fakültesi Koleksiyonu

Permanent URI for this collectionhttps://hdl.handle.net/20.500.13091/10834

Browse

Browsing Bilgisayar ve Bilişim Fakültesi Koleksiyonu by Author "Akdağ, Ali"

Filter results by typing the first few letters
Now showing 1 - 2 of 2
  • Thumbnail Image
    Article
    Citation - WoS: 4
    Citation - Scopus: 6
    Enhancing Signer-Independent Recognition of Isolated Sign Language Through Advanced Deep Learning Techniques and Feature Fusion
    (MDPI, 2024) Akdağ, Ali; Baykan, Ömer Kaan
    Sign Language Recognition (SLR) systems are crucial bridges facilitating communication between deaf or hard-of-hearing individuals and the hearing world. Existing SLR technologies, while advancing, often grapple with challenges such as accurately capturing the dynamic and complex nature of sign language, which includes both manual and non-manual elements like facial expressions and body movements. These systems sometimes fall short in environments with different backgrounds or lighting conditions, hindering their practical applicability and robustness. This study introduces an innovative approach to isolated sign language word recognition using a novel deep learning model that combines the strengths of both residual three-dimensional (R3D) and temporally separated (R(2+1)D) convolutional blocks. The R3(2+1)D-SLR network model demonstrates a superior ability to capture the intricate spatial and temporal features crucial for accurate sign recognition. Our system combines data from the signer's body, hands, and face, extracted using the R3(2+1)D-SLR model, and employs a Support Vector Machine (SVM) for classification. It demonstrates remarkable improvements in accuracy and robustness across various backgrounds by utilizing pose data over RGB data. With this pose-based approach, our proposed system achieved 94.52% and 98.53% test accuracy in signer-independent evaluations on the BosphorusSign22k-general and LSA64 datasets.
  • Thumbnail Image
    Article
    Citation - WoS: 2
    Citation - Scopus: 2
    Isolated Sign Language Recognition Through Integrating Pose Data and Motion History Images
    (Peerj Inc, 2024) Akdağ, Ali; Baykan, Ömer Kaan
    This article presents an innovative approach for the task of isolated sign language recognition (SLR); this approach centers on the integration of pose data with motion history images (MHIs) derived from these data. Our research combines spatial information obtained from body, hand, and face poses with the comprehensive details provided by three-channel MHI data concerning the temporal dynamics of the sign. Particularly, our developed finger pose-based MHI (FP-MHI) feature significantly enhances the recognition success, capturing the nuances of finger movements and gestures, unlike existing approaches in SLR. This feature improves the accuracy and reliability of SLR systems by more accurately capturing the fine details and richness of sign language. Additionally, we enhance the overall model accuracy by predicting missing pose data through linear interpolation. Our study, based on the randomized leaky rectified linear unit (RReLU) enhanced ResNet-18 model, successfully handles the interaction between manual and non-manual features through the fusion of extracted features and classification with a support vector machine (SVM). This innovative integration demonstrates competitive and superior results compared to current methodologies in the field of SLR across various datasets, including BosphorusSign22k-general, BosphorusSign22k, LSA64, and GSL, in our experiments.