Research Area:  Machine Learning

Abstract:

Using gestures can help people with certain disabilities in communicating with other people. This paper proposes a lightweight model based on YOLO (You Only Look Once) v3 and DarkNet-53 convolutional neural networks for gesture recognition without additional preprocessing, image filtering, and enhancement of images. The proposed model achieved high accuracy even in a complex environment, and it successfully detected gestures even in low-resolution picture mode. The proposed model was evaluated on a labeled dataset of hand gestures in both Pascal VOC and YOLO format. We achieved better results by extracting features from the hand and recognized hand gestures of our proposed YOLOv3 based model with accuracy, precision, recall, and an F-1 score of 97.68, 94.88, 98.66, and 96.70%, respectively. Further, we compared our model with Single Shot Detector (SSD) and Visual Geometry Group (VGG16), which achieved an accuracy between 82 and 85%. The trained model can be used for real-time detection, both for static hand images and dynamic gestures recorded on a video.

Keywords:  
convolutional neural network
hand gesture
digital image processing
YOLOv3
artificial intelligence

Author(s) Name:  Abdullah Mujahid ,Mazhar Javed Awan ,Awais Yasin ,Mazin Abed Mohammed,Robertas Damaševičius,Rytis Maskeliūnas and Karrar Hameed Abdulkareem

Publisher name:  MDPI

DOI:  10.3390/app11094164

Volume Information:   Volume 11,Issue 9

Paper Link:   https://www.mdpi.com/2076-3417/11/9/4164