Exploiting Deep Learning-Based LSTM Classification for Improving Hand Gesture Recognition to Enhance Visitors’ Museum Experiences

Hand gesture recognition (HGR) is one of the main axes of the Human-Computer Interaction (HCI) research field. And computer vision is a very active dedicated research area. However, traditional vision-based methods, like using a fixed camera to record video sequences of sign language, have some serious drawbacks inherent to the fixed camera location, complex lighting conditions, and cluttered backgrounds. Motivated by these potential limitations, the present paper addresses the detection and classification of hand gestures based rather on wearable video monitoring data. A new feature extraction strategy based on five hand's partial occupancy areas in images is provided. And a deep learning formalism, using the Long Short-Term Memory (LSTM) algorithm, has been implemented to ensure an effectual separation between classes. To analyze the performances of the classification, available data have been used for experiments, and both virtual and real museum scenarios are considered. The obtained results demonstrated that the combined five area ratios and LSTM classification were not only able to recognize different hand gestures but it was also able to distinguish between actions with a high degree of similarity (like slide left and slide right classes). The use of deep learning-based LSTM algorithm in the classification phase helped in reducing significantly the number of misclassifications, and achieving an outstanding recognition performance when challenged with real-world data.