Fingertip gestures recognition using leap motion and camera for interaction with virtual environment

Inam Ur Rehman*, Sehat Ullah, Dawar Khan, Shah Khalid, Aftab Alam, Gul Jabeen, Ihsan Rabbi, Haseeb Ur Rahman, Numan Ali, Muhammad Azher, Syed Nabi, Sangeen Khan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


The emergence in computing and the latest hardware technologies realized the use of natural interaction with computers. Gesture-based interaction is one of the prominent fields of natural interactions. The recognition and application of hand gestures in virtual environments (VEs) need extensive calculations due to the complexities involved, which directly affect the performance and realism of interaction. In this paper, we propose a new interaction technique that uses single fingertip-based gestures for interaction with VEs. The objective of the study is to minimize the computational cost, increase performance, and improve usability. The interaction involves navigation, selection, translation, and release of objects. For this purpose, we propose a low-cost camera-based system that uses a colored fingertip for the fastest and accurate recognition of gestures. We also implemented the proposed interaction technique using the Leap Motion controller. We present a comparative analysis of the proposed system with the Leap Motion controller for gesture recognition and operation. A VE was developed for experimental purposes. Moreover, we conducted a comprehensive analysis of two different recognition setups including video camera and the Leap Motion sensor. The key parameters for analysis were task accuracy, interaction volume, update rate, and spatial distortion of accuracy. We used the Standard Usability Scale (SUS) for system usability analysis. The experiments revealed that camera implementation was found with good performance, less spatial distortion of accuracy, and large interaction volume as compared to the Leap Motion sensor. We also found the proposed interaction technique highly usable in terms of user satisfaction, user-friendliness, learning, and consistency.

Original languageEnglish (US)
Article number1986
Pages (from-to)1-20
Number of pages20
JournalElectronics (Switzerland)
Issue number12
StatePublished - Dec 2020

Bibliographical note

Publisher Copyright:
© 2020 by the authors. Licensee MDPI, Basel, Switzerland.


  • Fingertip-based interaction
  • Gesture recognition
  • Human computer interaction
  • Virtual environment
  • Virtual reality

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Signal Processing
  • Hardware and Architecture
  • Computer Networks and Communications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Fingertip gestures recognition using leap motion and camera for interaction with virtual environment'. Together they form a unique fingerprint.

Cite this