Operating Computer Cursor Using Eye And Face Movements

The advent of modern human computer interfaces has seen a considerable progress in Hands-free Human Computer Interaction (HCI) solutions. This project focuses on developing a methodology to facilitate computer cursor control for people with physical disabilities such as Quadriplegics and amputees. The proposed methodology takes real-time video input from the user using OpenCV and performs face recognition. The 68-point landmark algorithm is used to locate the various facial features which can be used for cursor control. Opening/closing the mouth based on Mouth Aspect Ratio (MAR) indicates activation/deactivation of the cursor control. The nose tip is used for controlling and moving the cursor in all 4 directions by moving the head left, right, up and down. Eye Aspect Ratio (EAR) is used to detect eyes and eye flickering. Left and right eye blinks indicate left and right clicks respectively. Squinted eyes indicate scrolling of pages, which is beneficial while working with PDFs and other such documents. The proposed system requires very basic requirements like webcam and a few Python libraries such as OpenCV, Numpy, imutils, dlib and PyAutoGUI. Thus it would help the physically disabled users to efficiently use the computer, thus eliminating the need of a physical mouse interaction.