Image Based Sign Language Recognition on Android

A mediator person is required for communication between deaf person and a second person. But a mediator should know the sign language used by deaf person. But this is also not possible always since there are multiple sign languages for multiple languages. It is difficult for a deaf person to understand what a second person speaks. And therefore deaf person should keep track of lip movements of second person in order to know what he is speaking. But the lip movements do not give proper efficiency and accuracy since the facial expressions and speech might not match. To overcome the above problems we have proposed a system, an Android Application for recognizing sign language using hand gesture with the facility for user to define and upload their own sign language into the system. The features of this system are the real time conversion of gesture to text and speech. For two-way communication between deaf person and second person, the speech of second person is converted into text. The processing steps include: gesture extraction, gesture matching and conversion of text to speech and vice-versa. The system is not only useful for deaf community but can also be used by common people who migrate to different regions and do not know local language.