Joydeep Paramanik F&LA, Semester VI Roll No. 07 NIFT Kolkata 1 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
ABSTRACT
Keywords: Deaf and Dumb People, Sign Language, Gesture Recognition, Accelerometer Sensor. “Speech” and “gestures” are the expressions, which are mostly used in communication between human beings. Learning of their use begins with the first years of life. In human communication, the use of speech and gestures is completely coordinated. Every normal human being sees, listens and then reacts to the situations by speaking himself out. But there are some unfortunate ones who are deprived of this valuable gift. This creates a gap between the normal human beings and the deprived ones. For deaf people, communicating with hearing people is often a struggle, involving writing things out, hand gestures, lip reading and confusion all around. It’s an idea to translate sign language into audio so deaf people can communicate with hearing people.
2 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
ACKNOWLEDGEMENT
I take this opportunity to express my profound gratitude and deep regards to various people for their guidance and help. Firstly, I would like to thank my mentor, Professor Chirapriya Mondal, for his patient guidance, enthusiastic encouragement and useful critiques on this research work. His precious and timely inputs helped me in keeping the focus and working in the proper direction which was much required for completion of the project. I am also thankful to my friends and batchmates for their constant support and encouragement. Special thanks should be given to our institute, for including such an interesting project in our curriculum. Finally, I wish to thank my family for their support, blessing and care along with the continuous encouragement to face all the challenges in life.
3 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
CONTENTS ABSTRACT ......................................................................................................................................... 2 ACKNOWLEDGEMENT ...................................................................................................................... 3 Chapter 1 .......................................................................................................................................... 6 Introduction ...................................................................................................................................... 6 1.1
Background ....................................................................................................................... 6
1.2
Problem Identification: Lack of communication gap with Deaf and Dumb people ......... 6
1.3
Causes of Deaf and Dumbness ......................................................................................... 7
1.4
Aim of the Research ......................................................................................................... 7
1.5
Research Methodology .................................................................................................... 7
1.6
Organization of Chapters.................................................................................................. 7
1.7
Need Statement ............................................................................................................... 8
1.8
Conclusion ........................................................................................................................ 8
Chapter 2 .......................................................................................................................................... 9 Literature Review ............................................................................................................................. 9 ...................................................................................................................................................... 9 Introduction .................................................................................................................................. 9 2.1
About Communication ..................................................................................................... 9
2.2
Types of Communication ................................................................................................ 10
2.3
Role of Communication .................................................................................................. 10
Chapter 3 ........................................................................................................................................ 12 Study of existing Communicator between Deaf and Dumb People and Normal People ............... 12 3.1
Need for communicator device for Deaf and Dumb People .......................................... 12
3.2 Study of types of existing Communicator between Deaf and Dumb People and Normal People 13 3.3
Conclusion ...................................................................................................................... 15
3.4
Limitations related to existing device ............................................................................ 15
Chapter 4 ........................................................................................................................................ 16 Research Analysis and Conclusion.................................................................................................. 16 4.1
Introduction .................................................................................................................... 16
4.2
Stakeholders ................................................................................................................... 16
4.2.1 4.3
User......................................................................................................................... 16
Main observation of user study...................................................................................... 17
4 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
4.4
Conclusion ...................................................................................................................... 17
Chapter 5 ........................................................................................................................................ 18 Design Phase................................................................................................................................... 18 5.1
Design Requirements ..................................................................................................... 18
5.2
Design specifications ...................................................................................................... 18
5.3
About the product ...................................................................................................... 18
Chapter 6 ........................................................................................................................................ 20 Scope of Future Work..................................................................................................................... 20
5 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
1
Introduction 1.1
Background This project is mainly inspired from one of the incident which happened with me while traveling in bus. I was coming to college from Shyambazar and mistakenly I sat on a physical handicap sit and suddenly after tow – three stoppages a man came, stand in front of me and tried to say something but I couldn’t understand any. After a minute I realized that I was sitting in a physical handicap sit and that person want to sit there. This was a small incident but that day I realized how hard it is to explain anything for them. So I started this project.
1.2
Problem Identification: Lack of communication gap with Deaf and Dumb people Dumb people are usually deprived of normal communication with other people in the society. It has been observed that they find it really difficult at times to interact with normal people with their gestures, as only a very few of those are recognized by most people. Since people with hearing impairment or deaf people cannot talk like normal people so they have to depend on some sort of visual communication in most of the time. The gap between normal human beings and deaf and dumb is wide and ever increasing day-by-day. Today, the national count of hearing disabled and mute persons throughout India is approximately 17 lakh. Despite of this large number, very less research is done in order to bridge the communication barrier
6 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
1.3
Causes of Deaf and Dumbness Deaf and Dumbness can occur for a number of reasons:a) Deaf and Dumb from birth b) Deaf and Dumb by accident In both the above cases, the problem is communicating with normal people is very difficult.
1.4
Aim of the Research The aim of the research is bridge the communication barrier, we propose an Application which helps normal and deaf and dumb people to effectively communicate with each other. Human Computer Interaction and Image Processing are the related areas of research which will help us build a solution to this problem.
1.5
Research Methodology Product research from users is an approach towards developing a user centered product. The methodologies are inherent to understand the needs of the user and analyze them to create solutions in the form of some devise. This application will help for both of them to communicate with each other. The system is mainly consists of two modules, first module is drawing out Indian Sign Language(ISL) gestures from real-time video and mapping it with human-understandable speech.
1.6
Organization of Chapters  Chapter 2 briefly describes about communication and its types.
7 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter 3 includes a study on the need for communicator device for Deaf and Dumb People and Study of types of existing Communicator between Deaf and Dumb People and Normal People. Limitations related to existing communicator device. Chapter 4 discusses about the survey of the stakeholders and their findings. It tries to understand the inherent problems in the present types of furniture and also understand the requirements of the users, i.e. residents of studio apartments. Chapter 5 is a culmination of the previous chapters. It includes the conceptualization of the product. Finally Chapter 6 speaks the further possibilities of the research.
1.7
Need Statement To design a product/device/system that will help the deaf and dumb people who are unable to communicate, or having difficulties in communication. This device helps in communication if attached to both the person involved in the communication who may be deaf, dumb, and Normal person.
1.8
Conclusion The deaf and dumb people use their standard sign language which is not easily understandable by common people. This system converts the sign language into voice which is easily understandable by the people. The sign language is translated into text form, to facilitate the deaf people to convey their messages as well to the others.
8 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
2
Literature Review Introduction Communication is simply the act of transferring information from one place to another. Although this is a simple definition, when we think about how we may communicate the subject becomes a lot more complex. There are various categories of communication and more than one may occur at any time.1 2.1
About Communication The process of interpersonal communication cannot be regarded as phenomena which simply 'happens', but should be seen as a process which involves participants negotiating their role in this process, whether consciously or unconsciously. Senders and receivers are of course vital in communication. In face-toface communication the roles of the sender and receiver are not distinct as both parties communicate with each other, even if in very subtle ways such as through eye-contact (or lack of) and general body language. There are many other subtle ways that we communicate (perhaps even unintentionally) with others, for example the tone of our voice can give
1
http://www.skillsyouneed.com/general/what-is-communication.html#ixzz471HsnESe 9 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
clues to our mood or emotional state, whilst hand signals or gestures can add to a spoken message. In written communication the sender and receiver are more distinct. Until recent times, relatively few writers and publishers were very powerful when it came to communicating the written word. Today we can all write and publish our ideas online, which has led to an explosion of information and communication possibilities 2.2
Types of Communication The different categories of communication include: Spoken or Verbal Communication: face-to-face, telephone, radio or television and other media. Non-Verbal Communication: body language, gestures, how we dress or act - even our scent. Written Communication: letters, e-mails, books, magazines, the Internet or via other media. Visualizations: graphs and charts, maps, logos and other visualizations can communicate messages.
2.3
Role of Communication Communication is a process of sending and receiving information among people. Humans communicate with others not only by face-toface communication, but also by giving information via the Internet and printed products such as books and newspapers. Many people believe that the significance of communication is like the importance of
10 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
breathing. Indeed, communication facilitates the spread of knowledge and forms relationships between people.2
2
www.hopespeak.com 11 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
3
Study of existing Communicator between Deaf and Dumb People and Normal People 3.1
Need for communicator device for Deaf and Dumb People Deaf and dumb is a term means a person who could not either hear or both hear and speak. The number of deaf and dumb in the world continuously increasing and they are introverted closed society. The education of the deaf is only about one century old. Since sign is the earliest way of communication in the world when there is no appropriate language, so the sign language is preferred among the deafdumb people for education. As with other forms of manual communication, Sign language depends on finger spelling. The simplest visual form of finger spelling is simulating the shape of letters in the air, or tactually, tracing letters on the hand. Finger spelling can use one hand such as in American Sign Language, French Sign Language and Irish Sign Language, or can use two hands such as in British Sign Language. Uneducated Deaf-Dumb people can communicate with other people (normal or handicaps) with sign language only, so they face serious problems in their daily life. For example: restaurants, transportation, hospitals, government offices...etc. Therefore, they need an effective
12 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
tool to translate their words from sign language to Arabic or English language directly. This tool can facilities their communication with normal people and encourage them to learn both Arabic and languages. Also, Deaf-Dumb kids needs to learn sign, Arabic and English languages in an interesting way. For the above reasons, the motivation of our application is to offer a service to the society in general and to Deaf-Dumb people in particular. This work is an integrated system that can easily solve most of their problems in one application. Therefore, the present work aims to: - Help deaf and dumb to interact more with normal people - Offer a great tool for parents to teach their deaf and dumb kids - Introduce Sign language keyboard. - Introduce quizzes and games for training deaf and dumb to identify Arabic and English words. 3.2
Study of types of existing Communicator between Deaf and Dumb
People and Normal People Uses a glove fitted with sensors that can interpret the 26 English letters in American Sign Language (ASL). The glove uses MEMS sensors and accelerometers in different dimensions to gather data on each finger’s position and the hand’s motion to differentiate the letters.
13 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Figure 1
Figure 2 Flex sensor attached to glove in the hand is used to perform Gesture by deaf and dumb people.
Figure 3 System Diagram
14 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
3.3
Conclusion Maximum product is not available in the market; they are only at a level of research. Some product which are there in the market cost too much and not so easy to handle.
3.4
Limitations related to existing device Always required some extra big device attached like a gloves or some kind of monitor.
Figure 4 Voice chip APR 9600 with voice output
Figure 5 a person using the device
15 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
4
Research Analysis and Conclusion 4.1
Introduction
In continuation of the previous chapter that essentially talks about different types of communicator now tries to explore and present the data gathered from observations and through interview findings. This chapter first identifies the various stakeholders.
4.2
Stakeholders
Normal people/User and Deaf and Dumb people
4.2.1
User
Users can be defined as people that has authority to use an application, equipment, facility, process, or system, or one who consumes or employs a good or service to obtain a benefit or to solve a problem, and who may or may not be the actual purchaser of the item.3 The visually impaired people to communicate with speech impaired people, since the former cannot see the latter using the sign language, there is currently no means of easy and affordable way of communication between such people who are unfortunately in significantly large numbers in a country such as India.
3
http://www.businessdictionary.com/definition/user.html 16 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
4.2.2
Deaf and Dumb people
Same as the normal people the deaf and dumb people will be freer to communicate with and any people and this can make their life much better.
4.3
Main observation of user study
The device should be small in size and easy to use Existing device are heavy and have extra outfit. The most preferred to make something which can be attached with their mobile only.
4.4
Conclusion
All the stakeholders play distinct and important role. In many instances the research questions overlap with one another. This shows the common interest, significance and shared role of the individual stakeholders. It is highly desirable that all of them perform accordingly mentioned in a prescription.
17 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
5
Design Phase The design phase inculcates the user experience process to conceptualize the product. It involves the conceptualization of the product according to the needs of the user. The users are normal people as well as the Deaf and Dumb people. The initial user testing takes the feedback of both the stakeholders. The designed product is deemed to cater the needs of the users in a satisfactory and efficient way.
5.1
Design Requirements
The design requirement needs to be simplified. It has to meet the essential requirement of the users. Design requirements of the product: Combination of functions – May be some kind of app in Google Play Store which can translate Indian Sign Language gestures
5.2
Design specifications
The product will fulfill a primary function of translating Indian Sign Language gestures by conversion of speech.
5.3
About the product
Making the normal person able to communicate successfully with the deaf and dumb person. The normal human being is capable of giving natural language as input from his side. So, in order to process this natural language, 18 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
the first step is to generate text data out of audio speech data. It is easy to process to text data as compared to audio speech signals. The Google SST (Speech to Text) API provides us with all the functionalities and provisions to convert audio speech to corresponding text for further processing. Now there should be some app which can convert the Indian Sign Language gestures in audio format or in text.
5.4
Conclusion
The proposed system Two Way Communication between Deaf and Dumb People and Normal People aims at bridging the communication gap between two strata of the society. This paper adds in complete both way communication in an efficient manner because the system will be implemented as a handy mobile application. So, it truly serves its needs in all aspects.
19 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA
Chapter
6
Scope of Future Work Further enhancements, can be made in terms of implementing the device with other sign languages like American Sign Language, accent recognition for various ascents throughout the globe, emotion recognition in sign language and language translation.
References 1. Aditi Kalsh, N.S. Garewal Sign Language Recognition for Deaf and Dumb International Journal of Advanced Research in Computer Science and Software Engineering Volume 3, Issue 9, September 2013 ISSN: 2277 128X. 2. http://www.engineersgarage.com//microcontroller 8051project 3. http://www.scribd.com//Hand Talk Assisting system 4. Venketraman.s,T.V.Padmavathi,”speech for the disabled”
20 RESEARCH ON COMMUNICATION BETWEEN DEAF AND DUMB PEOPLE AND NORMAL PEOPLE JOYDEEP PARAMANIK SEMESTER VI F&LA NIFT KOLKATA