International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 06 Issue: 01 | Jan 2019
p-ISSN: 2395-0072
www.irjet.net
HANDSPEAK SYSTEM USING ARTIFICIAL INTELLIGENCE Payal Das1, Aaditya Ghag2, Lalita Maurya3 1,2,3U.G.
Student (BE), Department of Electronics and Telecommunication Engineering, Konkan Gyanpeeth College of Engg. Karjat, Maharashtra, India ----------------------------------------------------------------------***--------------------------------------------------------------------Abstract - A man-machine interaction project is described which aims to establish an automated voice to sign language translator for communication with the deaf using integrated open technologies. The first prototype consists of a robotic hand designed with low cost self designed Acrylic assembly which smoothly reproduces the alphabet of the sign language controlled by voice only. The core automation comprises an Arduino UNO controller used to activate a set of servo motors that follow instructions from a Raspberry Pi mini-computer having installed the open source speech recognition engine Julius. We discuss its features, limitations and possible future developments.
Our Voice to SL Translator consists of a robotic hand aiming to satisfy an important fundamental human need such as face to face communication. In this way translation of the human voice into actual signs using a mechanical hand assembly with an electronic board able to command its fingers by voice inputs. 1.1 Standard Sign Language Chart
Key Words: Artificial Intelligence, Man-machine interaction, Open technologies. 1. INTRODUCTION Sign language is a visual language that is used by deaf and dumb people as their mother tongue. Unlike acoustically conveyed sound patterns, sign language uses body language and manual communication to fluidly convey the thoughts of a person. It is achieved by simultaneously combining hand shapes, orientation and movement of the hands, arms or body, and facial expressions. It can be used by a person who has difficulties in speaking or by a person who can hear but could not speak and by normal people to communicate with hearing disabled people. As far as a deaf person is concerned, having access to a sign language is very important for their social, emotional and linguistic growth. Sign language should be recognized as the first language of deaf people and their education can be proceeded bilingually in the national sign language as well as national written or spoken language. Indian Sign Language is used by deaf, dumb and hard of hearing people for communication by showing signs using different parts of body. All around the world there are different communities of deaf and dumb people and thus the language of these communities will be different.
Fig 1 : Sign language for Alphabets
The Sign Language used in USA is American Sign Language (ASL); British Sign Language (BSL) is used in Britain; and Indian Sign Language (ISL) is used in India for expressing thoughts and communicating with each other. The “Indian Sign Language (ISL)” uses manual communication and body language (non-manual communication) to convey thoughts, ideas or feelings. As we know it is not easy for a normal person to learn a Sign language to communicate with the deaf people therefore there is a need of automated system that will convert human speech into Standard Sign Language. © 2019, IRJET
|
Impact Factor value: 7.211
|
Fig 2 : Sign language for numeric ISO 9001:2008 Certified Journal
|
Page 1779
International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 06 Issue: 01 | Jan 2019
p-ISSN: 2395-0072
www.irjet.net
1.2 Block Diagram
The Raspberry Pi is a low-cost credit-card sized singleboard computer. The Raspberry Pi was created in the UK by the Raspberry Pi Foundation. The Raspberry Pi Foundation's goal is to "advance the education of adults and children, particularly in the field of computers, computer science and related subjects. Many people have used the Raspberry Pi to make things like cameras, gaming machines, robots, web servers and media centres. There are a few different versions of the Raspberry Pi, each made for different uses. All of the current versions use a microSD card for the operating system and file storage. They are powered by a micro-USB port, have one HDMI port, one audio/video jack socket, and a 40-pin GPIO connector.
Fig. 2 : Block Diagram of HandSpeak System
3.2 Arduino UNO
This system comprises of a Robotic Hand which will show the sign language gestures corresponding to the user voice input. The user have to simply speak through the USB mic. Speech Recognition will be done by the Julius AI model which is running on the Raspberry Pi Operating System. Further Raspberry Pi will send the speech input in a proper serial format to the Arduino UNO which in turn will control the Robotic Arm. 2. Related Work 2.1. Mohamed Jemni, Oussama El Ghoul, Nour Ben Yahia, Mehrez Boulares. “SIGN LANGUAGE MMS TO MAKE CELL PHONES ACCESSIBLE TO DEAF AND HARD-OF-HEARING COMMUNITY” Author proposed a system is developed to enhance communication with deaf, hard-of-hearing and speech disabled individuals. The originality of this tool consists on the use of new technologies to make the mobile phone a device of communication with deaf people and a tool for the integration of deaf to the society. They have tested this system by the Bluetooth, and they plan to test it really with an operator of telecom and to make it available as a new service of mobile phone. 3. Hardware Section 3.1 Raspberry Pi
Fig 4 : Arduino Uno Board Arduino is an open source computer hardware and software company, that designs and manufactures singleboard microcontrollers and microcontroller kits for building digital devices and interactive objects that can sense and control objects in the physical world. Arduino board designs use a variety of microprocessors and controllers. The boards are equipped with sets of digital and analog input/output (I/O) pins that may be interfaced to various expansion boards (shields) and other circuits. The boards feature serial communications interfaces, including Universal Serial Bus (USB) on some models, which are also used for loading programs from personal computers. The microcontrollers are typically programmed using a dialect of features from the programming languages C and C++. In addition to using traditional compiler toolchains, the Arduino project provides an integrated development environment (IDE) based on the Processing language project. 3.3 Mechanical Arm using servos A mechanical arm is a machine that mimics the action of a human arm. Mechanical arms are composed of multiple beams connects by hinges powered by actuators. One end of the arm is attached to a firm base while the other has a tool. They can be controlled by humans either directly or over a distance. A computer-controlled mechanical arm is called a robotic arm. However, a robotic
Fig 3 : Raspberry Pi
© 2019, IRJET
|
Impact Factor value: 7.211
|
ISO 9001:2008 Certified Journal
|
Page 1780
International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 06 Issue: 01 | Jan 2019
p-ISSN: 2395-0072
www.irjet.net
arm is just one of many types of different mechanical arms.
Thompson and Peter Green as an independent project. The initial build was completed in June 2012. The operating system is still under active development. Raspbian is highly optimized for the line’s performance ARM CPUs. 5. Working An Arduino UNO controller is set to command six servo motors following instructions received by an USB connected Raspberry Pi mini-computer with the speech recognition decoder Julius. Raspberry Pi model will run the Julius program and corresponding recognized words will be forwarded to hand movement system character by character. 6. CONCLUSION We have introduced, and evaluated the feasibility of a voice to SL translator for letters and numbers with affordable open technologies (i.e. Arduino UNO controller and Raspberry Pi mini-computer). As far as we know, our results are the first instance of a voice-controlled robotic hand specially built with the aim to help deaf individuals. To translate automatically a spoken language into a specific SL is certainly the most challenging target since SL is mainly a visual language also incorporating complex gestures, facial expressions, head movements and body language.
Fig 5: Mechanical arm using servos 4. Software Section 4.1 Julius AI model
REFERENCES [1] World Health Organization (WHO), 2013 Deafness and Hearing Loss Statistics, at : http://www.who.int/mediacentre/factsheets/fs300/ en
Julius is an open-source large-vocabulary speech recognition software used for both academic research and industrial applications. It executes real-time speech recognition of a 60k-word dictation task on low-spec PCs with small footprint, and even on embedded devices. Julius supports standard language models such as statistical Ngram model and rule-based grammars, as well as Hidden Markov Model (HMM) as an acoustic model. One can build a speech recognition system of his own purpose, or can integrate the speech recognition capability to a variety of applications using Julius.
[2] Wikipedia on "Sign Languages relation with Spoken Languages" at : http://en.wikipedia.org/wiki/Sign_language [3] Raspberry Pi credit-card sized computer at http://www.raspberrypi.org [4] The Arduino Uno microcontroller board a http://www.arduino.cc Servo.h library available at : http://arduino.cc/en/reference/servo
4.2 Raspbian OS Raspbian is Debian-based computer system for Raspberry Pi. There are several versions of Raspbian including Raspbian Stretch and Raspbian Jessie. Since 2015 it has been officially provided by the Raspberry Pi Foundation as the primary operating system for the family of Raspberry Pi single-board computers. Raspbian was created by Mike Š 2019, IRJET
|
Impact Factor value: 7.211
|
ISO 9001:2008 Certified Journal
|
Page 1781
International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 06 Issue: 01 | Jan 2019
p-ISSN: 2395-0072
www.irjet.net
BIOGRAPHIES Prof S. V. Sonar ME in Electronics and Telecommunication Engineering.
Payal Das Pursuing BE in Electronics and Telecommunication Engineering, Konkan Gyanpeeth College of Engineering, Karjat. Aaditya Ghag Pursuing BE in Electronics and Telecommunication Engineering, Konkan Gyanpeeth College of Engineering, Karjat. Lalita Maurya Pursuing BE in Electronics and Telecommunication Engineering, Konkan Gyanpeeth College of Engineering, Karjat.
Š 2019, IRJET
|
Impact Factor value: 7.211
|
ISO 9001:2008 Certified Journal
|
Page 1782