Mobile application for reading display boards having kannada text

Mobile Application for Reading Display Boards having Kannada Text Mujassama Sayed1, Shanmukhappa A. Angadi2 1

Department of Computer Science and Engineering, Visvesvaraya Technological University, PG Centre, Belagavi, Karnataka, India 2 Department of Computer Science and Engineering, Visvesvaraya Technological University, PG Centre, Belagavi, Karnataka, India,

Abstract- Mobile application development is in trend nowadays. People feel very comfortable in using mobile apps. This paper presents a mobile application for capturing display boards having Kannada text, using high resolution mobile camera of 16 megapixels and further giving its English meaning. This will be very beneficial to people who are unaware of Kannada language, so that they can easily understand what is written on display boards. The system first takes images of Kannada signboards, and segments image to line, words and character components. Aim of this application is to find the meaning of word detected. The system first reads the image from the file and then the image is passed for processing. The image is further processed by initially removing unwanted portion in the image and background noise is also removed. The image is then converted to grayscale image, which is segmented and the result is displayed in the app which is nothing but English meaning of word detected. The results are precise and accurate with the level of accuracy reached to 89.94%.

Keywords: Android, JAVA, JVM, Image processing, Neural Network.

I. INTRODUCTION Mobile Application development is a term used to describe the demonstration or procedure by making use of which application programming is produced for cell phones, for example, individual advanced aides, endeavor computerized partners or cellular telephones. These applications can be pre-introduced on telephones amid assembling stages, or conveyed as web applications utilizing server-side or customer side preparing (e.g., JavaScript) to present an "application-like" affair inside a Web program.

Application programming designers additionally should consider the screen sizes, equipment details, and arrangement in views while designing an application. As a major aspect of the advancement procedure, versatile (UI) configuration is likewise a key in the production of portable applications. Portable UI considers imperatives, settings, screen, info, and versatility as blueprints for configuration. The client is regularly the center of connection with their gadget, and the interface involves parts of both equipment and programming. Client information takes into account the clients to control a framework, and gadget's yield permits the framework to demonstrate the impacts of the clients' control.

Portable UI plan limitations incorporate restricted consideration and structure elements, for example, a mobile phone's screen size for a client's hand(s). Versatile UI connections signal prompts from client action, for example, area and booking that can be appeared from client associations inside a portable application. By and large, portable UI configuration's objective is essentially for a justifiable, easy to use interface. The UI of portable applications ought to: consider the clients' restricted consideration, minimize keystrokes, and be arranged with a base arrangement of capacities.

This usefulness is bolstered by versatile undertaking application stages or coordinated advancement situations (IDEs).

The paper presents an application in which an image is taken as input to the system. The image then undergoes several steps of image processing to get the features and then a KNN neural

@IJRTER-2016, All Rights Reserved

179

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

network classifier is used to classify the images. The image is captured using high resolution mobile camera of 16 megapixels. The system is designed using Matlab. First the color image is converted to grayscale and then into binary image. The image under goes some pre-processing steps like background removal and noise elimination. In the next phase the image is processed. Segmentation is done on image which segments the image into lines, words and characters and on each character zoning is performed. Each character is divided into 6 zones and feature extraction is done. The features extracted are like seven Invariant moment features from each zone of the character. These extracted features are given for training the classifier. The classifier used here is K- Nearest Neighbor (KNN) method. After the classifier is trained it is checked for the accuracy of classifying the new images. The test images are used to check the accuracy of the classifier implemented and finally English meaning of the word is given. An android application is developed for converting the Kannada text in the image to give equivalent English meaning.

The remaining part of the paper is organised into four parts: Literature review, explanation of Methodology, Experimental results and finally conclusion. The four parts are introduced as follows II. LITERATURE REVIEW Recently Sharath B and Manjunath A E [1] developed Optical Character Recognition system for an Android Application for reading Kannada signboards, newspapers etc. The built in camera of Android mobile is used to extract text from Kannada signboards. OCR technology is been implemented using Kohonen’s algorithm. Dr. Shobha G, B M Sagar and Dr. Ramakanth Kumar P [2] proposed OCR for printed Kannada text which can be converted to machine editable format. Here the database approach is been used for recognizing Kannada characters. Here lines are segmented into words and characters. The space present between Kannada characters are used for segmentation.

Dr. H N Prakash, Aravinda C V and Lavanya S [3] proposed Recognition system for Kannada handwritten characters using multi feature extraction techniques. Handwritten character recognition is chosen because it is very complex task to identify various writing styles of different individuals.

Here the pre-processing techniques used are Normalization and Binarization. Back propagation neural network is used as a recognition process. Here the image is first converted to editable format.

This text can then be saved or opened for future editing. The current system can also be extended to identify votaksharas. Suvog Oswal, Sagar Karkera, Darshan Ingle and Ashish Titus [4] proposed a system using OCR for Android Travel mate Applications and its translation to other language. This application is helpful for tourists and travelers who own an Android cell phone. They can easily convert Native language text to their own country language. This system uses Tesseract OCR engine and for translation to other language image processing is used. Nitin Mishra and C Patvardhan [5] developed an Android Travel mate application. Here the built-in OCR converts text in the captured image to Unicode text format. This application both recognized and translated text can be copy, paste, search and share for travel related queries like places, restaurants, hotels etc. This application use both image processing and OCR suite on Android device, so there is no computing overhead.

The studies reveal that only text conversion is done so far, no image conversion is performed. An android application which will be very useful to people in present days is also not developed.

The paper presents application developed to convert the Kannada text on Signboards to give its English meaning. In this system an image is taken as input to the system. This image then undergoes several steps of image processing to get the features and then a KNN neural network classifier is used to classify the images. The image is captured using high resolution mobile camera of 16 megapixels. First the color image is converted to grayscale and then into binary image. The image under goes some pre-processing steps like background removal and noise elimination. In the next phase the image is processed. Segmentation is done on image which segments the image into lines, words and characters and on each character zoning is performed. Each character is divided into 6 zones and feature extraction is done. The features extracted are like 7 Invariant moment features from each zone of the character. These extracted features are given for training the classifier. The

@IJRTER-2016, All Rights Reserved

180

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

classifier used here is K- Nearest Neighbor (KNN). After the classifier is trained it is checked for the accuracy of classifying the new images.

The test images are given to check the accuracy of the classifier implemented and to give its English meaning. An android application is developed for converting the Kannada text in the image to give equivalent English meaning.

III. METHODOLOGY Due to the impact and advancements in Information Technology, nowadays more emphasis is given in Karnataka State, to use Kannada language at all levels and hence the use of Kannada in Computer Systems is also a necessity. Currently there are many systems available for handling printed English documents with reasonable levels of accuracy. It is difficult to find the systems for Kannada with good accuracy.

An android application which gives Kannada to English translation for the Images is developed.

The application will be very useful for people who are not familiar with Kannada language.

A. Proposed System

Kannada Characters

Display Board

Knowledge Base & Server Application Required for Image Processing & Translation

Sufficient Light

Capture from Mobile

Mobile Application

Display of Translated Text

Figure 1: Block diagram of the system

The proposed solution builds an application which first captures the Kannada display board image from mobile camera and gives its English meaning. The RGB image is first converted to Grayscale image using rgb2gray function. After this any background noise from the image is removed and unwanted portion in the image is been cropped and removed. The pre-processed image is further segmented into lines, words and characters. On each segmented character, Zoning is performed. Each character is divided into 6 zones and from each zone seven Invariant moments features are extracted and stored in feature vector. The images in the database are trained using KNN classifier, and further recognized for reading of Kannada Text. The detailed description of Feature Extraction and Classification is summarized in following paper.

@IJRTER-2016, All Rights Reserved

181

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

B. Feature Extraction Each of the segmented images is then subjected to feature extraction process. Feature extraction is done based on zoning. Each character is divided into 6 zones and from each zone 7 Invariant moment features are extracted. So total of 42 features are extracted from each character image. After zoning is been done, each of the separated characters are stored in one folder which are named meaningfully.

Figure 2: Features extracted from a character

C. Classifier (K-Nearest Neighbor) The classifier is used for training and testing of the images. K-nearest neighbor classifier assumes every instance corresponding to some points in n-dimensional space. The nearest neighbors are the instances which are formulated in terms of standard Euclidian distance. In this K-nearest neighbor learning the target function can be either discrete valued or real value.

The experimentation is carried for 13 images. So there are 199 different characters to train the network using KNN. The features from each character are extracted for training and a knowledge base is created. Once the neural network is trained the next step is to test the images IV. EXPERIMENTAL RESULTS An android app is been designed which selects the Image from the gallery and its English meaning is been given. The captured images will be stored in the gallery of the mobile. The image is been selected from the folder in gallery to which English conversion is needed. Once we select the required image, its English meaning is been displayed in the application. The flow of android application developed is shown below.

@IJRTER-2016, All Rights Reserved

182

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

Figure 3: Loading the Image

Figure 4: selecting folder from gallery

@IJRTER-2016, All Rights Reserved

183

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

Figure 5: Selecting the Image

Figure 6: Submitting the Image

@IJRTER-2016, All Rights Reserved

184

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

Figure 7: Displaying English meaning

The below table (Table 1) shows the analysis of some testing images. The results are extremely good. Misclassification occurs if the images contain any distortions or any background noise.

Scene Word Image Expected as Recognizes as Err

@IJRTER-2016, All Rights Reserved

A ti thi

6

A ti thi

6

0

ku l s chi v ru

10

ku l s chi v ru

10

0

U p ha r

5

U p ha r

5

0

gra h

4

gra h

4

0

185

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

@IJRTER-2016, All Rights Reserved

Aa d Li t

6

Aa d Aa t

4

2

vi bha g

6

vi bha g

6

0

Ku l p ti g laa

10

Ku l p ti g laa

10

0

v s ti

4

v s ti

4

0

gra h ke

6

gra h ke

6

0

da ri

4

da ri

4

0

Pra de shi k

9

Pra de E shi k

10

1

k che ri

6

K che ku ri

8

2

kri daa

6

kri l daa

7

1

san ki rna

8

san ki l n raa

10

2

E di sel

6

E chi di chi sel

12

6

graanthalya

11

graanthalya

11

0

Hankasu

7

Hankasu

7

0

186

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

Athikariglu

11

Athikariglu

11

0

Kulptiglaa

10

Kulptiglaa

10

0

Schivalya

9

Schivalya

9

0

Pradeshik

9

pradelaashik

12

3

Kcheri

6

Kchekurip

9

3

belaagavi

9

Belaagavi

9

0

Vlya

4

Vlya

4

0

Sthanik

7

Sthanik

7

0

abhiyaantr

10

Abhiyaantr

10

0

Vibhag

6

Vibhag

6

0

Table 1: Performance Analysis of Testing Images

Total characters considered : 199 Error characters : 20 Total correctly recognized characters: 199-20 = 179 Accuracy: 179/199*100 = 89.9497% The overall performance of the system is approximated to 89.94%. Figure 8 gives the performance graph of the system.

@IJRTER-2016, All Rights Reserved

187

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457]

Figure 8: Performance Graph

The graph is plotted using number of characters versus its accuracy. From a total of 199 different characters considered, 20 error characters were found giving an overall accuracy rate of 89.94%.

V. CONCLUSION An Android Application is built which will be helpful for users who are not familiar with Kannada language. The implementation uses K-Nearest Neighbor neural network for better accuracy and speed.

The application makes use of already defined characters to train the network. The user makes use of built in camera on the device to take a picture containing Kannada characters or words. This application will convert the Kannada characters or words on the signboards, to give its English meaning.

The system is implemented to work on an Emulator. In future the system can be made as an application for mobile devices so that the users can take it along with them wherever they go.

REFERENCES 1.

2.

3.

4.

5.

6.

7.

Manjunath A E, Sharath B, January 2013, “Implementing Kannada Optical Character Recognition on the Android Operating System for Kannada Sign Boards” International Journal of Advanced Research in Computer and Communication Engineering, Volume 2, Issue 1.

B.M. SAGAR, Dr. SHOBHA G, Dr. RAMAKANTH KUMAR P, June 24-26 2008, “OCR for printed Kannada text to Machine editable format using Database approach” 9th WSEAS International Conference on AUTOMATION and INFORMATION (ICAI'08), Bucharest, Romania.

Aravinda.C.V, Dr.H.N.Prakash, Lavanya S, October 2014, “Kannada Handwritten Character Recognition Using Multi Feature Extraction Techniques” International Journal of Science and Research (IJSR), Vol 3, Iss 10.

Ashish Titus, Sagar Karkera, Suyog Oswal, Darshan Ingle, March 2015, “Android Travelmate Applications with OCR & Its Language Translation” International Journal of Advanced Research in Computer Science and Software Engineering, Vol 5, Iss 3.

Nitin Mishra, C Patvardhan, July 2012, “ATMA: Android Travel Mate Application” International Journal of Computer Applications (0975 – 8887, Vol 50- No. 16.

Mamatha H R, Sucharitha S, Srikanta Murthy K, December 2011,” Multi-font and Multi-size Kannada Character Recognition based on the Curvelet and Standard Deviation” International Journal of Computer Applications (0975 – 8887), Vol 35- No.11.

Sravan Ch, Shivanku Mahna, Nirbhay Kashyap, April 2015, “Optical Character Recognition on Handheld Devices” International Journal of Computer Applications (0975 – 8887, Vol 115- No. 22.

@IJRTER-2016, All Rights Reserved

188

International Journal of Recent Trends in Engineering & Research (IJRTER) Volume 02, Issue 08; August - 2016 [ISSN: 2455-1457] Amit Dhanwani, Ruhi Bajaj, May 2014, “WORLD TOURISTER – AN ANDROID APPLICATION AS TRAVEL MATE” Proceeding of 9th IRF International Conference.

9. Saiprakash Palakollu, Renu Dhir, Rajneesh Rani, October 2012, “Handwritten Hindi Text Segmentation Techniques for Lines and Characters” Proceedings of the World Congress on Engineering and Computer Science, Vol 1.

10. J.Pradeep, E.Srinivasan and S.Himavathi, Feb 2011, “DIAGONAL BASED FEATURE EXTRACTION FOR HANDWRITTEN ALPHABETS RECOGNITION SYSTEM USING NEURAL NETWORK” International Journal of Computer Science & Information Technology (IJCSIT), Vol 3- No. 1.

11. Vikas J Dongre, Vijay H Mankar, August 2011, “DEVNAGARI DOCUMENT SEGMENTATION USING HISTOGRAM APPROACH” International Journal of Computer Science, Engineering and Information Technology (IJCSEIT), Vol 1- No.3.

12. Ayatullah Faruk Mollah, Nabamita Majumder Subhadip Basu and Mita Nasipur, July 2011, “ Design of an Optical Character Recognition System for Camera-based Handheld Devices” IJCSI International Journal of Computer Science Issues, Vol 8, Iss 4.

13. Rachana R. Herekar, Prof. S. R. Dhotre, Jul-Aug 2014, “Handwritten Character Recognition Based on Zoning Using Euler Number for English Alphabets and Numerals” IOSR Journal of Computer Engineering (IOSR-JCE), Vol 16, Iss 4.

14. C Naveena, V.N. Manjunath Aradhya, 2012, “Handwritten Character Segmentation for Kannada Scripts” 2012 World Congress on Information and Communication Technologies.

15. Minoru Maruyama and Takuma Yamaguchi, 2009,” Extraction of characters on signboards in natural scene images by stump classifiers” 10th International Conference on Document Analysis and Recognition.

8.

@IJRTER-2016, All Rights Reserved

189