sm1603023

Deepak Sharma, International Journal of Advanced Trends in Computer Applications (IJATCA) Volume 3, Number 2, February - 2016, pp. 5-9 ISSN: 2395-3519

International Journal of Advanced Trends in Computer Applications www.ijatca.com

Review Paper on Zebra Crossing and Traffic Signals Using Mobile Phone Camera for Blind Persons 1

Deepak Sharma

Assistant Professor (CS) Punjab University ddbbpucc@gmail.com

Abstract: A computer vision-based system for pathfinding and navigation aid could enhance the mobility of visually impaired as well as blind people to travel independently. In such a way that it identifies zebra-crossing lights in the surroundings as well as gives opinion about the present period of the crucial light. Because of the constrained resources of mobile devices extremely proficient and exact calculations must be produced to guarantee the unwavering quality and the intuitiveness of the framework. In this paper, a comprehensive survey has been done on zebra crossing, and traffic signals using Mobile Phone camera.

Keywords: Blind, visually impaired, navigation, traffic lights, zebra crossing, mobile-phones, GPS.

I. INTRODUCTION Active interactions and travelling independently with the active surrounding environment are well known to exhibit noteworthy difficulties for people with serious vision hindrance, in this manner diminishing personal satisfaction and compromising security. So as to enhance the capacity of individuals who are visually impaired or have huge visual debilitations to get to, comprehend, and investigate encompassing situations, numerous assistant technologies and gadgets have been created to finish particular navigation objectives, obstacle identification, or path-finding jobs.

In this work, a deep study is done on mobile devices is shown which helps groups of people with visual hindrance cross streets with adjacent traffic-lights. Since guide dogs are excessively costly and zebra crossing lights are once in a while outfitted with acoustic or haptic signs, little mobile phones offer a low-priced and handy substitute.

Numerous electronic mobility assistant frameworks are produced in view of changing over sonar data into an audible-signal for the blind persons to translate [1, 2, 3, 4, 5]. On the other hand, they just give restricted data.

As of late, several analysts have concentrated on deciphering the visual data into high level representation before directing it towards the blind or visually impaired persons.

1.1 Traffic intersections Traffic intersections are among the most perilous spots that visually impaired and blind travelers experience.

Standard orientation along with mobility techniques (for example utilizing the white cane to decide the area of the restraint_cut adjoining a crosswalk, or pay attention to various traffic_sounds to surmise the timing of the traffic_lights) provide valuable information, and at some intersections accessible_pedestrian_signals (APS) [6] make available walk_light timing data in a perceptible form which a person can hear and understand.

Inappropriately, much essential data about the design of the intersection, the exact timing of the traffic_lights, as well as the explorerâ€&#x;s orientation along with alignment in respect to a wanted crosswalk might be troublesome or impossible for him/her to determine without vision.

1.2 Global Positioning System One specific technology, which is very useful at traffic intersections for self-localization as well as at several other places is mobile_GPS. While mobile global positioning system localization accurateness is adequate to figure out which traffic intersection the wanderer is standing at (and at times even which corner of the specific intersection), it is mainly restricted in urban_settings, trees, close by big structures, as well as transport stops, where blunders because of reflections and signal dropouts avoid exactness being superior than within a limited street addresses [7]; accordingly mobile GPS can't offer direct help with knowing where it is safe to cross.

Computer_vision is a characteristic decision of innovation to construe the presence and location of zebra crossing in the travelerâ€&#x;s environment, since this function nicely complements the information provided

www.ijatca.com

5

Deepak Sharma, International Journal of Advanced Trends in Computer Applications (IJATCA) Volume 3, Number 2, February - 2016, pp. 5-9 ISSN: 2395-3519

by mobile GPS. There is a variety of work (e.g., [8]) on computer vision-based self-localization, including a system [9] that infers location based on the detailed appearance of the skyline and a real-time system [10] that infers location from panoramic images on mobile devices. However, such systems require the use of detailed 3D models of urban surroundings that are unpredictable, memory-concentrated and might be hard to secure.

Rather than depending on a perplexing 3-Dimensional model we chose to harness a basic and promptly accessible 2-Dimensional model of the urban environment, which is based on satellite imagery for example, it could possibly be acquired easily from Google_Maps. Along with its straightforwardness, the 2Dimensional picture model of the specific intersection has the included point of preference of providing metric data with respect to the particular features, which matters to a visually-impaired or blind traveler – to be specific, crosswalk marking designs.

projects have the limitation that they analyze images one at a time. This limitation forces the user to capture the entire crosswalk of interest in a single video frame, which can be challenging for users who donâ€&#x;t have enough vision to know where to point the camera. In this paper we describe work in progress that tackles the problem of crosswalk detection and self-localization, building on recent work.

II. RELATED WORK Coughlan et al. [16] developed a method of finding crosswalks based on figure-ground segmentation, which they built in a graphical model framework for grouping geometric features into a coherent structure.

Ivanchenko et al. [17] further extended the algorithm to detect the location and orientation of pedestrian crosswalks for a blind or visually impaired person using a cell phone camera. The prototype of the system can run in real time on an off-the-shelf Nokia N95 camera phone. The cell phone automatically took several images per second, analyzed each image in a fraction of a second and sounded an audio tone when it detected a pedestrian crosswalk.

Advanyi et al. [18] employed the Bionic eyeglasses to provide the blind or visually impaired individuals the navigation and orientation information based on an enhanced color preprocessing through mean shift segmentation. Then detection of pedestrian crosswalks was carried out via a partially adaptive Cellular Nanoscale Networks algorithm.

(a)

Se et al. [19] proposed a method to detect zebra crosswalks. They first detected the crossing lines by looking for groups of concurrent lines. Edges were then partitioned using intensity variation information.

(b) Fig.1. a) Camera cell phone held by blind user. (b) Schematic diagram shows overhead view of zebra crosswalk and two users holding cell phone system: cell phone on left is aligned with crosswalk and makes an audio tone; cell phone on right is not aligned and makes no sound

By contrast, a 3-Dimensional model consisting of features extracted from buildings, vegetation and other environmental structures is likely to contain few crosswalk features, which necessitates additional information to locate the crosswalk features relative to the model.

Past work on the Crosswatch project [11, 12, 13, 14] and the work of Ahmetovic et al. [15] also analyze 2D images to detect and locate zebra crossing, but these

Uddin et al. [20] proposed a bipolarity-based segmentation and projective invariant-based method to detect zebra crosswalks. They first segmented the image on the basis of bipolarity and selected the candidates on the basis of area, then extracted feature points on the candidate area based on the Fisher criterion. The authors recognized zebra crosswalks based on the projective invariants.

Omachi et al. [21] proposed an image-based traffic sign detection method using image shape and color information. They further improved their method by adding the traffic sign structure based on Hough transform as a critical factor [22].

www.ijatca.com

6

Deepak Sharma, International Journal of Advanced Trends in Computer Applications (IJATCA) Volume 3, Number 2, February - 2016, pp. 5-9 ISSN: 2395-3519

Charette and Nashashibi implemented a real time image processing system for traffic sign based on the generic “adaptive templates” [23].

Alefs and Eschemann designed a computer vision-based system for reliable road sign detection [24]. The detection system is based on the method of feature selection and matching using edge orientation histograms.

Lausser et al. [4] introduced a visual zebra crossing detector based on the Viola-Jones approach.

Shoval et al. [25] discussed the use of mobile robotics technology in the Guide-Cane device, a wheeled device pushed ahead of the user via an attached cane for the blind to avoid obstacles. When the Guide-Cane detects an obstacle, it steers around it. The user immediately feels this steering action and can follow the GuideCane's new path. Tian's group has developed a series of computer vision-based methods for blind people to independently access and navigates unfamiliar environments.

processing required for locating and detecting the status of zebra crossing signals in the immediate environment.

As real-time image processing is demanding in terms of computational resources, mobile devices with limited resources fall short in achieving accurate and timely detection. The development of a mobile vision system to detect zebra crossing lights very accurately is challenging due to several real world conditions. The chosen mobile capture device limits the possibilities of computer vision algorithms in several aspects: a. The resolution of the capture device is relatively low. b. Mobile devices often provide only poor image quality, e. g. falsified colors and unsharpened images due to automatic white balance and auto focus. c. Computation power and memory resource are restricted.

Not only the capture device, but also the objects to be captured impose restrictions by design and location:

III. ZEBRA CROSSING IDENTIFICATION A zebra crossing is a set of parallel, uniformly painted, white stripes on a dark background. The gaps separating the white stripes are “dark stripes”. Each stripe is a rectangle or, in case of diagonal crossings, a parallelogram. United States regulation dictates that zebra crossings be at least 6ft (180cm) wide, with white stripes 6in (15cm) to 24in (60cm) thick. The thickness of the dark stripes is not regulated.

Multiple types of road surface markings are used across the world to define zebra crossings. In the United States, at least two different types of zebra crossing markings are available1. The transverse marking consists of two white lines, perpendicular to the road direction, with width between 6in (15cm) and 24in (60cm). The separation between the two lines is at least 6ft (180cm).

Zebra crossings, known as “continental crossings” in USA, can be visually detected at larger distances than other crosswalk markings in the same illumination [26].

Thus, zebra crossings are common when the crossing visibility is paramount for the zebra crossings‟ safety, for example near schools and hospitals. As such, they also inform drivers to pay more attention to the crosswalk, which is desirable for zebra crossings with visual impairments that cannot rely on sight for noticing incoming vehicles.

1. Zebra Crossing Guidance Challenges The inherent difficulty of providing real-time crossing guidance to blind users at intersections is the fast image

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 2 Challenges of detecting traffic lights in images: (a) minimal distance, (b) maximal distance, (c+d) two traffic lights, (e) occlusion, and (f) rotation ([27])

Pedestrian lights have different appearances in different countries and even for different manufactures as shown in figure below.

1. The distance to the zebra crossing lights could vary between approximately 4 and 24 meters. Therefore, the scale of a traffic light in an image is very small (see Fig. 2(a) and (b)).

www.ijatca.com

7

Deepak Sharma, International Journal of Advanced Trends in Computer Applications (IJATCA) Volume 3, Number 2, February - 2016, pp. 5-9 ISSN: 2395-3519

2. There may be many traffic lights in the image but only one is crucial (see Fig. 2(c) and (d)).

situation changes. Blind users are the final decision makers who make travel decision and react to local events within the range of several meters. In this paper, a brief study has been done on previous work done by several researches in the field of visual impaired or blind people for navigation purpose.

REFERENCES

(a)

(b)

(c) Fig. 3 Difficult illumination: (a) dusk, (b) front lighting, and (c) night

Sight and light conditions may complicate the traffic light detection: 3. Traffic lights can be temporarily occluded by vehicles (see Fig. 2(e)).

4. Traffic lights could be hardly visible in bad weather situations like fog, heavy rain or snowfall.

5. The illumination condition varies between night and daylight. Thus, the captured colors of one traffic light depend on the capture time (see Fig. 3).

Finally, the user of the system could hold the mobile capture device in an unfavorable position: 6. The image could have been captured with a nonneglected rotation (see Fig. 2(f)).

7. Another major challenge faced is the short battery life of the mobile device. A continuous video recording approach to the problem exhausts the battery of the mobile device too soon, causing service interruption.

CONCLUSION Instead, a navigation aid can help blind users to gain improved perception and better understanding of the environment so that they can aware the dynamic

1. Bousbia-Salah, M.; Redjati, A.; Fezari, M.; Bettayeb, M. An Ultrasonic Navigation System for Blind People, IEEE International Conference on Signal Processing and Communications (ICSPC), 2007; pp. 1003-1006.

2. Kao, G. FM sonar modeling for navigation, Technical Report, Department of Engineering Science, University of Oxford. 1996.

3. Kuc, R. A sonar aid to enhance spatial perception of the blind: engineering design and evaluation, IEEE Transactions on Biomedical Engineering, Vol. 49 (10), 2002; pp. 1173–1180.

4. Laurent B.; Christian, T. A sonar system modeled after spatial hearing and echolocating bats for blind mobility aid, International Journal of Physical Sciences, Vol. 2 (4), April, 2007; pp. 104-111.

5. Morland, C.; Mountain, D. Design of a sonar system for visually impaired humans, The 14th International Conference on Auditory Display, June, 2008.

6. Barlow JM, Bentzen BL, Tabor L. National Cooperative Highway Research Program. 2003. Accessible pedestrian signals: Synthesis and guide to best practice.

7. Brabyn JA, Alden A, Haegerstrom-Portnoy G, Schneck M. GPS Performance for Blind Navigation in Urban Pedestrian Settings. Proceedings, Vision; 2002; Goteborg, Sweden. Jul, 2002.

8. Zhu Z, Oskiper T, Samarasekera S, Kumar R, Sawhney HS. Real-time global localization with a pre-built visual landmark database. Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on; IEEE. 2008. Jun, pp. 1–8.

9. Ramalingam S, Bouaziz S, Sturm P, Brand M. Skyline2gps: Localization in urban canyons using omniskylines. Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on; IEEE. 2010. Oct, pp. 3816–3823.

10. Arth C, Klopschitz M, Reitmayr G, Schmalstieg D. Real-time self-localization from panoramic images on mobile devices. Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on; IEEE. 2011.

Oct, pp. 37–46.

11. Coughlan J, Shen H. The Crosswatch Traffic Intersection Analyzer: A Roadmap for the Future. 13th International Conference on Computers Helping People with Special Needs (ICCHP „12); Linz, Austria. July 2012.

12. Coughlan J, Shen H. Crosswatch: a System for Providing Guidance to Visually Impaired Travelers at Traffic Intersections. Special Issue of the Journal of Assistive Technologies (JAT) 2013;7(2) 13. Ivanchenko V, Coughlan J, Shen H. Staying in the Crosswalk: A System for Guiding Visually Impaired Pedestrians at Traffic Intersections. Association for the

www.ijatca.com

8

Deepak Sharma, International Journal of Advanced Trends in Computer Applications (IJATCA) Volume 3, Number 2, February - 2016, pp. 5-9 ISSN: 2395-3519 Advancement of Assistive Technology in Europe (AAATE 2009); Florence, Italy. Sept. 2009.

14. Ivanchenko V, Coughlan J, Shen H. Real-Time Walk Light Detection with a Mobile Phone. 12th International Conference on Computers Helping People with Special Needs (ICCHP „10); Vienna, Austria. July 2010.

15. 1.Ahmetovic D, Bernareggi C, Mascetti S. Zebralocalizer: identification and localization of pedestrian crossings. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI „11); ACM, New York, NY, USA. 2011.

16. Coughlan J.; Shen, H. A fast algorithm for finding crosswalks using figure-ground segmentation. The 2nd Workshop on Applications of Computer Vision, in conjunction with ECCV, 2006.

17. Ivanchenko, V.; Coughlan, J.; Shen, H. Detecting and Locating Crosswalks using a Camera Phone, Computers Helping People with Special Needs Lecture Notes in Computer Science, Vol. 5105, 2008; pp. 1122-1128.

18. Advanyi, R.; Varga, B.; Karacs, K. Advanced crosswalk detection for the Bionic Eyeglass, 12th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA), 2010; pp.1-5.

19. Se, S. Zebra-crossing Detection for the Partially Sighted, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2, 2000; pp. 211-217.

20. Uddin, M.; Shioyama, T. Bipolarity and Projective Invariant-Based Zebra-Crossing Detection for the Visually Impaired, 1st IEEE Workshop on Computer Vision Applications for the Visually Impaired, 2005.

21. Omachi M.; Omachi, S. Traffic Light Detection with Color and Edge Information, IEEE International Conference on Computer Science and Information Technology, vol.2, pp. 284-287, Aug 2009.

22. Omachi M.; Omachi, S. Detection of Traffic Using Structure Information, IEEE International Conference on Computer Science and Information Technology, Oct. 2010; pp 809-812.

23. Charette R.; Nashashibi, F. Traffic light recognition using image processing compared to learning processes, IEEE International Conference on Intelligent Robots and Systems, Oct. 2009; pp. 333-338.

24. Alefs, B.; Eschemann, G.; Ramoser and H.; Beleznai, C. Road Sign Detection from Edge Orientation Histogram, IEEE Intelligent Vehicles Symposium, June 2007 25. Shoval, S.; Ulrich, I.; Borenstein, J. Computerized Obstacle Avoidance Systems for the Blind and Visually Impaired, Invited chapter in Intelligent Systems and Technologies in Rehabilitation Engineering. Editors: Teodorescu, H.; Jain, L.; CRC Press, ISBN/ISSN: 0849301408, pp. 414-448.

26. K. Fitzpatrick, S. T. Chrysler, V. Iragavarapu, and E. S. Park. Crosswalk marking field visibility study. Technical report, 2010.

27. K. Rothaus, J. Roters, and X. Jiang, “Localization of pedestrian lights on mobile devices,” in Proc. of APSIPA Annual Summit and Conference, Sapporo, Japan, 2009.

www.ijatca.com

9