Int. Journal of Electrical & Electronics Engg.
Vol. 2, Spl. Issue 1 (2015)
e-ISSN: 1694-2310 | p-ISSN: 1694-2426
Design of Image Segmentation Algorithm for Autonomous Vehicle Navigation using Raspberry Pi 1
Ankur S. Tandale, 2Kapil K. Jajulwar 2
1
1,2
M.Tech Student, Research scholar Department of Communication Engineering, G.H.Raisoni College of Engineering,Nagpur 1
ankurtandale@gmail.com,2kapil.jajulwar@raisoni.net
Abstract—In the past few years Autonomous vehicles have gained importance due to its widespread applications in the field of civilian and military applications. On-board camera on autonomous vehicles captures the images which need to be processed in real time using the image segmentation algorithm. On board processing of video(frames)in real time is a big challenging task as it involves extracting the information and performing the required operations for navigation. This paper proposes an approach for vision based autonomous vehicle navigation in indoor environment using the designed image segmentation algorithm. The vision based navigation is applied to autonomous vehicle and it is implemented using the Raspberry Pi camera module on Raspberry Pi Model-B+ with the designed image segmentation algorithm. The image segmentation algorithm Fig. 1. Prototype of Autonomous vehicle Moving in Right has been built using smoothing,thresholding, morphodirection logical operations, and edge detection. The reference images of directions in the path are detected by the vehicle and accordingly it moves in right or left directions or localisation and maps the environment using the predefined stops at destination. The vehicle finds the path from source indoor environment area and the vision based form. It to destination using reference directions. It first captures involves complex computations and geometry to find the video,segments the video(frame by frame), finds the the path and obstacles in the path to map the edges in the segmented frame and moves accordingly. The environment. Raspberry Pi also transmits the capture video and In vision based autonomous vehicle navigation ,segmentasegmented results using the Wi-Fi to the remote system for tion of the captured frame is the fundamental step in image monitoring. The autonomous vehicle is also capable of finding obstacle in its path and the detection is done using processing. Segmentation is the process of grouping pixels of an image depending on the information needed for the ultrasonic sensors.
further processing. Various segmentation techniques are
Index Terms—Autonomous Vehicle, Graphical User Inter- present based on the region,edges,textures and intensities. face(GUI), Raspberry Pi, Segmentation, Ultrasonic Sensor As vehicles pro- ceeds with navigation using on- board
processing it possess a problem to the use of powerful computational units; secondly cost of the system hardware, I. I NTRODUCTION In the recent years, Autonomous vehicles have gained though having dropped in recent years, is still a limitation in importance due to its widespread applications in various robotics [1]. Therefore, robots requires powerful and fast fields such as Military, Civilian, industrial etc. Autonomous processing speed to perform on board processing of images. vehicle navigation has the ability to determine its ow In the last few years the demand for autonomous vehicles position and finding the path from source to destination. and robots has increased which have brought us a range of Navigation mainly defines the self localisation and finding ARM architecture computational devices such as the the destination path. Vehicle navigation has long been a Raspberry Pi or the even more powerful Quad- Core fundamental goal in both robotics and computer vision ODROID-U2 and these devices can perform on board research. While the problem is largely solved for robots real time image segmentation. equipped with active range- finding devices, for a variety of The proposed work uses a Raspberry Pi for real time reasons, the task still remains challeng- ing for vehicles processing and a camera connected to the raspberry pi for equipped only with vision sensors. On-board computing providing the vision. The prototype of the autonomous using the computer vision is the most demanding areas of vehicle is implemented as shown in figure 1. It is robotics. The need for autonomy in vehicles in indoor based having onboard Raspberry Pi, Microsoft Lifecam, Ultrasonic sensor, navigation systems demands high computational power in power supply and DC motors etc. The captured real the form of image processing capabilities. The Simultaneous time video is processed such that it is first segmented localisation and mapping(SLAM) algorithm performs the and the edges are found depending upon which the self NITTTR, Chandigarh
EDIT -2015
184
Int. Journal of Electrical & Electronics Engg.
Vol. 2, Spl. Issue 1 (2015)
vehicle moves in right, left or in certain angles. The complete task of segmentation is done using Raspberry Pi on board the vehicle in real time. The captured video using the Raspberry Pi camera is also transmitted using the WiFi to the remote computer. II. R ELATED W ORK Navigation can be done by designing proper Image segmentation algorithm. In literature [2], the stereo vision applied to small water vehicles using the low cost computers, which can drive autonomous vehicles capable of following other vehicle or boats in water is developed. The system uses 2 stereovision cameras which are connected to raspberry-pi for real time image processing using open computer vision libraries(OpenCV). This autonomous vehicle performs control of yaw and speed, line tracking and detecting obstacles. This system is capable of identifying and following targets in a distance of over 5 meters. In literature [3], the image segmentation algorithm is used for real time image processing as it is demanded by micro air vehicle(MAV) for navigation. Here, the image segmentation is implemented on FPGA for on board fast processing. The system finds vast application in military applications and for surveillance of structures like roads and rivers [4]. Real time autonomous visual navigation system is presented in [5] using approaches like region segmentation to find the road appearance and road detection to compute road shape. Monocular cameras along with proximity sensors are used to detect roads. Two algorithm are designed and there outputs are combined using a Kalman filter to produce a robust estimation of road which is used as a control policy for autonomous navigation in indoor and outdoor environments. Image matching is another approach for navigation and it is often used in unmanned aerial vehicle (UAVs) as used in [6]. The images can also be used in infrared range using CCD sensors for the purpose of navigation in day and night time. [7]. III. B LOCK D IAGRAM OF P ROPOSED S YSTEM The proposed title aims to design the Segmentation algorithm for autonomous vehicles on Raspberry Pi to help find obstacles and navigate the vehicles in an unknown environment. The below block diagram in figure 2. shows the proposed system for raspberry-pi Camera Feedback for Navigation based mobile robot navigation. The navigation is provided by designing the segmentation algorithm using images captured through camera on board the vehicles. • Camera: Camera is connected to the Raspberry pi and it acquires the video(24fps) from which the frame is taken as input and it is further processed. • Filter: The filter removes the noise from the acquired image so that the necessary information in image is not lost.
e-ISSN: 1694-2310 | p-ISSN: 1694-2426
F EATURES
OF
TABLE I R ASPBERRY-P I M ODEL B+
Features CPU
Raspberry-Pi Model B+ 700MHz-ARM 11-S core
Memory On board Ethernet
512MB RAM(shared with GPU) 10/100
Memory Storage Power Ratings
uSD Card Slot 8/16GB 700mA-1.8mA, 5V DC
USB Ports Video Outputs Operating Systems
4 HDMI Raspbian OS, Debian OS
Processing Unit: The processing unit is where the image segmentation is performed such that the gradient and edge tracking is done. From the edges it is possible to determine the reference image and so the vehicle moves accordingly. The ultrasonic sensor also gives the input to this unit so that the distance between the obstacle and autonomous vehicle is known and if the obstacle is near then the vehicle stops and starts moving in other direction to overcome it. All this processing is performed using the minicomputer called as Raspberry-Pi. The image segmentation algorithm is processed using the raspberry- pi. • Display Unit (GUI): The display unit is where the cap- tured video and segmented output is displayed using the WiFi sdapter on the remote screen. • Feedback: The segmented output is continuously moni- tored to find the gradient and edges and it is given as feedback to Raspberry Pi along with the sensor output to check the obstacle continuously. IV. I NTRODUCTION TO R ASPBERRY P I The design of image segmentation algorithm is done using C++ on Raspberry Pi board using the Open Computer Vision (OpenCV) [8]. The Raspberry pi is a handheld computer on the board consists of ARM processor and best suitable for real time operation. It runs on raspbian operating system which has the Linux environment. Officially launched in February 2012, the Raspberry Pi personal computer took the world by storm, selling out the 10,000 available units immediately. It is an inexpensive credit card sized exposed circuit board, a fully programmable PC running the free open source Linux operating system. The Raspberry Pi can connect to the Inter- net; can be plugged into a TV, and costs very less. Originally created to spark school childrens interest in computers, due to the variety of features mentioned in Table I, the Raspberry
Fig. 2. Block Diagram of Proposed System
185
NITTTR, Chandigarh
EDIT-2015
Int. Journal of Electrical & Electronics Engg.
Vol. 2, Spl. Issue 1 (2015)
Fig. 3. Raspberry pi Model-B+ Setup
Pi has caught the attention of home hobbyist, entrepreneurs, and educators worldwide. Estimates shows the sales figures around 1 million units as of February 2013.The figure 3. shows the Raspberry Pi model B+ setup with monitor and Ethernet connected to it. Qt creator is used for Qt GUI application development framework. Qt creator is a cross platform C++,javascript integrated development environment. The program is build on Qt creator and it is compiled using the Linux terminal. V. P ROPOSED M ETHODOLOGY F OR AUTONOMOUS V EHICLE NAVIGATION Ability to navigate in ones environment is important for a fully autonomous vehicle (AV) system. One critical task in navigation is to be able to recognize and stay on the path.The Raspberry Pi is connected with Microsoft LifeCam and the number of frames per second is 24fps. The image processing algorithm makes use of OpenCV libraries. The Video captured is first processed using the designed segmentation algorithm and the processing of input is done frame by frame. A. Image Segmentation Algorithm The image segmentation algorithm is designed using smoothing, thresholding,morphological operations, edge de- tection and tracking.The vehicle continuously tracks the refer- ence direction to move the vehicle from source to destination. Once the reference direction arrow is detected the Raspberry Pi processes the captured by using the algorithm. The indoor room environment with designed algorithm for Autonomous Vehicle navigation is shown in figure 4. The arrows are reference direction marks pasted or stuck on the wall at ground level. The wall is detected as obstacle and the Autonomous vehicle moves in backwards and checks for reference direction. The reference directions shown in figure 5. such as GO , left arrow, right arrow and STOP are used as mentioned above. When the camera detects the reference images it performs following: 1) The video captured is processed frame by frame. 2) The frame is converted into gray scale to limit the computational requirements. 3) Smoothing: The image is then blurred to remove the noise.
NITTTR, Chandigarh
EDIT -2015
Fig. 4.
e-ISSN: 1694-2310 | p-ISSN: 1694-2426
Example of Autonomous Vehicle Navigation in Indoor Room Environment
Fig. 5. Reference Directions Symbol
4) Finding Gradients: Only local maxima are marked as edges and they are marked where the image has large magnitudes . 5) Double Thresholding: Potential edges are then deter- mined by thresholding. 6) Edge Tracking By Hystersis: Final edges are determined by suppression of all edges that are not connected to certain edges. The detected edges should be as close as possible to real edges to determine the reference direction and move vehicle accordingly. The GO reference image denotes start, for Right arrow, the vehicle moves in right direction and so on. B. Vision Based Navigation: The vision based navigation can be done by reference object color recognition and the other method which is implemented here using the reference direction images. The figure 6. shows the flow chart of the proposed methodology for autonomous vehicle navigation. When the system is started all the libraries which are used in processing will be first initialized. When the Raspberry Pi onboard the vehicle starts, the camera on board the vehicle captures the GO image frame and the vehicle starts. This is done by capturing the GO reference image and then it is processed using the image segmentation algorithm, and the output is the threshold canny image which is then matched with edges in image and defined action with input database image of GO. Once the edges are matched, the vehicle starts moving forward until it finds the second reference image in the same direction to reach final destination. The vehicle then keeps moving and searching for the second reference image on wall for the next action in order to reach the final destination. When it finally captures the STOP(at destination) reference image it keeps on processing the frames and when any frame
186
Int. Journal of Electrical & Electronics Engg.
Vol. 2, Spl. Issue 1 (2015)
e-ISSN: 1694-2310 | p-ISSN: 1694-2426
Fig. 7. Segmented output of Right Reference Image
Fig. 8. Segmented output of STOP Reference Image
Fig. 6. Flow Chart of Proposed System
edges matches with the database image STOP edges, the vehicle stops at the final destination. Like this autonomous vehicle performs the navigation to reach destination. C. Obstacle Detection The ultrasonic sensors are used with the Raspberry Pi to detect the obstacles in vehicles path. The ultrasonic sensors are mounted on the vehicle and they are interfaced with Raspberry Pi. They mainly calculate the distance between the obstacle and vehicle and gives the output to the Raspberry Pi which then processes the inputs and the vehicle moves in backward direction and then it moves left and try to avoid collision with obstacle. The detection distance of ultrasonic sensors is 2cm450cm. VI. R ESULTS The image segmentation result of Right and STOP reference image captured in real time is shown in figure 7. and figure 8. The segmented outputs denotes the movement of vehicle in certain directions. The autonomous vehicle is capable of moving in indoor environment and detects the obstacles.The vehicle moves slowly due to lower RPM DC motors(10RPM) used due to high computation speed requirement for image segmentation algorithm. The Raspberry Pi also displays the captured video and segmented output on remote desktop using the WiFi network.
187
VII. C ONCLUSION The autonomous vehicle navigation implemented using the reference directions images on wall(at ground level) is done using the designed image segmentation algorithm. In the implementation, the vehicle is affected due to rough surfaces in indoor. At smooth surface, the vehicle moves properly towards desired direction to reach final destination by using reference images. The speed of vehicle can be increased by using good RPM motors but the computation(segmentation algorithm) tasks for each frame makes it difficult to obtain desired results using high RPM motors. In the future, autonomous vehicle navigation can be performed using the color object recognition in the indoor environment. Different color objects will be recognised by calculating the HSV values and then performing the segmentation of color object such that every color has certain movement defined in the system. Also,the mapping of indoor environment can be done by using the video frames on remote desktop to map environment using the MATLAB. REFERENCES [1] C. K. Chang, C. Siagian, and L. Itti, “Mobile robot monocular vision navigation based on road region and boundary estimation,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Algarve, Portugal, October 2012, pp. 1043–1050. [2] R. Neves and A. C. Matos, “Raspberry pi based stereo vision for small size asvs,” in IEEE International Conference. [3] Shankardas, D. Bharat, A. I. Rasheed, and V. K. Reddy, “Design and asic implementation of image segmentation algorithm for autonomous mav navigation,” in Proceedings of 2013 IEEE Second International Conference of Image Information Processing (ICIIP-2013), 2013, pp.352–357. [4] S. Rathinam, P. Almeida, Z. Kim, and S. Jackson, “Autonomous searching and tracking of a river using an uav,” in Proceedings of American Control Conference, New York City,USA, July 2007, pp. 359–364. [5] L. F. Posada, K. K. Narayanan, F. Hoffmann, and T. Bertram, “Floor seg- mentation of omnidirectional images for mobile robot visual navigation,” in IEEE,RSJ International conference on Intelligent Robots and Systems, Taipei,Taiwan, October 2010, pp. 804–809. [6] Z. Zhang, B. Sun, K. Sun, and W. Tang, “A new image matching algorithm based on multi-scale segmentation applied for uav navigation,” in IEEE, 2010. [7] A. Lenskiy and J.-S. Lee, “Terrain images segmentation in infra-red spectrum for autonomous robot navigation,” in IEEE, IFOST 2010 Proceedings, 2010. [8] G.Bradski and A.Kaehler, Learning OpenCV. ’Reeilly Media Inc., 2008.
NITTTR, Chandigarh
EDIT-2015