IJSTE - International Journal of Science Technology & Engineering | Volume 3 | Issue 09 | March 2017 ISSN (online): 2349-784X
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System Ramkumar. R PG Student Department of Computer Engineering Kumaraguru College of Technology Coimbatore, India
Kaliappan.S Assistant Professor Department of Computer Engineering Kumaraguru College of Technology Coimbatore, India
Vignesh.L PG Student Department of Computer Engineering Kumaraguru College of Technology Coimbatore, India
Abstract Every year farmers practice huge losses due to pest plague in crops & this in turn impacts his living. In this paper we discuss a novel advance to solve this problem by regularly monitoring crops using video dispensation, cloud computing and robotics. The paper concentrates in methodologies to categorize pests in one of the most admired fruits in the world – the tomato. An approaching into how the thought of the Internet of Things can also be conceptualized in this project has been elaborated. Keywords: Early Pest Detection, Fruit Plucking, Growth Stages, Internet of Things (IOT), Tomato Detection, Video Processing ________________________________________________________________________________________________________ I.
INTRODUCTION
For a increasingly developing nation such as India, agriculture is one of the most significant sources of income to the country [1]. It must be noted that agriculture only constitutes almost 22% of the country’s Gross Domestic Product (GDP), which is indeed a significant number [2]. It is due to this fact, that governments of several developing countries set aside a large part of their economic budget for the growth, development and introducing systematic and technologically sophisticated methodologies for farming each year. In a farmer’s perspective, such initiatives would assist them in producing better surrender and safeguarding their manufacture, thus improvising their lifestyles as well. However, even though such programmes exist, farmers extend to be plagued by three major areas of concern – insufficient water supply (irrigation), attack of crops by pests & insects and thirdly - failure in properly storing manufacture which in turn might be attacked by pests and rodents [3]. Over the past few years, a lot of research has been carried out to overcome the aforesaid problems. Usage of cloud seeding techniques and construction of barrage or dams near villages have reduced the water disaster to some extent [4]. Modern storage techniques which include utilizing rodent exterminators keep rodents at cove [5]. But when it comes to prevention of vermin or insects from attacking crops, the usual practice is to spray continual amounts of pesticides. Though efficient this practice comes with its own set of disadvantages. Extreme use of pesticide can lead to beyond repair damage to the environment. Being at the top end of the food chain, this would also create dangerous health issues to humans, including birth defects. With respect to farmers, direct contact to these chemicals often lead to skin diseases and extended use might lead to cancer [6]. A solution to this issue, is by spraying bug killer on crops only when need required and also by eliminating those crops or fruits already affected by vermin. This solution though simple, would require a constant monitoring by the farmer which would turn out to be infeasible due to various reasons such as the lack of the agriculturist at all points of time. Ill health of farmers might also constitute for his/her lack from the farm. Therefore, in order to address this major issue, we describe a work of fiction, innovative and a fully automated method to perform a constant observation of the farm at all points of time, even in the absence of the farmer. The major attentiveness of this paper lies in identifying fruits of crops in their early immature stages and also to monitor them for any disease or bug infestation. If in case any disease or bug activity is detected, such fruits could be immediately eliminated in turn preventing spreading of the same to other healthy crops. Pesticides, insecticides and also bio-control of pests [7] could be utilized for immediate removal. For all experiments conducted, we utilized the Tomato plant (Solanum lycopersicum), since it is one of the most widely grown crops in the world [8s]. Tomatoes commonly get affected, in their growing stages (unripe stages), by a pest known as the borer insect Helicoverpa armigera [9], which is usually recognized by a black color hole in the fruit through which it enters [10]. In order to identify the same, a complete video processing application using Java was developed and was deployed using cloud computing which has been detailed further in this paper. The next section describes few other existing methods using real time video processing to identify immature tomatoes. Section III of this paper discusses the setup used for our testing and also that deployed on the cloud. The complete video processing algorithm developed is elaborated in section IV along with the results obtained in section V. Section VI deals with the future scope of this research with some insights into further extensions. All rights reserved by www.ijste.org
532
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
II. EXISTING METHODOLOGIES As of date, a lot of researchers have discussed new methodologies to notice fruits and vegetables on plants utilizing various techniques such as video processing and have also explained systems which can fortitude or segregate the same. The first ever publication to discuss the same, was in the year 1995 [11]. In this paper, the authors introduce a work of fiction idea to identify tomatoes in plants using several visual sensors and also to fortitude the same using suction cups attached to a robot. Unfortunately, no further explanation on how to achieve the same has been elaborated. In [12] a complete explanation of automatic tomato recognition on plants using a normal laser and an infrared laser has been described. The system though accurate proves to be cost ineffective. [13] - [15] utilize image processing techniques and hi-fi cameras to detect tomatoes but involve difficult and time consuming algorithms such as principal component analysis and K-Means clustering, which slow down the complete process. Several other research works exist which deal with the detection of oranges [16], pomegranates [17] and the egg plant [18] but use techniques not applicable to this project. Several methodologies to identify tomatoes in controlled environments within industries on conveyor belts also exist [19]-[21], but cannot be utilized since these approaches do not address the problem of intervention of leaves, stems and other environmental factors. It must also be noted that all the papers mentioned above deal exclusively with detection of ripe fruits in turn helping in harvesting, whilst our scope of research deceit in the fact that detection has to be done at the immature stages itself. Also, there is limited amount of discussions on detection of vermin and more particularly the borer insect. The next section discusses the Setup used for our conducting tests in detail. The setup described can be used in combination with the robot explained in [22]. III. SETUP The initial setup basically consists of a camera which is mounted on a robotic car. The robotic car is designed such that it can move around the field, in the space among two rows of crops, with the camera capturing images or obtaining a live video give food to of the crops. For the primary experiments conducted, we utilized an Intex 5.0 mega pixel web camera with an image resolution of 356x320 [23]. The camera was located at a distance of 1.5 feet absent from the tomato plants. The total frame rate of the same was 30 frames per second. This in turn was connected to a laptop on which the main video processing algorithm runs. Based on the decisions taken by the algorithm, a robotic arm or a spraying system could be designed to spray the necessary amount of pesticide on the tomatoes. Though this proved to be a highly speed well-organized solution, the main disadvantage of this setup was the overall power obsessive. Since the camera, motors of the robot and also the laptop have to be powered using a moveable power supply, such a solution would prove to be difficult to the farmer, since a constant watch on the power levels of the battery has to be taken care of and everyday recharges become necessary. In order to take care of this problem and by doing so, reduce the power consumed by the system, it was predictable to think of a solution in which the processing – the most power keen task of the system [24], could be performed elsewhere. In other words, create a system which could wirelessly communicate with the camera and the robot and do the necessary processing on a remote server. Therefore, in order to achieve the above mentioned necessities we utilized concepts of cloud computing as part of our setup. An illustration of the setup in the playing field using a wireless web camera can be seen in Fig. 1.
Fig. 1: Design of the setup by means of a wireless web camera
According to the National Institute of Standards and Technology (NIST) [25], cloud computing is distinct as a model which is capable of creating a universal, “all over the place available”, on-demand network access to a distantly located shareable group of computer resources that can be utilized with minimal involvement of the end user. Any cloud computing setup utilizes three major service models which are found in our setup as well. The complete cloud computing explanation for our system can be seen in Fig. 2, comprehensive of the three major service model layers – The Software as a service (Saas), the platform as a service (Paas) and the Infrastructure as a service (Iaas) [26]. The same has been elaborated below.
All rights reserved by www.ijste.org
533
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
Infrastructure as a service (IaaS) Infrastructure as a service mainly defines the hardware or the processor on which the applications on the cloud would be executed. For all our experiments an Intel i3 32-bit core processor was utilized. Platform as a service (Paas) In order to organize the application created on the hardware mentioned above, there is a need of a software operating system on which the expansion kit or the necessary libraries for the applications to work can be installed. In this case we utilized the Microsoft Azure software as the Paas. Software as a service (Saas) The software application developed and which is to be deployed on the Paas/Iaas is nothing but the Saas. In this case, the Java application accountable for detecting the unripe fruits and the borer bug using the wireless feed from the camera constitutes the Saas in our system.
Fig. 2: Cloud computer based architecture for smart agricultural
The complete algorithm of the application developed has been elaborated in detail in the next section. IV. OUR METHODOLOGY As mentioned before, a novel and a highly speed well-organized video processing algorithm was developed using the Java programming language. Speed effectiveness was necessary since decisions on whether to spray the pesticide or not has to be taken directly and sent back to the robot from the cloud. The first phase of the algorithm deals with the actual immature tomato detection on the plant and later phase of the algorithm concentrates on finding the borer bug. The steps followed for the same have been shown in the flowchart in Fig 3. And the explanation for the same has been described in detail below. Color Conversion Each frame of the real time video feed obtained from the system explained in the previous section is first subjected to color change. It must be noted, that the frames obtained are in Red, Green and Blue color system. Though it is of common idea that green tomatoes could be directly identified using the green color plane, it is not true. The main reason for the same is the dependence of the RGB color space on brightness [27]. Hence, if the brightness changes, the range of the RGB values on behalf of the immature tomato also changes, making it impossible to identify tomatoes under different brightness situation. Therefore, in order to recompense for this effect, the frames obtained are first converted to a model known as the YCbCr model [28]. The conversions are protected by equations (1)-(3) as defined by the ITU-R BT.601 [29] and have been given below. Y' = (0.299 × R) + (0.587 × G) + (0.114 × B) (1) Cb=0.16*R−0.33*G+0.500*B (2) Cr = 0.5*R - 0.418688*G - 0.081312*B (3) All rights reserved by www.ijste.org
534
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
Fig. 3: Flowchart depicting the steps involved in our algorithm
Where, Y represents the concentration, Cb represents the chroma blue color component –in other words blueness of the image devoid of brightness and Cr – the chroma red part in other words redness devoid of brightness. The color converted frames are then segmented using the Cb and Cr mechanism alone, to obtain the region of interest. Another important advantage of color change in this case, is that the amount of memory necessary to store each pixel of the pictures reduces to only 16 bits as different to 24 bits which was used with the RGB model. This helps to decrease the overall memory necessity by 1/3 rd. Segmentation of Immature Tomatoes Segmentation, in image processing, is the task of separating the region of attention from the set. In this case, our region of interest is exclusively the immature green tomatoes and the challenge deceit in the fact that, this has to be segregated from a background mainly consisting of stems, leaves and other artifacts false closely in the same range as that of the tomato. Several experiments were conducted on over 100 images of tomatoes, and it was found that green tomatoes have a separate range of Cb and Cr which were obtained through histogram analysis. The histograms procured for the same have been shown in Fig. 4. Similar histograms were obtained for various images taken under changeable environmental conditions.
Fig. 4: Histograms of Cb and Cr of the tomato images with the ranges marked
All rights reserved by www.ijste.org
535
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
Utilizing these values, segmentation was performed in turn making the tomatoes black (0) and the background white (255). Assuming θcr1 and θcr2 are the ranges for the Cr components of the object of notice and θ cb1 and θcb2 - the ranges for the Cb component, then the segmented image S obtained can be represented by (4).
255, otherwise
(4)
Noise Removal As with all segmentation techniques, one of the major issues faced after conclusion of the process, is the incidence of noise. In this case, a few parts of the stem and leaves are chosen up as part of the tomato and consequently it is necessary to eliminate the same. In this paper, noise removal was performed using a methodology known as corrosion. A structuring element of 3x3 was applied on the resultant image ‘S’ obtained in the previous step. Upon conclusion of this process, most of the noise is obliterated but along with it comes a major disadvantage – corrosion causes even the edges of the tomato to disappear. Therefore, in order to abolish this effect, we go in for the reverse process – dilation. Whilst corrosion is a process of checking whether any surrounding pixel is black and then making the pixel under consideration black as well (if it is white), dilation on the other hand is quite the opposite [30]. Hence on application of the dilation algorithm, we keep back the edges and the shape of the tomatoes to some extent. A structuring element of 13x13 was utilized for this process. Obtaining back the Tomatoes in Color After implementation the above mentioned steps, we now require to get back the tomatoes in color, so that further steps can be processed. In order to perform the same, we utilize the dilated image (D) obtained as a mask and logically “AND” the same with the RGB image (C) captured from the camera. By doing so, we now attain only the tomatoes in color and the background remains black. This can be further represented by (5). Extracted Image(i,j) = D(i,j) & C(i,j) (5) Once all these processes complete, phase 2 i.e. the identification of the borer insect on the tomatoes extracted, begins. This has been explained further in detail. Identification or segmentation of the borer insect The final phase of the experiment is to take out the borer insect, if present on the tomato. In order to perform the same, the color image obtained above (after masking) is now subjected to color change once again. Since borer insects are indicated by black color holes on the tomato, it is sufficient therefore to convert the color image to an strength component (Y). This is due to the fact that, once histogram analysis is performed, the pixels belonging to the borer would always recline in the former end of the histogram. This can be seen in Fig. 5.
Fig. 5: Histogram of the extracted tomato with borer insect
Utilizing (4) and the new threshold values, successful origin of the borer insect was possible. An corrosion operation with a structuring element of size 3x3 and a dilation of structuring element size 7x7 was performed to retrieve the borer with exactness. A simple counting algorithm could now be initiated, to count the total number of borer insects affecting the fruit or an area calculation can also be performed to find the extent of damage. The next section presents the results obtained for the algorithm explained in this part.
All rights reserved by www.ijste.org
536
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
V. RESULTS As mentioned in Section III, all algorithms were designed and developed using the Java programming language in combination with the Java Development Kit (JDK) 1.6 and Java Media Framework (JMF) 2.1. All programs were deployed as part of the Saas layer of the cloud computing standard on a 32- bit Windows system. Testing was conducted on over 100 images of tomatoes (immature) procured from actual farms and also from the internet. Fig. 6 shows an example of an immature tomato picture taken 1.5 feet away from the plant.
Fig. 6: Immature tomato picture in use 1.5 feet away
It can be clearly seen that the top right most tomato has been exaggerated by the borer bug whilst the bottom left few tomatoes are still undamaged. As explained in the previous part, the first step would be to change the input image into the Cb and Cr components correspondingly. Fig. 7a and 7b show the same.
Fig. 7: Cb and Cr components of the Tomato picture procured
The image obtained after applying the histogram based segmentation can be seen in Fig. 8.
Fig. 8: Image obtained after histogram based segmentation
All rights reserved by www.ijste.org
537
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
As discussed earlier, there is an excessive amount of noise in the image. The results obtained after the corrosion and dilation operation can be seen in Fig 9a and Fig. 9b respectively. The regions which contain the tomato can also be seen clearly. Once this procedure completes, the next step is to perform masking using the mask obtained after dilation.
Fig. 9: Images obtained after erosion and dilation
The resultant image obtained after masking can be seen in Fig. 10.
Fig. 10: Image obtained after masking
It can be clearly seen that the immature tomatoes have been successfully extracted and the borer recognition algorithm can be performed. Extraction of the borer follows the steps mentioned in the previous part. It can be visualized from Fig. 11, that the borer bug has been successfully detected. The non-affected tomatoes have been completely disregarded.
Fig. 11: Borer insect successfully extracted from the tomato
Further to this, a speed relationship i.e. a complete comparison of the implementation time of our algorithm with that of the algorithms explained in [15] & [14] to detect green tomatoes, was performed and tabulated in Table 1. Table - 1 Comparison of Timings of Three Algorithms
It can be clearly seen from Table I that the algorithm designed and implemented in this paper takes a mere 0.46s in comparison with the other two well-known algorithms which were used in [14] and [15]. This would in turn assist in a faster judgement of tomato borer infestation and action can be taken more quickly.
All rights reserved by www.ijste.org
538
Early Detection of Borer Insects Tomatoes by IoT based on Smart Farming System (IJSTE/ Volume 3 / Issue 09 / 106)
VI. CONCLUSION In this paper a novel methodology to perform smart agricultural and also to detect the borer bug on tomato plants, has been described. The complete model implemented on a cloud computing framework was also elaborated. It can also be seen that the algorithm explained is 1.71 times faster than two other popular methods of immature tomato finding. Based on the resultant obtained from the algorithms, a choice can be taken as to how much amount of bug killer should be sprayed. This prevents unnecessary spraying of pesticides as well as protection the health of farmers, consumers and also the environment. Future versions of this implementation could include a unique IP address for the robot, in turn controlling the same via cloud computing or by a farmer elsewhere. In other words a complete “Internet of Things” move toward can be utilized. Governments of various countries can adopt this plan and provide reprieve to hundreds of farmers worldwide. REFERENCES [1]
[2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28]
[29]
Huddar, S.R., Gowri, S., Keerthana, K., Vasanthi, S., Rupanagudi, S.R.,"Novel algorithm for segmentation and automatic identification of pests on plants using image processing," Computing Communication & Networking Technologies (ICCCNT), 2012 Third International Conference on , vol., no., pp.1,5, 2628 July 2012 Ministry of Statistics and Programme Implementation, Press Note on Quarterly Estimates of Gross Domestic Product for the Third Quarter (OctoberDecember) of 2013-14, Government of India, February 2014. The International Rice Research Institute, “Suphan Buri, Thailand, 1974 and 1975,” in Constraints to High Yields on Asian Rice Farms: An Interim Report, Philippines, 1977, Ch 7, pp 255. Mohana A. Krishnan, “The promise of cloud-seeding”, The Hindu, April 13, 2014. [Online]. Available: http://www.thehindu.com/opinion/openpage/ thepromise-of-cloudseeding/article5906186.ece [Accessed Sept.10, 2014]. Sahay, K.M., Singh, K.K., “Storage,” in Unit Operations of Agricultural Processing, Noida, India: Vikas Publishing House, 2004, Ch 4, pp 164- 172. World Health Organisation, Preventing Disease through Healthy Environments, Exposure to Highly Hazardous Pesticides: A Major Public Health Concern, Geneva, 2010. Singh, A., Parmar, N., Kuhad, R.C, “Biological Control of Pests,” in Bioaugmentation, Biostimulation, Biocontrol, Springer Science & Business Media, 2011, Ch 10, pp 223-225. Ghaffari, R., et.al, "Early detection of diseases in tomato crops: An Electronic Nose and intelligent systems approach," Neural Networks (IJCNN), The 2010 International Joint Conference on, vol., no., pp.1,6, 18-23 July 2010. Peña, J. E., “Biology, Ecology and Management of the South American Tomato Pinworm, Tuta absoluta,” in Potential Invasive Pests of Agricultural Crops, Volume 3 of CABI invasive species series, CABI, 2013, Ch 6, sec 1, pp 98. Missouri Botanical Garden, “Tomato Fruit Problems,” Last updated: April 2012.[Online]. Available:http://www.missouribotanicalgarden.org/gardensgardening/yourgarden/help-for-the-home-gardener/advice-tips-resources/visualguides/tomato-fruit-problems.aspx. [Accessed Sept. 10, 2014]. Kondo, N., Monta, M., Fujiura, T., Shibano, Y., "Intelligent Robot to Harvest Tomato," Robotics and Automation, 1995. Proceedings., 1995 IEEE International Conference on , vol.3, no., pp.V19,, 21-27 May 1995 Gotou, K., Fujiura, T., Nishiura, Y., Ikeda, H., Dohi, M., "3-D vision system of tomato production robot," Advanced Intelligent Mechatronics, 2003. AIM 2003. Proceedings. 2003 IEEE/ASME International Conference on , vol.2, no., pp.1210,1215 vol.2, 20-24 July 2003. Yang, L., Dickinson, J., Wu, Q. M J, Lang, S., "A fruit recognitionmethod for automatic harvesting," Mechatronics and Machine Vision in Practice, 2007. M2VIP 2007. 14th International Conference on , vol., no., pp.152,157, 4-6 Dec. 2007. Kundu, S., Hazra, A, Deb, K., Hazra, P., "Dimensionality reduction of morphological features of tomato leaves and fruiting habits," Communications, Devices and Intelligent Systems (CODIS),2012 Hongpeng Yin, Yi Chai, Yang, Simon X., Mittal, G.S., "Ripe tomato extraction for a harvesting robotic system," Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on , vol., no., pp.2984,2989, 11-14 Oct. 2009 Cakir, Y., Kirci, M., Gunes, E.O., Ustundag, B.B., "Detection of oranges in outdoor conditions," Agro-Geo-informatics (Agro-Geo-informatics), 2013 Second International Conference on , vol., no., pp.500,503, 12-16 Aug. 2013 Akin, C., Kirci, M., Gunes, E.O., Cakir, Y., "Detection of the pomegranate fruits on tree using image processing," Agro-Geoinformatics (AgroGeoinformatics), 2012 First International Conference on , vol., no., pp.1,4, 2-4 Aug. 2012 Jian Song, "Target identification based on automatic threshold for eggplant picking robot," Information Science and Engineering (ICISE), 2010 2nd International Conference on , vol., no., pp.1567,1569, 4-6 Dec. 2010 Gejima, Y., Houguo Zhang, Nagata, M., "Judgment on level of maturity for tomato quality using L*a*b* color image processing," Advanced Intelligent Mechatronics, 2003. AIM 2003. Proceedings. 2003 IEEE/ASME International Conference on , vol.2, no., pp.1355,1359 vol.2, 20-24 July 2003 Guo Feng, Cao Qixin, "Study on color image processing based intelligent fruit sorting system," Intelligent Control and Automation, 2004. WCICA 2004. Fifth World Congress on , vol.6, no., pp.4802,4805 Vol.6, 15-19 June 2004 D Amato, Juan Pablo, Garcia Bauza, Cristian, Boroni, Gustavo, "Color Based Fruits classification using GPU," Latin America Transactions, IEEE (Revista IEEE America Latina) , vol.9, no.3, pp.346,350, June 2011 Nezhad, M.AK.B., Massh, J., Komleh, H.E., "Tomato Picking Machine Vision Using with the Open CV's Library," Machine Vision and Image Processing (MVIP), 2011 7th Iranian , vol., no., pp.1,5, 16-17 Nov. 2011 Intex Technologies, “Intex IT-305WC 5MP Webcamera” [Online]. Available: http://tinyurl.com/lzp68uq. [Accessed Sept. 10, 2014]. Azhar, S., Vasanthi, S., Rupanagudi, S.R., "Low Power Optimization of a 3G UMTS Filter," International Conference on Frontiers of Computer Science. (ICFOCS), 2011, vol., no., pp., August 2011 National Institute of Standards and Technology, The NIST Definition of Cloud Computing, Recommendations of the National Institute of Standards and Technology, Special Publication 800-145, Gaithersburg, 2011. Shao, J., Wang, Q., Mei, H., “Model Based Monitoring and Controlling for Platform-as-a-Service (PaaS),” Cloud Applications and Computing (IJCAC), International Journal of , vol. 2, issue 1, 2012. Ravoor, P.C., Ranjani, B.S., Rupanagudi, S.R., "Optimized fingertip blob recognition for image processing based touch-screens," Recent Advances in Computing and Software Systems (RACSS), 2012 International Conference on , vol., no., pp.104,108, 25-27 April 2012 Dhruva, N., Rupanagudi, S.R., Sachin, S.K., Sthuthi, B., Pavithra, R., Raghavendra, "Novel segmentation algorithm for hand gesture recognition," Automation, Computing, Communication, Control and Compressed Sensing (iMac4s), 2013 International Multi-Conference on , vol., no., pp.383,388, 22-23 March 2013 [29] Radiocommunication Sector of ITU, Recommendation ITU-R BT.601- 7, Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios, March 2011. Gonzalez, R.C., “Morphological Image Processing,” in Digital Image Processing, India: Peasrson Education, 2009, Ch 9, sec 2, pp 652-656
All rights reserved by www.ijste.org
539