Development of Algorithms for an IoT-Based Smart Agriculture Monitoring System

Sensor-based agriculture monitoring systems have limited outcomes on the detection or counting of vegetables from agriculture ﬁ elds due to the utilization of either conventional color transformations or machine learning-based methods. To overcome these limitations, this research is aimed at proposing an IoT-based smart agriculture monitoring system with multiple algorithms such as detection, quanti ﬁ cation, ripeness checking, and detection of infected vegetables. This paper presents smart agriculture monitoring systems for Internet of Things (IoT) applications. The CHT has been applied to detect and quantify vegetables from the agriculture ﬁ eld. Using color thresholding and color segmentation techniques, defected vegetables have also been detected. A machine learning method-convolutional neural network (CNN) has been used for the development and implementation of all algorithms. A comparison between traditional methods and CNN has been simulated in MATLAB to ﬁ nd out the optimal method for its implementation in this agricultural monitoring system. Compared to the traditional methods, the CNN is the optimal method in this research work which performed better over the previously developed algorithms with an accuracy of more than 90%. As an example (case study), a tomato ﬁ eld in Chittagong, Bangladesh, was chosen where a camera-mounted mobile robot captured images from the agriculture ﬁ eld for which the proposed IoT-based smart monitoring system was developed. This system will bene ﬁ t farmers through the digitally monitored output at an agriculture ﬁ eld in Bangladesh as well as in Malaysia. Since this proposed smart IoT-based system is still driven by bulky, costly, and limited powered sensors, in a future work, for the required power of sensors, this research work is aimed at the design and development of an energy harvester (hybrid) (HEH) based on ultralow power electronics circuits to generate the required power of sensors. Implementation of multiple algorithms using CNN, circular Hough transformation (CHT), color thresholding, and color segmentation methods for the detection, quanti ﬁ cation, ripeness checking, and detection of infected crops.


Introduction
A smart agriculture monitoring system outcomes a digitized scenario of existing crops in the fields to the farmers who usually have an economic concern on their products of agriculture field [1] using sensors or transducers. Sensors or transducers receive physical quantities or signals from the agriculture environment and return data in digitized forms.
The economic concern of farmers has a functional dependency on these data (digitized form) as digitized outcome scenarios are received from the smart monitoring system since both their concerns and outcome scenarios consist of common attributes such as quantity, ripening, damaged crops, infected crops, and many more. Hence, farmers will obtain information on the economic values of crops in advance. This research will benefit farmers and upgrade the traditional methodologies used in Bangladesh and Malaysia from a nondigitized agricultural field to an IoT-based smart agricultural system. Since the supply power of sensors and transducers during working hours in the field is another challenge [2], the research intends to harvest energies from the surrounding environment and drain it to the input pin (VCC) of sensors and transducers. Hence, we have decided to design and simulate a hybrid energy harvester [3] to save up energies from solar power and electromagnetic (EM) sources.
In this research, we have developed and simulated a smart agriculture monitoring system and a proposal that will receive power from a designed and simulated hybrid energy harvester. The primary outcome of the smart agriculture monitoring system is the detection of ripen/unripe vegetables with defects in the crop field. A tomato field has been chosen as the case study.
1.1. The Problem Statement and Its Justification. FA Farmers' traditional way of cultivating and perceptions on crops in the field [1] do not reflect the accurate economic value much, since ripening of tomatoes likely depends on colors. That is why, the detection and the quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields.
Traditional methods are appropriate for agricultural monitoring on a small scale [1]. In such systems, the circular Hough transformation (CHT) is appropriate for objects with regular forms, such as circles [4] [5] [6] [7] [8]. The thresholding and color segmentation techniques are advantageous [9]. Simultaneously, the detection and quantification of vegetables (tomatoes) using the Internet of Things (IoT) will provide an accurate scenario of the existing crops in fields. Ni et al. [7] The method, CHT was applied on overlapped objects 2 Sa et al. [20] Applied the deep CNN along with RGB color followed by nearinfrared and faster R-CNN 80% to 83%. 3 Jimenez et al. [21] The CCD sensor 85% 4 Zhang et al. [22] The FSCABC algorithm and feed-forward neural-network (FNN) 87% 5 Zheng et al. [23] Deep learning method was applied, and deep learning classifier was trained having a larger dataset 99% 6 Mureșan and M. Oltean [24] Neural network -7 Kuang et al. [25] HOG features were applied along with LBP and Gabor-LBP followed by global color, global shape features, and histogram A lower miss rate of 0.0377 with false positives at 0.0682 per image 8 Ali et al. 2017 [26] Applied HOG features for detection and counting mangoes in trees - Crow [27] Applied the object-detection procedures along with mapping of the texture procedures 2 Zhuang et al. [13] Applied the cascaded-classifiers using Haar-features with an objective of improvement of the paleness 3 BaoHua et al. [28] Color transformation and machine learning were applied 4

Barbedo [1] Mukhopadhyay and
Chaudhuri [6] Yuen et al. [8] Applied the CHT method for detection of objects 5 Liebig [9] Color transformation and color segmentation were applied 6 Rizon et al. [29] D'Orazio et al. [5] Applied the object counting method 7 Ni et al. [7] Applied the CHT for detection among intersection objects 8 Sa et al. [20] Applied the deep convolutional neural networks for fruit detection 9 Zheng et al. [23] Deep learning procedures for the large datasets have been implemented here 10 Kuang et al. [25] HOGm LBP-, Gabor-, and histogram-based methods have been utilized here 11 Wang et al. [30] With the cascaded classifiers, Otsu method, with color thresholding, and an RGB camera, a laser rangefinder was used to estimate the mangoes length 12 Ali et al. [26] Applied the HOG features for detection and counting of mangoes in trees 2
CHT can be used to detect and quantify tomatoes, while thresholding and segmentation can be used to detect defects or black patches on the outer peel [16].
Detecting and quantifying ripened, green, and tomatoes with black spots on the outer peel in the field would undoubtedly lower farmers' workloads while continuing to play a critical role in sustaining market food values throughout tomato package distribution.
Due to energy constraints of sensors, to provide a continuous required supply of power, an energy harvester will be used, and a hybrid energy harvester will be designed and implemented, which will harvest energies from multiple sources of ambient energies from the environment [17] [18] [19].
Previous studies at a glimpse are highlighted in Tables 1-5, where those researched were limited to detecting or counting crops, fruits, or vegetables from fields. Detection quantification of tomatoes and classifying them as ripened, semiripened, or green followed by identifying defective vegetables have not yet been addressed. Moreover, a digitized monitoring system with a hybrid energy harvester that will scavenge energies from multiple sources added another novelty in this research.

1.2.
Objectives. The research outlines the following objectives: (i) To design and simulate a hybrid energy harvester to achieve the required power using solar and EM energy (ii) To develop and implement algorithms for the smart agriculture monitoring system using convolutional neural network to detect, quantify, and classify objects in machine learning platform (iii) To integrate and validate the hybrid energy harvester IoT-based smart agriculture monitoring system This robot and the camera will consume a maximum power of approximately 8500 MWh. Such an amount of power supply from a single source will be difficult, and multiple external sources need to be considered to design and integrate within a system. That is why, a hybrid energy harvester has been planned as a significant component of this research to be designed and prototyped.

Literature Review
A comparative tabulated list is shown in Table 1 on the accuracy of different image processing methods and technologies.

Technologies Used in Smart Agriculture Monitoring
System. Table 2 focuses on a comparison of characteristics of related research.

Wireless Communications and Mobile Computing
interface for better outcomes. In [21], a computer visionbased survey system was developed, whereas in [22,28] the research work was mainly based on design and development of machine learning-based system.

Technologies for Hybrid Energy
Harvester. Wang et al. [34] investigated performances of energy harvesters in terms of material design, system optimization, laboratory, and field works led by a numerical model in theory. For the proposal, we have tabulated the comparison of different sources in Table 3.
Reviews on energy harvesters using single sources are highlighted in Table 2, and those of using multiple sources are highlighted in Table 4.

Development of Smart Agriculture Monitoring System.
According to the flow chart (Figure 3), the circular Hough transformation (CHT) method has been applied to detect and quantify tomatoes in the fields. Outer skin black spots as defects of tomatoes have been detected using thresholding and segmentation.
The two approaches listed below have been implemented for the detection, quantification, ripeness checking, and identification of damaged (or defected) tomatoes.
(i) The color thresholding (CT) (ii) The cascaded object detection (COD) method A machine learning algorithm using CNN with the COD was also applied for the same purpose. In the end, a comparison between those traditional methods and machine learn-ing algorithms (CNN) was projected to conclude the optimal feasible method to obtain research objectives. Table 4, it is shown that solar panels and electromagnetic sources are optimal sources of energies to be saved up by energy harvesters. Hence, this research also proposes to harvest energies from these two sources. As limitation and availability of power is a challenge, we are planning to use both the sources (solar and electromagnetic) within a single hybrid generator to harvest energy to supply power to sensors of the smart agriculture monitoring system.

Result and Discussion
Consider the following scenario: assume that thepnumber of the positive type of datasets, on the same notennumber of negative datasets, are available, and also, there is availability of database of optimum amount of bootstrapping negative version of the datasets represented by d.
Let there be "g" learning objectives mentioned; subsequently, a cascade of useful binary nodes of the classifiers are available as stated below.
Here, n > >p. The learning will be based on the number of positive samples at each of the learning stage H given by where TP R stands for true positive rate value, whereas FP R for false positive rate values, and on the same note FN R represents false negative rate values. Tomatoes are sorted into three groups using a cascaded classifier: the ripened stage, the semiripened stage, and the green colored stage as shown in (Figure 4).   (Table 6) accurately reflects the classification of the stage of the ripened tomatoes is based on the rubrics using diverse color weighted values. Based on the category, wise applicability rules are divided into three parts. The first rule applies to ripened selections, whereas the second and third rules apply to the semiripened and also the green selections, correspondingly.
Based on Table 7, sensitivity was set to 0.92 (comparing with the previous value 0.90) for obtaining optimal detection. Since using circles, the tomato detection was lower when the color-transformation-method was utilized.
The object polarity here is established as dark and gloomy (the detector performs better detection of objects which are brighter than the visibility of the background portion of the image) The method of circle finding was also updated yet again, this time using Phase-code, which is both faster type and also more vigorous to noise when compared to the previous method involving two-stage procedure.
In addition, the type of outcome has been revised from Table 8. False positive type of outcomes has been eliminated; nevertheless, the number of undiscovered tomatoes has been increased to two. a. steps to detect tomatoes (with presence of solid nature of background) 1 Input image acquisition Process for Tomatoes 2 Detection process and also Counting by means of carrying out circular Hough transformation b. steps to detect tomatoes (with presence of natural type of background) 1 Input image acquisition Process for Tomatoes 2 Carrying out adjustment process of colors in the images 3 Modification of Internal type of Threshold values 4 Modification of sensitivity values 5 Modifying the polarity of Objects recognised 6 Detection process and also Counting by means of carrying out circular Hough transformation Step 1: Image acquired and Read Step 2: Calculate range of radius with minimum and maximum acceptable values Iterative process from Step 3 and upto 7 th Step until the desired value is attained Step 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94 Step 4: Call function to find circle Step 5: Lower value of edge threshold Step 6: Return values of centre and radius Step 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1 Step 8: Add label reading COUNT variable and display END IFINDCIRCLES Step 1: Call function corresponding to circles and their methods Algorithm 1: Pseudocode for this algorithm.
Step 1: Read the input image Step 2: Compute radii range by Minimum radius and Maximum radius Repeat from Step 3 to Step 7 until the desired output is NOT found from a single image Step 3: To upsurge the detection sensitivity values, start assuming that all the circles involved < internal threshold, the sensitivity value here is set exact to a value of 0.94 Step 4: Call function to find circle Step 5: Lower value of edge threshold Step 6: Return values of centre and radius Step 7: Insert circles centre and radius for yellow coloured tomatoes which were detected, call function to find circle to add boundary type of circles, say COUNT is equal to COUNT + 1 Step 8: Add label reading COUNT variable and display END IFINDCIRCLES Step 1: Call function corresponding to circles and their methods

Wireless Communications and Mobile Computing
The machine learning method outperformed the traditional method in terms of accuracy and run time complexity when it came to tomato detection. However, from the analysis, it has been found that by utilizing the COD, the ripeness complexity is Oðn 2 Þ. In contrast, with the CSM, the ripeness complexity is reduced to OðnÞ (refer to Table 9).
Ion the basis of Table 10, it has been found that the detection ratio is higher in proportion to the actual number of tomatoes harvested. In addition, the rate of false-positive type of the results has been lowered in relation to the rate of detected tomatoes (Table 11).

Performances of Algorithm Detection and Quantification.
With both the color thresholding and machine learning methods, from input images (Figures 4(1), 5, and 6(a)), Tomatoes from fields were detected and quantified (Figures 6(b)     10 Wireless Communications and Mobile Computing algorithms (whose flow chart is in Figure 10). In contrast, from input images of Figure 4(1), ripened tomatoes have been detected and quantified (Figures 4(2) and 4(4)) using color thresholding, color transformation methods, and cascaded object detector (COD) of machine learning method (whose flow chart is in Figure 7).

Performances of Algorithm in Identification of Tomatoes with Defects.
With the color segmentation method, using an   Figure 10: Algorithm1: detection of crops.
Sum before BW Figure 11: After the color segmentation method of red, green, and blue (before BW).
Original image Figure 12: Original image of a damaged tomato as input.
Sum after BW Figure 13: After the color segmentation method of red, green, and blue (after BW).

Wireless Communications and Mobile Computing
input image (Figure 4), defected tomatoes were identified and output images are given in Figures 11 and 12 where RGB values were applied before the binary conversion and in Figure 13 where RGB values were applied after the binary conversion.

Discussion
The color transformation method was added with color thresholding (for input image in Figure 4(1)) due to uneven brightness all over the image. The color adjustment was required for input Figure 4(1) to get both ripened and green tomatoes individually detected and quantified (Figures 4(2) and 4(3)). The object polarity was set to dark for common thresholding and object polarity. With this, the obtained accuracy for the color thresholding method was 85% approximately, and that for machine learning was 95% applying the Convolutional Neural Method (CNN) (see Figures 14 and 15).

Conclusion
The traditional method comprising CHT and CSM performed an accuracy of 84%, whereas the COD with the CNN outperformed the traditional method with an accuracy of 92%. For detection and ripeness checking, this research was only conducted in sunlight, whereas in the future, the research will apply these methods under shadow or different weather parameters. Besides, damaged tomatoes with black spots at their outer surface have been considered only. In the future, damaged tomatoes with other characteristics such as changing shapes water elements in outer skin will be identified to help farmers achieve more accurate economic value. The research has only compared the output of different energy sources. In contrast, a hybrid energy harvester will be designed and simulated to achieve the required power using solar and EM energy.

Wireless Communications and Mobile Computing
Hence, the future research of this study is aimed at developing a hybrid energy harvester that will harvest energies for sensors of this digitized agriculture monitoring system from multiple sources of energy. Additional vegetables will be added under monitoring with the different methodology for a better comparison of the system's performance.

Data Availability
The processed data are available upon request from corresponding author.

Conflicts of Interest
The authors declare that they have no conflicts of interest to report regarding the present study.