Grading Method of Potted Anthurium Based on RGB-D Features

A grading method of potted Anthurium based on machine vision is proposed. A detection system is designed to acquire color images and depth images of potted Anthurium, and the three-dimensional point-cloud image is reconstructed after registration. According to the testing requirements of potted Anthurium, the minimum enclosing rectangle method is used to measure the width of crowns and spathes. )e bubble sequencing method is used to measure the plant height, and the clustering segmentation method is used to calculate the number of spathes. Online automatic grading software for potted Anthurium is developed. Compared with manual measurement, the average measurement accuracies of machine vision for crown width, plant height, spathe width, and spathe number are 98.4%, 98.4%, 98.8%, and 86.7%, respectively. )e accuracy rate of grading is 85.86%, which can meet the requirements of automatic grading of potted Anthurium.


Introduction
With the change of consumption fashion and the improvement of purchasing power, people's demand for flowers is increasing. Potted Anthurium, a highly ornamental tropical flower, has gained in popularity in recent years. In 2016, the yield of potted Anthurium in China alone was 35 million pots, and China has become the largest consumer of potted Anthurium in the world. Grading is an important part of large-scale production of potted flowers. e traditional manual method is time-consuming and laborious, lacks objective consistency, and may cause damage to flowers, in addition to being difficult to adapt to current large-scale production. Automated equipment is urgently needed to replace manual grading. Nondestructive testing technology based on machine vision has become an inevitable trend of flower grading due to its good objective consistency and high efficiency [1][2][3][4][5][6][7][8][9][10].
Flower grading based on machine vision has been studied by many researchers. Most of the publications are based on two-dimensional (2D) digital images and mainly focus on the defects, colors, and shapes of flowers [11][12][13][14][15][16][17]. However, limited by its own dimensionality, it is difficult to obtain a flower's real morphology from a 2D image. In recent years, increasingly, more attempts have been made to study plant phenotypes based on 3D information, but there are few reports of such studies of potted Anthurium [18][19][20][21][22][23][24]. In 2010, Wang et al. used binocular stereo vision technology to reconstruct a 3D model of corn leaves and measured the leaf length [25]. Uhrmann et al. developed a 3D scanning system for a single plant based on the principle of structured light. e 3D reconstruction of Arabidopsis thaliana was completed, and the plant height, leaf size, and growth angle were measured [26]. Bellasio et al. used structured light and fluorescence imaging 3D reconstruction technology to track the growth process of crops and obtained 3D information of leaf size, leaf angle, and plant height [27]. In 2017, Si et al. used the Harris algorithm to measure the plant height from the depth image of vegetables [28]. In 2019, Si et al. segmented overlapping particles from the registered depth image and obtained accurate walnut counts [29]. e goal of this study is to design an automated grading system for potted Anthurium based on machine vision. e advantage of this system is that the crown width, plant height, spathe width, and spathe number of potted Anthurium can be measured by a single image capture. To achieve this goal, a series of feature parameter acquisition algorithms based on RGB-D images were studied. Grading tests were then carried out to test the accuracy of measurement and grading.  [30][31][32], potted Anthurium can be divided into two varieties: small flower (spathe width <6 cm) and large flower (spathe width ≥6 cm), and each variety can be divided into three grades. e grading basis is shown in Tables 1 and 2 , respectively. e lowest grade of all evaluation parameters is taken as the grade of potted Anthurium, and potted Anthurium lower than the third level is removed directly. erefore, the evaluation parameters of potted Anthurium were determined such as crown width, plant height, spathe width, and spathe number.

Image Capture System.
To reconstruct a 3D model of Anthurium in a computer environment, an image capture system was built. e system consists of a conveying unit, image capture unit, control unit, and execution unit. As shown in Figure 1, the testing room size is 1800 mm × 1300 mm × 1800 mm and a KinectV2 camera was used to obtain both color and depth images in the top view. e resolution of RGB images is 1920×1080, and that of depth images is 512 × 424. e distance between the camera and tray is 1271 mm. e work flow of this system is as follows: the potted Anthurium to be tested are placed on trays of the conveyor belt at a certain distance and then transported to the testing room. When they reach the testing point under the camera, a photoelectric sensor is triggered and a signal is sent to the control unit. e control unit triggers the camera to take pictures, processes them, and returns the results. When the flower reaches the corresponding grading point, the photoelectric sensor is triggered, and the control unit drives the corresponding cylinder to push the flower out, completing the grading.

Image Capturing and Registration.
Since the resolution of the depth and color images is different, the pixels cannot correspond one-to-one, so image registration is necessary. In this study, the depth and color images are registered through the ICoodinateMapper interface provided by Kinect SDK software. e color and registered depth images are shown in Figures 2(a) and 2(b), respectively.

Construction of 3D Point Cloud.
e 3D point-cloud images generated from the RGB and registered depth images are shown in

Feature Parameter Extraction
(1) Crown width e crown width of potted Anthurium refers to the mean value of its crown projection in two vertical directions. In this study, the average side length of the smallest circumscribed rectangle is taken as the crown width. e original RGB image is shown in Figure 4(a), and the calculation steps are the following. First, the second-order color matrix method was used to separate the crown from background, as shown in Figure 4 e Ostu threshold segmentation method (threshold T �10) was then used for binarization, and the results are shown in Figure 4(c). It can be seen that there is still a significant amount of speckle noise. In order to obtain a complete and clear crown contour, two morphological opening operations were used. e first operation is performed to cut off the narrow connections and make the edges smoother, as shown in Figure 4(d). e second operation is conducted to remove speckles, as shown in Figure 4(e). After inverse calculation, the minimum circumscribed rectangle of the crown is drawn, as shown in Figure 4(f), and the average side length calculated.
(2) Plant height Plant height H is the distance from the edge of pot to the plant top, while the data measured by the depth camera are the distances from the plant top to the camera, h1. In this study, the distance from the edge of the pot to the camera is a fixed value of 1240 mm, so the   calculation formula for plant height is H � 1240−h1, as shown in Figure 5(a). e calculation steps are the following. First, taking Figure 4(b) as the template, the depth image of the crown is extracted and the depth value of the background is set to 0. e result is shown in Figure 5(b). en, open operation is used to remove speckled noise, and a complete and clear crown depth image is obtained, as shown in Figure 5(c). Finally, the bubble sort method is used to find the point of the minimum depth value, which is the vertex of the flower, as shown in Figure 5(d), so that the plant height can be obtained.

(3) Spathe width
Spathe width is the transverse diameter of the spathe, which represents the size of the flower. e largest spathe width is usually taken. According to the growth characteristics and top advantages of potted Anthurium, the largest flowers are usually spread at the top. us, the calculation steps are the following. First, the image of the spathe can be obtained by binarizing Figure 4(b) (threshold T �160), and the result is shown in Figure 6(a). Second, the morphological opening operation is used to remove the speckle noise to obtain a complete and clear spathe image, as shown in Figure 6(b). Finally, the Canny algorithm is used to extract the contour, and the smallest circumscribed rectangle of the largest spathe is calculated. As shown in Figure 6(c), the short side length is the spathe width. (4) Spathe number It can be seen from the above discussion that the difference between spathes and background is obvious, but the spathe number cannot be accurately extracted when the spathes are occluded in the 2D image. However, owing to the different heights of the spathes, they can be accurately segmented by using height difference in 3D point-cloud images. It can be seen from Figures 7(a) and 7(b) that two spathes occluded in the top view are easily segmented by height difference in the side view. erefore, in this study, 3D point-cloud images are used to calculate spathe number. e spathes were counted by the clustering segmentation method based on color region growth. As shown in Figure 7(c), the points in the same cluster are given the same color and the total number of colors is spathe number.

Grading Experiment.
Based on the method detailed above, grading software of potted Anthurium was developed. As shown in Figure 8, this software has a clear interface and simple operation. To verify the effectiveness and accuracy of the measuring method and grading effect of this grading system, machine vision and manual measurement methods were used for grading experiments, and the measurement and grading results were compared and analyzed. e experimental samples were 92 large variety potted Anthurium purchased from the Guangdong Academy of Agricultural Sciences, China. Tables 3-6 present the comparison results of machine-vision measurement and manual measurement of crown width, plant height, spathe width, and spathe number, respectively. e results show that the relative errors of the maximum, minimum, and average values of crown width and plant height are small, and the measurement accuracies are all above 98%. e maximum error of the spathe width is slightly larger, which is mainly due to the tilt of the spathe. e machine vision measures the spathe's projection width, so the value is smaller, which leads to the error, but the average accuracy is still high. e maximum error of spathe number is large, which is mainly because when two spathes are located at the same height and overlap, they cannot be accurately segmented, and a counting error occurs.

Mathematical Problems in Engineering
However, for 92 samples, this situation is rare, and the average measurement accuracy can reach 86.7%. Table 7 shows the comparison results of machine-vision and manual grading. It can be seen from the results that, compared with manual grading, the accuracy of machine-vision grading can reach 85.86%. According to the grading basis, the lowest grade of all evaluation parameters is taken as the grade of potted Anthurium, so the accuracy of grading is close to the lowest accuracy of measurement indicators. If the segmentation problem of overlapping spathes with the same height can be effectively solved, the accuracy of grading will be greatly

Conclusions
Based on the grading standard of potted Anthurium, a measurement method based on RGB-D image processing technology is proposed. is machine-vision method determines the grade of Anthurium by measuring the four phenotypic data (crown width, plant height, spathe width, and spathe number). e average measurement accuracies of crown width, plant height, spathe width, and spathe number measured by this method are 98.4%, 98.4%, 98.8%, and 86.7%, respectively. Compared with the manual method, the grading accuracy of the machine-vision method is 85.86%, which shows that the method is effective and feasible for potted Anthurium grading. is study has important reference value for automatic grading of potted flowers.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.