Stomatal State Identification and Classification in Quinoa Microscopic Imprints through Deep Learning

,


Introduction
e stomata are an essential part of phylogenetic relationship studies and their physical characteristics are important for plant origin and classification studies [1]. Leaf stomata in various plants possess significantly different characteristics such as density, size, and shape [2]. e stomata are surrounded by epidermal cells known as guard cells. When the plant has enough water in its cells, the guard cells swell up, which helps in opening the stomata. Conversely, when the plant cells do not have enough water, the guard cells cannot swell up; therefore, the stomata remain closed. In this state, the photosynthesis process stops, and plants eventually face starvation [3,4]. Because of stomata's importance in regulating plant growth and response to the environment, they are the key functional attribute of interest for researchers in plant biology [5]. e programs that are working on the improvement of crops have modified stomata phenotypes to increase the yield [6]. e stomata easily get affected by external environment conditions and their micromorphological characteristics are often used for plant classification [7,8]. e state of stomata helps plants to reduce water loss by changing their state from open to closed according to environmental conditions. Researchers across various disciplines in plant biology will phenotype stomatal traits for decades to come because of their physiological importance [9]. However, researchers from agriculture and horticulture disciplines are unable to classify the stomata states in leaf. In different environments, stomata have different states. In the stress, environment stomata are commonly present in closure or semiclosure states because different factors affect stomata. e factor drought stress is one of the major causes of crop loss every year and it is also a cause of loss of aesthetic value in ornamentals [10,11]. Moreover, the scientists count the stomata to investigate their distribution in leaves using different sampling and counting techniques. e purpose of stomata counting is to inspect their numbers density and distribution on leaf surfaces, to know about their numbers that are open or closed at any time, and to see the adaptations of plants to environmental conditions [12]. e first approach to detect stomata through the HOG feature was presented [13,14]. is study uses a convolutional neural network to classify single-clipped pores as open and closed, and it involves a binary image segmentation process for the automatic pore measurements.
is algorithm requires a manually defined range of parameters (like area, spindle, length, solids, and centroid coordinates). Moreover, this study is unable to identify pores when the size or shape is not the same as the predefined parameter range. Recently, a method was presented to determine the stomatal state in microscopic imprints using Mask RCNN [15]. However, this study mainly depends on expert horticulturist knowledge to manually label stomata states (open and close). Furthermore, identification of stomatal states in images through naked eyes leads to inaccurate results due to the fact that the stomatal state is affected by the area of pore aperture in a leaf [16].
A method for the measurement of stomatal pores using image processing software, like ImageJ [17], involves manual measurement of stomata classification and detection system in microscope images for maize cultivars [18,19]. is type of method wants to manually label points of interest from researchers, such as height, width, and boundaries in pores. e drawbacks of this method are that, first, it requires manual attention and, second, it requires a huge amount of data. Using this method, only some of the data points can be used to build the model, which leads to inaccurate results [20].
In this study, we propose an automatic system for the identification of stomatal state (open and close) and stomata counting in microscopic imprints using a deep convolutional neural network, Single Shot Detection (SSD-Mobi-leNet V2). SSD goes straight from pixels of the image to bounding box coordinates and class probabilities. e architecture of this model is based on the inverted residual structure in which the input and output of residual blocks are thin bottleneck layers as compared to traditional residual models. Moreover, this model also provides a prediction about the category of each pixel along with the bounding box around the object. is method was used for the detection, counting, and state identification of stomata in microscopic imprints of quinoa crop leaf. e results of this approach were evaluated visually. Furthermore, stomatal states were classified through a Support Vector Machine (SVM), which is a supervised machine learning model that uses a classification algorithm for two-group classification problems.

Related Work
is section presents machine learning techniques presented by researchers on stomata study using image processing and deep learning. e research on stomata by using image processing techniques started in the early 90s when the first approach [21] was proposed. is technique measures the stomata features in grayscale images using Fourier transform and threshold filter for image segmentation. A study [22] proposed a system of automatic stomata counting in a microscopic image: this technique first converts RGB images into CieLAB to select the best channel for analysis. e detection of stomata is performed by wavelet spot detection and morphological operations. is system achieves 90.6% accuracy by using a watershed algorithm.
An approach was published, which compared tomato cultivars using many morphological features, stomata measurements, and the microscopic images of the different crops, which are obtained by the scanning electron microscope (SEM) method [23]; in this approach, the segmentation was performed by a watershed algorithm resulting in one stoma per image followed by morphological operations.
e Sobel kernel filters were used to remove noise and obtain stomatal boundaries. is method uses 100 images of tomato cultivars and a multilayer perceptron algorithm and gets 96.6% accuracy. In 2017, an approach of automatic stomata detection and pore measurements for a grapevine was presented [24]. is approach uses a cascade object detection (COD) algorithm. It consists of two steps: In the first step, the first cascade classifier was trained by using stomata and nonstomata images. In the second step, a sliding window was used on microscopic images to identify the stomata inside the image. e detection of the pore measurements was performed using binary segmentation and skeletonization with ellipse fitting, which estimates complete pore measurement for incomplete stomata. As a result, this method reached 91.6% precision. An automatic stomata counting system using a deep convolutional neural network (DCNN) was proposed [25]. is technique identifies and counts the stomata in a variety of different micro imprints stomata. is network achieves 98.1% identification on the Ginkgo scanning electron micrograph. To improve the accuracy and performance of the traditional Single-Shot Multibox Detector (SSD) object detection model, an approach [26] was presented.
is technique works on the defect of a weak correlation between object prediction score and positioning accuracy in the SSD model. e improved model enhanced the correlation between these two by adding an intersection over the union (IoU) prediction loss branch. Deep learning rose to its prominent position in digital image processing and computer vision when neural networks were applied in various types of medical image analysis datasets [27,28]. Recently, an approach [29] has been published; in this study, the author develops an automated system for stomata detection, which can detect individual stomata boundaries regardless of the plant species. However, this study mainly focuses on stomata detection and segmentation of different species and not providing information about the state of stomata in leaf images.

Dataset.
In this work, the dataset of quinoa microscopic images is photographed through Leitz DMRD light microscope (Leica MikroSkopie & camera (Leica DFC 420) [16,30]. e surface of the quinoa crop leaf was cleaned using a soft brush. A thin layer of fingernail polish was applied on the surface of the leaf and let it be dry for 10-15 min. A crystal-clear tape was used as the desired area of the leaf surface and drawn off carefully. A 1 mm 2 grid was superimposed on the images for the calculation of stomata density [31]. Some example of captured imprint is shown in Figure 1. A total of 400 images are acquired with JPEG format; further, these images are preprocessed using Python libraries in preprocessing, and we resize the image to 512 × 512 and remove noise from the images for the further processes.

Proposed System
e objective of this study was to develop an automatic system for stomatal state identification, classification, and counting in microscopic imprints. e methodology flow of the proposed system is shown in Figure 2. e output of the system was stomata detection with their state (open and close) and total stomata in a microscopic imprint as shown in Figure 3.
e proposed system consists of the following steps: (i) Stomata detection (ii) State identification and counting (iii) State classification

Stomata Detection.
For stomata detection, DCNN (SSD-MobileNet V2) was implemented for creating a learned model using the transfer learning approach. SSD-MobileNet v2 takes one shot to detect multiple objects in the image. Its approach is based on a feed-forward convolutional network that generates the fixed-size collection of bounding boxes and scores for the object present in the bounding boxes [32]; the architecture of SSD is shown in Figure 4. e model consists of two parts: Firstly, it extracts the feature map and secondly applies convolutional filters to detected objects. e first step for training the model is to manually label the stomata region in the microscopic imprints. From labeling, two subfolders (train and test) were created. ese subfolders are used to encode the visual properties of labeled objects into feature vectors (train_label and test_label). e ratio of the data in the subfolders is 70 : 30, respectively. In the second step, the feature vector train label is used as the input for the learning model. e learned model for the stomata detection task is finally the classified model of the feature vector test label. e input to the model is stomata microscopic imprints and the output was the stomatal pore anatomy parameters, including the length and width of the stomata and aperture area.

State Identification and Counting.
e stomata state identification task was performed through the three following steps: (i) Detected stomata (ii) Measuring the stomatal and aperture area (iii) Measuring the corelation between stomata and aperture area e stomata detection is obtained from the learned model, which generates the binary image with the same size as the input image for each stoma in the image. Each pixel in the image is used as the coordinates point. Each point value is either true or false, where true values mean the area of stomata and aperture in the image and false values mean that there is no area of pores. Each input image generates "x" binary maps, where "x" is the number of detected stomata in the image. e learned model extracts the boundary coordinates of stomata and aperture. e coordinates values are Xmin, Xmax, Ymin, and Ymax. e width and height of detected stomata and aperture are calculated using the equations. e area of aperture (AoA) and the area of stomata (AoS) are calculated using equation (2); the area obtained from the coordinates values was in pixel; the area has been converted in µm 2 [16]; finally, the corelation between the stomata and their aperture area is measured by using equation (3).
x � set of stomata in an image. (3) e state of stomata is determined based on corelation between stomata and their aperture area because the stomatal pore aperture is affected by irrigation and biochar treatment [16] and all detected stomata in the image were counted using Python script.

State Classification.
e stomatal state classification was performed through the Support Vector Machine (SVM).
is study uses a linear SVM classifier to classify open and closed stomata. e learned model evaluates a total of 1490 stomata in a dataset with their states and save their features (height, width, area, states, and corelation) in CSV format for the classification task. e target values for the classifier were stomata states (open and closed) in the dataset. In SVM, we plot points of data items in n-dimensional space (where n is a number of features in the dataset) with the value of each feature of a particular coordinate. In the end, the stomata classification was performed by finding the hyperplane which differentiates the two classes as illustrated in Figure 5. e support vectors in the graph are the simple coordinates of individual observation. e trained classifier is the frontier that best segregated the two classes as hyper plane/line.

Results and Discussion
e proposed system has correctly identified, counted, and recognized stomata state when stomata are present in an image.

Stomata Detection and Counting.
e stomata detection and counting results are shown in Figure 6. e proposed model achieved a good performance regarding stomata detection, counting, and state identification. A total of 120 test images were used to evaluate the model performance.    Complexity rough statistical analysis of 118 images, an average of 1490 stoma were detected by our learned model. To analyze model performance quantitatively, the stomatal pore was manually labeled through the LabelImg tool. e annotation results were used as ground truth (GT). e GT results were compared with results obtained from the proposed model. e average precision (aP) was 98.6%, and average recall (AR) was 97.87 as shown in Table 1.

Stomata State Identification.
e proposed system automatically calculates the area of stomata and their aperture by which it recognizes how many stomata are open or closed in the image as shown in Figure 3. e state identification is performed by finding the corelation between stomata and aperture area as described in Section 3.2. e quinoa stomatal parameters of ground truth value and corresponding predicted values of model were compared, and the corresponding relative error of stomatal measurement was calculated as in Table 1. e calculation process was as per the following: (1) Each manually marked stomatal pore was compared to the results produced by the model according to maximum IoU, (2) through the stomatal state identification section and the formulas proposed in Section 3.2, evaluation indices, the anatomical features of the manually labeled stomata region, and the anatomical parameters from the learned model were obtained, and (3) the relative errors between these were calculated and the average

Accuracy.
e accuracy is the ratio of correctly predicted stomata to the total stomata, and the classifier accuracy was measured using the following equation:

Precision.
e precision is also known as a positive predicted value and it is the ratio of positive predicted value to the total predicted values. For the proposed classifier, the precision was calculated for open and close stomata using the two following equations: precision � true close true close + false close .

Recall.
Recall is a sensitivity and probability of detection, that is, true positive rate. It is the ratio of correct positive prediction to the total positive prediction. Recall was also calculated for both open and close stomata prediction by the model using the two following equations: recall � true open true open + false close , recall � true close true close + false open .

F1
Score. e F1 score is a measure of model accuracy on a dataset. e F1 score for the proposed classifier is calculated using the following equation: 4.3. Discussion. Stomata are important and functional parts of plant leaves; however, counting stomata is a hard task; scientists used many sampling and counting techniques to count stomata in the leaf. We developed a fully automatic system for stomata state identification and counting. Moreover, the system can be iteratively retrained with new imprints for the performance improvement and for adjusting the need of users. However, the previously proposed methods [25] do not identify the open and closed state of stomata. Our proposed system firstly detects the stomata and secondly it measures the aperture and stomata area, and, on the basis of their corelation, it predicts the stomatal state (open and close). e stomata states (open and close) perform a significant role in describing the plant health. At last, the states of stomata that are identified by the system were classified.

Conclusion
By all counts and proven results, we are presenting an automated stomata state identification, counting, and classification system for micrographs of quinoa leaves. e proposed system has identified the state and counted the stomata when stomata are present in the leaf image. e experimental results showed that the proposed system has provided more accurate measures of stomata parameters and provided more accurate information about the state of stomata in leaf micrograph. In addition, this method has an ability to process thousands of stomata in minimum time, reducing the need for researchers to manually measure stomata and speed up the process of analyzing the plant health. e system would be helpful for the plant biology community. In the future, we will develop a generalized  toolkit that supports more plant species by applying proposed method for extracting stomata information from leaf image.
Data Availability e microscopic images data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare no conflicts of interest.