Moisture Content Quantization of Masson Pine Seedling Leaf Based on Stacked Autoencoder with Near-Infrared Spectroscopy

Masson pine is widely planted in southern China, and moisture content of the pine seedling leaves is an important index for evaluating the vigor of seedlings. For precisely predicting leaf moisture content, near-infrared spectroscopy analysis is applied in the experiment, which is a cost-effective, high-speed, and noninvasive material content prediction tool. To further improve the spectroscopy analysis accuracy, in this study, a new analysis model is proposed which integrates a stacked autoencoder for extracting hierarchical output-related features layer by layer and a support vector regression model to leverage these features for precisely predicting moisture contents. Compared with traditional spectroscopy analysis method like partial least squares regression and basic support vector regression, the proposed model shows great superiority for leaf moisture content prediction, with R value 0.9946 and root-mean squared error (RMSE) value 0.1636 in calibration set and R value 0.9621 and RMSE 0.4249 in prediction set.


Introduction
Masson pine is an important species for forestation in southern China due to its broad distribution.It has been widely applied in pulp and building materials, petroleum extraction, and forestry chemicals [1] because of its desirable features including fast growth, high yield, variable light tolerance, and drought resistance [2].To ensure the quality of masson pine seedlings, the moisture content of seedling leaves is an important evaluation index and should be assessed before plantation.Physiologically, leaf moisture content plays an important role in photosynthesis, material transport, and maintaining leaf morphological and physiological functions [3].Traditional moisture measurements are almost chemical-based methods, which are complex, time-consuming, and laborious [4,5].As a result, it is urgent to establish a fast, nondestructive, and accurate measurement method for moisture content prediction.
e near-infrared (NIR) spectroscopy technique has been widely used for material content prediction, such as moisture content [6], nitrogen content [7], and chlorophyll content [8] of plants.To date, many researchers have constructed different NIR spectral calibration models to detect the moisture content of different plants such as sunflower [9], populous tree [10], and eucalyptus tree [11].Although there are many successful spectroscopy applications, there is no research reported for the moisture content prediction of masson pine seedling leaves.
For spectroscopy data analysis, there are many common used calibration models such as partial least squares regression (PLSR) [12], multiple linear regression (MLR) [13], support vector machines for regression (SVR) [14][15][16], artificial neural networks (ANNs) [17], and so on.However, the prediction results from these models are hopeful to be improved due to the limitations of the models.PLSR and MLR are widely used methods in the spectroscopy area.Both of them are linear models, which means the model can hardly describe the nonlinear relationship between the input spectroscopy data and output material content value.SVR, a nonlinear model, is suitable for small dataset analysis.It usually has strong generalization ability and self-learning capacity.e performance of SVR model usually depends on the quality input features, and determined representative data features are the premise for training a good SVR model [16].ANNs offer a simple solution to model complex nonlinear data relationship [18].e multiple layers in the ANN model can map the data input to the high-level feature representations.With the increasing number of hidden layers, the imbalance between the trainable network parameters and the limited training samples hinders the network training process in practice [19].Stacked autoencoder (SAE) [20] is a variant of the ANN model, which usually is composed of an encoder and decoder.e encoder can map the input into few high-dimensional data features, and the decoder is expected to reconstruct the raw data.SAE can be trained without supervision, and the high dimensional data features from the encoder have been successfully applied in different tasks [21][22][23][24].
In this paper, to combine the advantages of different existing data analysis methods, a new model is proposed for predicting the moisture content in masson pine seedling leaves.In the model, a SAE is pretrained layer by layer without supervision to get the high-level data features.en, based on the properties of the ANN model, the high-level data can be fine-tuned in a supervised way.Once the highlevel data features are determined, the SVR model is established to describe the relationship between the highlevel data features and the moisture content.Compared with other traditional spectroscopy analysis methods, the proposed model shows better performance.

Materials.
A total of 100 annual masson pine seedling samples were obtained from forest farm in Huangping, Guizhou Province, China.Seventy-five of them were used to establish a calibration model, and the remaining 25 were used for prediction.Before measuring the spectra and moisture content, the leaves were cleaned to remove the impurities such as soil and sand, and then the surface of the leaves were naturally dried to prepare for the later experiment.

Spectra Acquisition and Moisture Quantitative Analysis.
All NIR spectra of masson pine seedling leaves were measured using an MPA Fourier-Transform Near-Infrared spectrometer (Bruker Optics, Inc., Germany), equipped with an PbS detector and controlled by OPUS Analyst version 7.0 using a spectral range from 4,000 to 12,493 cm −1 in the reflectance mode.Measurements were conducted with the resolution of 4 cm −1 , ensuring an adequate signal-to-noise ratio.ere are 2203 wavelength variables in the spectrum.e experiment was carried out at approximately 24 °C.
e spectral reflectance of the top, middle, and bottom areas of the sample were scanned two times each.e final value of the sample spectral reflectance was obtained by calculating the average of the six scans.From each sample, six spectra were collected and averaged for further analysis.Figure 1 shows the raw absorptance spectra of masson pine seedling leaves.
Total moisture content of masson pine seedling leaves was determined using the HB43-S Halogen Moisture analyzer (Mettler Toledo, Inc., Switzerland).e instrument is working based on the thermogravimetric principle, i.e., the moisture is determined from the weight loss of a sample dried by heating.After putting a sample in the sampling chamber, the temperature was immediately increased and maintained at 125 °C.e halogen lamp of the instrument heated the sample until the sample stopped losing mass; then the moisture content of the samples is automatically given by the instrument.Normally, the measurements were completed in a few minutes.
e reference moisture content value for the samples was 64.97% ± 2.21%, with a minimum of 59.13% and a maximum of 70.74%.

Spectra Pretreatment.
Two preprocessing transformations were applied as a standard preparation for the masson pine seedling absorbance curves.Firstly, a Savitzky-Golay (S-G) smoothing with a second-degree polynomial was used for denoising, and the first-order numerical derivative was made to correct the baseline drift.e window width of S-G smoothing was set as 17.Secondly, the spectral matrix was converted into the number between 0 and 1 by vector normalization, which was used to reduce the orders of magnitude difference among the data of different dimensions.

Autoencoder.
e basic structure of autoencoder is a kind of unsupervised ANN with one hidden layer, and it consists of an input layer, a hidden layer, and an output layer as Figure 2. e aim of autoencoder is to reduce the data dimension and to map the input data into high dimensional data features.
Define the input as x � [x (1) , x (2) , . . ., x (d) ] ′ , where d is the dimension of the inputs.e encoder maps x into the hidden layer h � [h 1 , h 2 , . . ., h d h ] ′ by the function f as follows: where d h is the dimension of the hidden layer variable vector h, W is a d h * d weight matrix, b is the bias vector, and s f is the nonlinear activation function, which can be chosen as the sigmoid function or other functions such as the tanh function and the rectified linear unit function.en, the hidden representation vector h is mapped to the output layer  x by the function  f.
where  W is a d * d h weight matrix,  b is the bias vector for the output layer, and s f  is the nonlinear activation function of decoder.
e aim of autoencoder is to search the pa- where N is the number of the training samples and x i � [x i(1) , x i(2) , . . ., x i(d) ] is the vector data of ith training sample.e loss function is defined as follows: 2 Journal of Electrical and Computer Engineering (3) en, the parameter set θ � W,  W, b,  b   is updated with the gradient descent algorithm.
In practice, multiple basic autoencoder structures are usually stacked together, and a new network structure is constructed, named SAE, for better dimensional reduction and feature extraction performance.
e extracted data feature can be further used for different tasks [25,26].

Support
Vector Machine Regression.Support vector machine (SVM) is a powerful and robust method based on the principle of structural risk minimization, which has an advantage of computational efficiency for the data sets that have many more variables than observations.Compared with ANN, SVM can obtain more reliable and better performance under same training conditions.Recently, SVM has been successfully extended to SVR, especially in chemometrics for quantitative analysis due to its excellent ability of dealing with nonlinearity and small sample size.e purpose of SVR is to find the underlying relationship between input and output, and a regression function y(x) � f(x) + b � w T ϕ(x) + b is used with an ε-insensitive loss function as follows: en, the regression problem is equivalent to the following formulations: where ξ i and ξ * i are slack variables, and the regularization parameter C > 0 is used to avoid overfitting.Further, the optimization problem can be converted into the new optimization problem as the following: where α i and α * i are Lagrange multipliers and K(x i , x j ) is a kernel function.e kernel function maps the nonlinear optimization problem to a linear problem in higher dimensional space.ere are several commonly used kernel functions such as linear kernel, polynomial kernel, and radical basis function kernel.In this study, radical basis function kernel is selected as follows: where c determines width of the kernel function.Once radical basis function kernel is selected, there are only three parameters to be decided.e parameter ε is to control the error.A larger ε can speed up the train process with low accuracy.Whereas, a smaller ε can achieve better accuracy and reduce the training speed.In this paper, the parameter is set fixed as 0.004.Besides the parameter ε, the regularization parameter C and the kernel function parameter c are both application-based parameters, which can greatly affect accuracy of the regression.In this paper, genetic algorithm is utilized to search optimal parameters for SVR [27].x( 2) x (2) x (d)

Regression Model Based on SAE with SVR.
To estimate moisture content in masson pine seedling leaves, a novel regression model based on SAE with SVR is proposed as shown in Figure 3. e procedure of the proposed method is described as follows: Step 1. Several AEs are stacked layer by layer to form deep neural networks.
e deep neural networks can convert a complex input data into a series of simple high-level features by reducing the dimensionality of the input data.To train the networks, the first AE is trained in an unsupervised manner.After training is finished, its decoder is abandoned, and the output of the hidden layer is used as the input of the second AE with same fashion.Till all AEs have been trained layer by layer, the weights of these AEs had been assigned to initialize the deep neural networks.e topmost hidden layer outputs the elementary high-level features of the input data.
Step 2. To achieve better feature representation of the input data, the supervision method is used to tune the weights.A two-layer neural network is added after the topmost layer, and the output of this neural network is the target output.
en, the back-propagation algorithm is utilized to update the weights layer by layer.At last, the fine-tuning weights have been obtained, and then the improved high-level features have been achieved.
Step 3. Feed the improved high-level features into SVR as input data, and then build the regression model using GA algorithm to select the optimal parameters.

Evaluation Criteria.
As normal, the root-mean squared error (RMSE) and the coefficient of determination are widely used as the evaluating criteria for the calibration model.RMSE is defined as follows: where y i and  y i are the actual and predicted target output values of the ith sample, respectively and N T is the number of the testing samples.RMSE indicates the accuracy of the model, and a small RMSE value shows better prediction performance than larger RMSE.e coefficient of determination R 2 represents a squared correlation between the actual and predicted output, and the reliability of model can be reflected as R 2 .R 2 is defined as follows: where y is the mean of the actual output in the testing samples.When R 2 is closer to 1, it indicates good prediction performance of the calibration model.In a word, a better calibration model should have small RMSE and large R 2 .

Result and Settings of the Proposed Model.
For the proposed model construction and test, 100 samples were divided into 75 samples for calibration dataset and 25 samples for prediction dataset.e first step is to decide the settings of the model.It is evident that the number of the hidden layers and the number of the neuron in each hidden layer can have major influence on the performance of the model.However, there is still no automatic method that can be utilized to achieve the selection of the parameters.In this paper, trial and error is used to choose these parameters, and RMSE is adopted to access the performance of the model.e weights and bias for each AE is obtained by greedy layerwise pretraining technique, and each AE is trained with gradient descent algorithm.After the SAE is layerwise trained, the two-layer neural network is connected to the output of SAE for weight fine tuning of ANN.After that, SVR is connected to the high-level features to output the predicted value.After experiments, the proposed model structure is composed of three-layer AEs, and the hidden neurons number is 1600, 1100, and 512, respectively.e batch size is set as 40 samples, and each AE is trained 500 iterations iteratively.Figure 4 shows the training loss trends with the iteration number for each AE.As shown in Figure 4, the SAE model can converge quickly in 500 iterations for each AE.
After the weights are pretrained and fine-tuned, the SVR is connected to output the predicted value.
e detailed prediction results are shown in Figure 5, and C and c of SVR are set as 4.79 and 1.97, respectively, using GA algorithm.
e RMSE and R 2 of the prediction dataset are 0.4249 and 0.9621, respectively.As can be seen from Figure 5, the predicted output can match well with the real values.e prediction error is mainly caused by the points which are located in the interval with few samples in the calibration dataset.

Discussion.
To verify the effectiveness of the proposed method, there are several methods to be used to build the calibration model.Each method is trained with the same calibration dataset, and the average result of 4-folder crossvalidation in each method is adopted in Table 1 and Figure 6.
e results of MLR, PLS, and SVR are directly obtained using PLS_Toolbox (Eigenvector Research, Inc., USA) running on MATLAB 2016.
e other methods are implemented by the deep learning framework Keras using Python.As can be seen in Table 1, MLR and PLS produce the worst prediction results since they cannot deal with nonlinear correlative data for its linear essence.SVR is inferior to other methods and just outperforms above two methods, because it cannot describe the nonlinear data adequately.
e last three methods all adopt the same network structure.e ANN method is composed of five layers of neural networks, and the first three layers have the same network structure as SAE.e last two layers have the same structure with neural networks as being connected to SAE. e same two-layer neural networks with the neuron layer structure [512,128] are connected to SAE for fine tuning the weights.ANN method outperforms above three methods, but it is easily trapped in local optima.By adopting the SAE, highlevel abstract features can be extracted layer by layer in SAE.Further, by using the pretrained weights and bias terms in the full connected network, the methods can avoid the inferior local optima and speed up the learning process.us, the last two methods can describe the complex data structure more accurately.Besides, a small size of training samples used in this study may also limit the predictive power of the ANN model.en, SVR with genetic algorithm is connected to SAE to achieve better performance in small training samples.e regression plot between the predicted moisture content and the reference content is shown in Figure 6.It can visualize the regression performance of different models.If the scatter data points are closer to the perfect correlation line (with slope � 1), the prediction model is more accurate (larger R 2 value and smaller RMSE

Conclusion
In this study, a new method of stacked autoencoder with SVR is introduced and applied to estimate the moisture content in masson pine seedlings.Compared with MLR, PLSR, SVR, ANN, and SAE-ANN, the SAE-SVR shows superior performances, and gets R 2 value 0.9946 and RMSE 0.1636 in calibration dataset, and R 2 value 0.9621 and RMSE 0.4249 in prediction dataset.e results of this study demonstrate that deep neural networks pretrained by SAE are feasible to be used as the data analysis method for moisture prediction in masson pine seedling leaves.

Figure 3 :Figure 4 :
Figure 3: e proposed network structure of deep learning with SAE-SVR.

Figure 5 :
Figure 5: e prediction performance on the calibration dataset (a) and the prediction dataset (b).

Table 1 :
Calibration and prediction results of the moisture content in masson pine seedling leaves using different regression models.