Improved GSO Optimized ESN Soft-Sensor Model of Flotation Process Based on Multisource Heterogeneous Information Fusion

For predicting the key technology indicators (concentrate grade and tailings recovery rate) of flotation process, an echo state network (ESN) based fusion soft-sensor model optimized by the improved glowworm swarm optimization (GSO) algorithm is proposed. Firstly, the color feature (saturation and brightness) and texture features (angular second moment, sum entropy, inertia moment, etc.) based on grey-level co-occurrence matrix (GLCM) are adopted to describe the visual characteristics of the flotation froth image. Then the kernel principal component analysis (KPCA) method is used to reduce the dimensionality of the high-dimensional input vector composed by the flotation froth image characteristics and process datum and extracts the nonlinear principal components in order to reduce the ESN dimension and network complex. The ESN soft-sensor model of flotation process is optimized by the GSO algorithm with congestion factor. Simulation results show that the model has better generalization and prediction accuracy to meet the online soft-sensor requirements of the real-time control in the flotation process.


Introduction
Based on the differences of the surface property of solid materials, flotation process is to separate useful minerals with gangue [1], in which the economic and technical indexes (concentrate grade and flotation recovery rate) are the key controlled indicators in the production process. Their control in the flotation process is mainly according to the flotation operators' experiences by observing the states (such as the color, the size, the flow rate, texture features, etc.) of the flotation froth on the flotation cell surface to adjust the flotation tank level and change the pharmacy addition. This method of artificial observation on flotation froth has limitations of the space, time, and subjectivity, and it cannot be organically combined with computer control system to achieve high-level control [2,3]. Inferential estimation (soft-sensor) technology can effectively solve the problem that the flotation process is difficult to online estimate the economic and technical indicators.
Domestic and foreign scholars apply digital image processing techniques to the froth feature extraction and the soft-sensor modeling of the key technical indicators in the flotation process and make a lot of achievements [4][5][6][7][8][9][10][11]. Hargrave and Hall study the diagnosis and analysis methods of the metal grade and quality and flow rate in flotation process by using the color and surface tissue. Then the statistical methods and mathematical models are utilized to find the relationship between parameters. The research results show that the color parameters of flotation froth can be used to forecast the concentrate grade in the beneficiation production process [5]. Bartolacci et al. use multivariate image analysis (MIA) and partial least squares (PLS) methods to establish the experience prediction model of flotation grade. On the other hand, the GLCM and wavelet transform analysis (WTA) methods are utilized to get the flotation froth characteristics [6]. Morar et al. utilize the machine vision method to predict the performance of the flotation process, such as concentrate grade and tailings recovery rate [7].  At home, Yang et al. aim at the question that bubble image quality is not ideal and the bubbles' size, shape, and gray scale are uneven in mineral flotation process and put forward a bubble image segmentation method based on the clustering presplit and the accuracy distance reconstruction. The size of the bubbles and other physical characteristics provide the basis for flotation control [8]. In that the multiple models additive can improve the overall prediction accuracy and the idea of robustness, Wang et al. present a multi-T-S fuzzy neural network soft-sensor model of flotation process based on the FCM clustering algorithm [9]. Yang et al. use the flotation froth video image features as auxiliary variables, establish a soft-sensor model of the flotation pulp pH value based on the sparse polynuclear least squares support vector machine (SVM), which combines the weighted local kernel function and global kernel function, and use Schmidt orthogonalization theory to reduce the multinuclear matrix [10]. Li et al. set up a soft-sensor mode by combining the principal component analysis (PCA) and extreme learning machine (ELM) methods [11].
The above established soft-sensor models of the flotation process only make use of the part of multisource heterogeneous information (real-time process datum, image feature information, and laboratory datum), not realizing information integration, coordination, and optimization of flotation process. The paper proposes ESN fusion soft-sensor method based on process datum and flotation froth image visual characteristic parameters (color features and texture features). Simulation results demonstrate the effectiveness of the proposed method.
The paper is organized as follows. In Section 2, the technique flowchart of flotation process is introduced. The ESN fusion soft sensing model of flotation process based on improved glowworm swarm optimization algorithm is presented in Section 3. In Section 4, experiment and simulation results are introduced in details. Finally, the conclusion illustrates the last part.

Technique Flowchart of Flotation Process
Flotation process is used to separate useful minerals and gangue based on the differences of the surface property of solid materials. Figure 1 is a typical iron ore flotation process consisting of the roughing, concentration, and scavenging [11]. The system input is the fine concentrate pulp which is early output of beneficiation process in the forepart. The pulp density is about 38% and concentrate grade is about 64%. Inlet pulp is fed into the high-stirred tank through the pulp pipeline by feed pump. At the same time, the flotation reagent according to a certain concentration ratio is also fed into high-stirred tank through dosing pump. On the other hand, the pulp temperature must reach a suitable flotation temperature by heating. If the dosage is appropriate, the flotation cells can output a grade of 68.5%-69.5% concentrate.
The control objective of flotation process is to ensure the concentrate grade and the tailings recovery rate are within a certain target range. In common, based on the off-line artificial laboratory to get grade values, the operators adjust the flotation cell level and the amount of flotation reagent addition. Due to the artificial laboratory for two hours at a time, when the process variables and boundary conditions change in the flotation process, they cannot timely adjust the flotation operation variables, which results in such phenomena that the flotation concentrate grade and the tailings recovery rate are too high or too low [11]. By analyzing the flotation technique, the process variables and boundary conditions mainly include feed grade 1 , feed flow rate 2 , feed concentration 3 , feed granularity 4 , and medicament flow rate 5 .

Structure of Soft-Sensor
Model. The structure of the proposed ESN soft-sensor model optimized by the glowworm swarm optimization algorithm is shown in Figure 2 [12]. The auxiliary variables of the proposed soft-sensor model are process variables, color features, and texture features (14 texture parameters based on the calculated gray-level cooccurrence matrix by flotation froth images, such as angular second moment, contrast, correlation, sum of squares, inverse difference moment, etc.). Then KPCA method is used to realize the dimension reduction of the high-dimensional input vector composed by the normalized auxiliary variables datum in order to reduce the ESN complex. Finally, the ESN network structure parameters are optimized by the improved GSO algorithm to realize the accurate prediction of the concentrate grade and tailings recovery rate in the flotation process.
Considering a multiinput multioutput (MIMO) system, the training sample set can be expressed as = { , | = 1, 2 . . . , }. is the output variable. represents the th input vector and can be expressed as = [ 1 , 2 , . . . , ] ( is the number of samples in the training set and is the number of input variables). Soft-sensing modeling requires a datum set from the normal conditions as the modeling data. Assume that the system has process variable and data vectors composing the test sample datum matrix ∈ × . In order to avoid the different dimension of the process variables affecting the results and obtain the easy mathematical treatment, it is necessary to normalize the datum. Set is the mean vector of and is the standard deviation vector of . So the normalized process variable is expressed as follows:̂= The input vector̂of the training samples is fed into the ESN to obtain the predicted output̂. Then the root mean square error (RMSE) is selected as the fitness of the softsensor model: where * is the actual output of training samples.

Extraction of Flotation Froth Color Features.
Flotation froth images are obtained by the CCD camera above the flotation tank, and the computer image acquisition card converts continuous analog signal into discrete digital signals, which is conveyed into the computer for the extraction of visual characteristics of flotation froth. Typical flotation froth image is shown in Figure 3 [13]. The froth images can be divided into three categories according to the flotation process and expert experience. (1) Bubble size is bigger, that is to say the big bubbles are mixed in the froth, the texture is shallow, texture is coarse, image complexity is small, color is pale, and SiO 2 of froth is less. In this case, the refined iron ore grade is low. (2) Bubble size is appropriate, more uniform, and stable, color is gray, the texture is fine, and the image is more complex. At this time, the flotation process is stable and the refined iron ore slurry grade meets the requirements. (3) Froth color becomes darker, even partial black, froth is finer, and even some bubbles are difficult to distinguish, and texture is very complicated. At this case, the SiO 2 content of froth is higher, although the iron concentrate grade is higher and the pharmaceutical is added excessively, which does not meet the economic requirements.
Flotation operators are mainly based on the color and gray closeness of the flotation froth, the luminance information of the froth surface, and the measured process variables to realize the real-time optimal control of the flotation process. Therefore, the froth color (or light intensity) reflects the information of the minerals concentration in the surface froth. The collected images are true color images, which adopt the red, green, and blue three components. But they are often closely related. In addition, the color information of 4 The Scientific World Journal hue, saturation, and intensity ( model) is relatively close to the people color visual perception. In model, is hue representing the different colors such as red, green, and blue; is saturation representing the color, such as magenta, red; is the brightness indicating the brightness level of the color. In industrial applications, the range of is [0, 1] corresponding from unsaturated to fully saturated state (without any white). The range of is [0, 1] corresponding from dark to light color. The conversion equations from to are expressed as follows: Thus, the saturation ( ) and brightness ( ) of the flotation froth images have got to be applied in representing the relationship between the concentrate grade, the tailings recovery rate, and the color characteristics of flotation froth images.

Extraction of Flotation Froth Texture Features Based on GLCM.
The texture statistical characteristics of the flotation froth image can reflect the working conditions of the flotation process. Image texture is formed by different gray values distributed in the space position and repeated alternate changes; thus two pixels will exist in a certain gradation relationship, which is known as the correlation characteristics of the gray space. Gray-level co-occurrence matrix (GLCM) is an important method used to analyze the image texture features, which is based on the second combination condition probability density function of estimated image [14]. Figure 4 is a GLCM schematic diagram, wherein and denote the gray scale value of the corresponding pixel.
The GLCM means a kind of statistical form of the joint distribution of two pixels, that is to say, the simultaneous occurrence probability ( , , , ) of two pixels. They are the pixel with gray scale from the image ( , ) and the pixel ( +Δ , +Δ ) with gray scale at declination and distance . The mathematical formula is expressed as follows: where , = 0, 1, . . . , − 1, and are the coordinates of the image pixel, and is the image gray level. According to the above definitions, the element in th row and th column of the constituted GLCM represents the appearance frequency of all pixels with the and gray level in the direction and length. GLCM has rich characteristics parameters describing the image textures with different angles. Haralick et al. [15] once proposed 14 GLCM based texture parameters, whose calculation formulas are shown in Table 1.

KPCA Based Dimension Reduction of Soft-Sensor Model.
The visual characteristic parameters (2 color features and 14 texture features) of the flotation froth images and 5 process variables are served as the input variables of the ESN fusion soft-sensor model to predict the concentrate grade and flotation recovery rate. A batch of flotation froth images and the measured values of the process variables in corresponding period are collected to establish the soft-sensor model. The input-output samples are shown in Table 2.
The flotation froth image characteristics and process variables and boundary conditions are selected as the auxiliary variables of the proposed soft-sensor model to realize the integration of multisource heterogeneous information in the flotation process. But there are the problems of the jumbled information and repeated expression. If the input vector dimension of the ESN model is too long, the network topology will be complex and training will become very complex. Therefore, the kernel principal component analysis (KPCA) method [16] is adopted to reduce the model dimension of the ESN soft-sensor model.  KPCA is a nonlinear promotion of introducing the concept of the kernel function into the principal component analysis (PCA) method, which has better ability to handle nonlinear problems than PCA. Its basic principle is described as follows [17].
Given sample set ( = 1, 2, . . . , ) and ∈ , the nonlinear mapping relation is given as follows: So the sample is mapped as ( ). Then the covariance matrix of new sample space is calculated according to the following equation: The eigenvalue decomposition is carried out according to the following equation: 6 The Scientific World Journal where ( > 0) is the eigenvalue of and is the corresponding eigenvector. By multiplying ( ) on both sides of (7), we obtain the following: And coefficient ( = 1, 2, . . . , ) exists to make the following equation: By combining the above two equations, matrix ( × ) is defined as follows: Set is the corresponding eigenvector of the kernel matrix . Then, consider the following: where = ( 1 , 2 , . . . , ) .
Put = ∑ =1 ( ) and = ( ( ) ( )) into (12) to obtain the following: The principal component of a new sample is obtained by projecting mapping sample ( ) of into , which is described in the following equation: For the sake of simplicity,̂= − − + is used to substitute kernel matrix of all mapping samples, among which ( ) = 1/ . The paper adopts the Gaussian function as the KPCA kernel function, which is described as follows: Based on the above mentioned principles, the procedure of KPCA algorithm is described as follows.
Calculate kernel matrix̂; calculate eigenvalues and eigenvectors of kernel matrix̂; sort eigenvalues with the descend order. Assume that . Calculate the contribution ratio by (16) to decide the number of the extracted character information ( ). Consider the following: (1) The eigenvectors in accordance with the previous (1 ≤ ≤ ) biggest eigenvalues are normalized according to (13).
The historical datum of input variables in the soft-sensor model is carried out by kernel principal component analysis, whose results are described in the Table 3. It can be seen that The Scientific World Journal 7 the contribution ratio of the previous 5 principal components has already exceeded 90%. Thus, the principal components obtained by the KPCA on the original variables datum are the input of the ESN, which not only reserved the character information of original variables but also simplified the structure of ESN. [18][19][20]. Its internal dynamic reserve (Dynamic Reservoir, DR) pool has a large number of sparse connected neural units, which contain the operational status of the system and have the short-term memory function (the ESN echo effect). The echo effect makes the network realize the approximation effect on the learning system. A typical ESN structure is shown in Figure 5. Its basic equations can be represented as follows:

Echo State Network. Echo state network (ESN) is a new type of recurrent neural network proposed by Jaeger
( + 1) = ( in ( + 1) + ( ) + fb ( )) , ( + 1) = out ( out ( ( + 1) , ( + 1) , ( ))) , where is the DR internal activation function, usually using the Sigmoid type function to make the ESN have good nonlinear characteristic; ( ) is the DR state variable on time; ( ) is the system input vector on time; ( ) is the network output; in ( × ) is the input weight matrix; ( × ) is the connection matrix among the DR internal neurons, which usually keeps the sparse connection of 1%∼5% and the spectral radius less than 1 in order to make the DR have dynamic memory ability; fb ( × ) is the feedback matrix between the output neurons and DR; out is the activation function of the input and output units, usually using a linear function; out ( × ( + + )) is the output weights matrix. in , , and fb are constructed before the network learning, but out is calculated after learning period.  so on. GSO has been successfully used on many fields, such as multimodal function optimization and multisource tracking and location [21,22]. Suppose the number of fireflies is , which is randomly distributed in the search space of objective function. ( ) represents the location of the th firefly, ( ( )) is the fitness function, and ( ) is the fluorescein concentration of the th firefly at the moment . The movements of fireflies are updated according to the following equation:

ESN Soft-Sensor Model Optimized by
where ∈ (0, 1) is the volatilization coefficient of fluorescein and is the enhancement factor of the volatilization coefficient. Suppose is the perception scope of fireflies and ( ) is the dynamic decision range (namely, decision radius) belonging to the th firefly at the moment , whose upper bound of the perception scope is (0 < ( ) < ). So the updating formula of decision domain range is represented as where is the changeable rate of field, is the neighborhood threshold controlling the neighbor number of fireflies, and ( ) is neighbors set of the th firefly at the moment . Then the formula determining the number of fireflies within the decision domain is where ‖ ⃗ ‖ is the norm of ⇀ . During the movement of fireflies, the fluorescein concentration of each firefly in its neighbor set determines the moving direction. Suppose that ( ) is the moving probability of the th firefly moving to the th firefly in the neighbor set at the moment , which is calculated by the following equation: Based on the moving probability ( ) , the roulette method is adopted to decide the moving direction of the th firefly. 0 is the initial fluorescein value. Suppose the moving step . Thus the following formula determines the location of the th firefly at the moment + 1: 3.6.2. Crowded Degree Factor. The crowded degree factor is introduced to avoid the local optimum phenomena caused by the overcrowding fireflies, which can make fireflies located near the optimum point reject each other. Its principle is shown in Figure 6. For maximum optimization problem, consider the following: where is the close level to the optimum value and max is the maximum number of fireflies belonging to its neighbor field. Suppose is the states of fireflies themselves, is the preceptor state value, and is the number of partners in fireflies' neighborhood. If /( ) < , is the overcongestion state. When it comes to the minimum optimization problem, = max , ∈ (0, 1]. When /( ) > , is at the state of over congestion.
Unifying the crowded degree factor and the number of fireflies in their neighborhood, the behavior of the fireflies attracting each other determines its influence on the optimization results. In Figure, the firefly 0 is the best one among the fireflies 1 , 2 , 3 , 4 , and 5 , whose attracting degree is . 1 and 2 are rounds having the same center 0 and different radiuses. The closer to the round center the much greater attraction to the fireflies.
If ≤ 1, all fireflies are attracted to 0 . If > 1 and 2 (attracting degree is / ) is the attracting degrees circle, the fireflies between 1 and 2 are attracted to 0 . At this moment, the larger is, the less the fireflies are attracted. If / ≤ 1, all fireflies in the vision are attracted to 0 . If / > 1, the fireflies, whose degree is greater than / , are attracted to move to 0 . The larger / is, the less the fireflies are attracted.

Algorithm Procedure.
The main parameters of ESN soft-sensor model are the input weight matrix in , the DR pool weight matrix , the output feedback weight matrix fb , and the output weight matrix out . There are two kinds of cases to optimize the ESN: one is to optimize in , , and fb ; the other is to optimize out . The paper adopts the locations of the fireflies in the improved GSO algorithm to correspond with the output connection weights matrix out of ESN during the ESN training stage. Through the optimized search, the output weight matrix out of ESN is trained in less samples and time. Its algorithm procedure is shown as follows.
Step 1 (initialize the algorithm parameters). Initialize the parameters in , , and fb of ESN, the parameters , , , , , , 0 , and of GSO, and the maximum iteration time max .
Step 3 (calculate fitness). Each firefly ( ) is set as the output weight matrix of ESN, and then the training samples are fed into the ESN soft-sensor model. The predict output is calculated by (17), and the fitness value ( ( )) is calculated by (2). In the end, (18) is used to convert the objective function values ( ( )) of firefly ( ) into the fluorescein value ( ).
Step 4 (update of the individual firefly position). Each firefly within ( ) makes up its neighborhood set ( ) (0 < ( ) < ) according to (20) by selecting those fireflies whose fluorescein values are higher to itself, and ( ) is regulated based on the crowded degree factor. The probability ( ) that the th firefly moves to the th firefly in its neighborhood at the moment is calculated by (21). The roulette wheel method is used to select individuals to move. Then the location is updated according to (22). In the end, the dynamic decision domain radius is updated according to (19).
Step 5 (judge the termination conditions of the proposed algorithm). If it meets the termination conditions (e.g., it reaches the maximum iteration number max ), the best firefly is recorded. Otherwise, = + 1 and go to the Step 3.

Simulation Results
With a typical flotation process as the research object, an ESN fusion soft-sensor model is established for predicting the concentrate grade and the flotation recovery rate. Firstly, the 300 input-output datum sets are determined as shown in Table 2 [17], which are defined as follows: where is the number of the predictive samples, ( ) is the predicted number, and ( ) is the actual sample values. The input dimension of ESN is 5 and the output dimension is 2. Moreover, the size of the DR pool is 100, the sparse connection rate of weight matrix of DR pool is 5%, the activation function of DR pool is tanh(), and the output unit uses the linear activation function. The initial values of parameters of ESN are selected as follows: in = 0.3, = 0.2, and fb = 0.03. The initial values of parameters of GSO are selected as follows: = 100, = 0.4, = 0.6, = 0.08, = 0.03, = 5, 0 = 5, and = 0.2. The maximum iteration time max = 500.
To illustrate the effectiveness of the proposed soft-sensor model, the improved glowworm swarm optimization (IGSO) based ESN soft-sensor model is compared with the original ESN method and the glowworm swarm optimization (IGSO) based ESN soft-sensor model. The predictive outputs and actual outputs under three methods are shown in Figure 7. The predictive error curves are shown in Figure 8. The prediction accuracies of three methods are shown in Table 4. Seen from Figures 7 and 8 and Table 4, the proposed IGSO-ESN soft-sensor model has higher predictive precision and generalization ability for the key technique index (concentrate grade and flotation recovery rate) of the flotation process than ESN soft-sensor model and GSO-ESN soft-sensor model. The proposed GSO algorithm with the crowded degree factor  can adjust the structure parameters of the soft-sensor model effectively.
In order to highlight the superiority of the proposed method, the comparisons have been made among IGSO-ESN soft-sensor models with two swarm intelligence based ESN soft-sensor models (GA-ESN and PSO-ESN). The predictive outputs and errors curves under three methods are shown in Figures 9 and 10. The predictive simulation has been carried out 10 times. Then the statistics analysis results of the model performances with 10 runs are listed in Table 4 based on the definition of predictive performance index. Seen from the simulation results, the proposed IGSO-ESN predictive model has higher accuracy than the GA-ESN and PSO-ESN soft-sensor model. The successful adoption of the predictive model in the flotation process for obtaining the real-time concentrate grade and flotation recovery rate has important significance in the field of improving the production capacity and reducing production costs.  Then the GSO algorithm with the crowded degree factor is used to optimize the ESN soft-sensor model.