Ground-Based Cloud-Type Recognition Using Manifold Kernel Sparse Coding and Dictionary Learning

,


Introduction
Clouds play an essential role in the circulation of water vapour and affects the earth's energy balance [1][2][3].In the study of weather forecasting and climate change, clouds are always regarded as the core factor [4]. e traditional cloud observation is much dependent on the observers' experience, and thus, it is time-consuming.
e substantial and noticeable development of hardware and digital imaging techniques makes it possible to observe the cloud automatically and continuously.Compared with satellite images, the ground-based images hold a high spatial resolution at a local scale [5].
Attributing to the application of sky imagers and ceilometer, automatic observation has been realized in cloud cover and cloud base height [6].To identify the cloud type accurately and effectively, many attempts have recently been made to address this challenging issue [5][6][7][8][9][10][11][12][13][14][15][16][17].Buch et al. [7] adopted texture, position, and pixel brightness features together from the whole-sky images and classified the data with decision trees.To test the texture feature extraction approach, autocorrelation, cooccurrence matrices, edge frequency, Law's features, and primitive length were applied for cloud recognition [8].Calbó and Sabburg [5] presented statistical texture features, Fourier transform features, and thresholded image features to identify the images taken by the whole-sky imager (WSI) and total sky imager (TSI).Heinle et al. [9] extracted 12 statistical features to represent the image's colour and texture and then employed the k-nearest-neighbour (KNN) classifier to distinguish seven different sky conditions.As a clear distinction exists in texture orientation between the satellite images and the ground-based images, Gabor-based multiple features were utilized for classification with support vector machine (SVM) and achieved an overall accuracy of 88.3% [10].Different from the typical local binary patterns (LBPs), weighted local binary patterns (WLBPs) [11] were presented by fusing the variance of a local patch to enhance the contrast for recognizing cloud types.Cheng and Yu [12] combined statistical features and LBPs with the Bayesian classifier to perform block-based classification (BC).In addition to texture features, Liu et al. [13] employed 7 structure features from the edge image to describe the structure characteristic of the infrared clouds.Zhuo et al. [14] indicated that simply using texture or structure features separately may not produce excellent classification performance.Hence, both texture and structure features were captured to obtain the cloud type with SVM.Furthermore, Xia et al. [6] and Xiao et al. [15] proposed to make use of multiple features together, including colour, texture, and structure features for cloud-type recognition, and the experiments validated that the integration of various features performed better than other cases.e physical features are also of great importance to represent clouds.Kazantzidis et al. [16] introduced the solar zenithal angle, the total cloud coverage, the visible percentage of sun, and the existence of rain in sky images to describe the physical property.Besides 12 image features extracted from the sky camera image, Tato et al. [17] combined another 7 cloud layer features from the ceilometer and adopted random forests for classification.
To represent the cloud image more effectively, Li et al. [18] put forward a discriminative model based on a bag of microstructures (BoMS), and it showed competitive performance in the cloud-type recognition.To make up the weakness of BoMS that it could not describe these complex categories well, the duplex norm-bounded sparse representation model [19] was reported.is model demonstrated promising classification performance and was validated to be capable of capturing the most prominent patterns from complex cloud categories and naturally to attain a higher accuracy.
Recently, the symmetric positive definite (SPD) matrix manifold has gained much popularity in the action recognition, object detection, face recognition, etc. [20,21].In addition, sparse representation on SPD matrix manifolds has been applied to these areas to achieve better performances [22,23].In spite of its effectiveness, the matrix manifolds method is seldom investigated to address the task of cloud classification [24].
In this paper, manifold kernel sparse coding and dictionary learning (MKSCDL) on the SPD matrix manifold is proposed for ground-based cloud classification.e rest of this paper is organized as follows: Section 2 introduces the dataset, and Section 3 describes the methodology of MKSCDL.Section 4 reports the experimental results and discussions.Ultimately, conclusions are summarized in Section 5.

Zenithal Dataset.
e zenithal dataset is provided by the National University of Defense Technology in China and acquired from historical ground-based infrared images taken by the whole-sky infrared cloud-measuring system (WSIRCMS) [25].e images are grouped into five categories: stratiform, cumuliform, waveform, and cirriform clouds and clear sky according to the morphology and generating mechanism of the cloud [26].
is dataset is composed of 100 cloud images in each category.e typical samples with a size of 320 × 240 pixels from each category are demonstrated in Figure 1.

SWIMCAT Dataset.
e SWIMCAT dataset [27] contains 784 images taken from a daytime WSI called the wide-angle high-resolution sky-imaging system (WAHR-SIS).e images are classified into 5 distinct categories: clear sky, patterned clouds, thick dark clouds, thick white clouds, and veil clouds.
e number of images in per category is 224, 89, 251, 135, and 85, correspondingly.e images were obtained over a period from January 2013 to May 2014.ey were selected based on visual characteristics and were categorized with the help of experts from Singapore Meteorological Services.
e representative samples with a size of 125 × 125 pixels from each category are demonstrated in Figure 2.

Method
In this section, the methodology is introduced mainly in three parts: feature extraction, dictionary learning, and classification, which is illustrated overall in Figure 3.

Feature Extraction and Stein Kernel.
Given an image I with a size of W × H, the feature image F is defined by computing d-dimensional features at every pixel: where f(I, x, y) is the feature mapping, for example: e covariance descriptor (CovD) of the feature image F is computed by the following equation: where μ represents the mean feature vector and n is the pixel amount of the image.
In general, the CovD is an SPD matrix.All SPD matrices form a Riemannian manifold S d ++ when endowed with a Riemannian metric.Note that the SPD matrix is adopted as the extracted feature to describe the image; therefore, it is different from traditional features used for classification in the Euclidean space.
In this paper, we adopt the Stein divergence as a Riemannian metric, and the SPD matrix manifold is mapped into the reproducing kernel Hilbert space (RKHS).e Stein divergence S is defined as follows: where A and B are the points on the SPD matrix manifold, and S(A, B) measures the distance between these two points.e Stein kernel is defined as follows:

Kernel Sparse Coding and Dictionary
Learning.In this section, we give a framework for manifold kernel sparse coding and dictionary learning (MKSCDL), which is outlined in Algorithm where x m is a sample from X, α represents the sparse coefficient, λ is a regularization parameter, and ‖ϕ(x m ) − ϕ(D)α‖ 2 2 denotes the reconstruction error.

Kernel Sparse Coding.
When the dictionary is fixed, the sparse coding problem is formulated as follows: e iterative shrinkage-thresholding algorithm (ISTA) [30] is adopted for the optimization solution.
Let h(α) � ‖ϕ(x m ) − ϕ(D)α‖ 2 2 , then the sparse vector is updated as follows: where t s is the step size, α (s) represents the sparse coefficient at the s-th iteration, and the shrinkage operator is defined as follows: Equation ( 10) is equal to T q (g) � max(|g| − q, 0)• * sign(g).Now, the problem is transformed to calculate the gradient of h(α) with respect to α: e first term of Equation ( 12) is Similarly, the second term of Equation ( 12) is As a result, the gradient ∇h(α) is obtained by adding the right-side terms of Equations ( 13) and ( 14). e sparse code is implemented with Equation (9).

Kernel Dictionary
Learning.First, the initial dictionary is achieved by k-means on Riemannian manifolds using the Frechet mean [29].It selects k points as the initial cluster centers from the training data randomly.en measured by Stein divergence, every point is allocated to the closest cluster center to recompute the corresponding cluster center iteratively based on the Frechet mean via the following equation: where ++ is a set of SPD matrices and m j is the updated cluster center.Ultimately, the m j When the sparse coefficient is fixed, the dictionary is updated atom by atom, and the dictionary learning problem is formulated as follows: We use ISTA [30] to update the dictionary as well.e Euclidean gradient of d i is As proved in [22], the Riemannian gradient grad J( e i-th atom d i is updated as follows: where t p is the step size and d (p) i represents the i -th atom at the p-th iteration.

4
Advances in Meteorology

Classi cation.
After the dictionary is learned from the training set, the sparse coe cient and the reconstruction errors "RE" of the testing set are obtained to predict the cloud type.Algorithm 2 details the procedure of classi cation.Since there are c types of the samples, the dictionary as well has c types, which are updated independently, as detailed in Section 3.2.2.
e sparse coe cient α * i of the testing sample x t is computed as follows: With the sparse coe cients on each kind of dictionary, the smallest reconstruction error indicates the class that the testing sample belongs to.
In this paper, the number of cloud classes c in the two datasets is both 5, and the reconstruction errors on 5 classes of the dictionary are compared to decide the cloud type.

Results and Discussions
In this section, compared with the baselines [11,12,27], the performance of MKSCDL is evaluated with the same experimental setup on two di erent image datasets: zenithal dataset, captured by WSIRCMS, and SWIMCAT dataset, gathered by WAHRSIS.Each experiment is implemented three times with 10-fold cross validation, and average values are treated as nal results.
Note that the feature map f is de ned independently on two di erent datasets due to the di erent nature between the grayscale and colour images.f(I, x, y) is de ned in Equation ( 2) to the grayscale image in the zenithal dataset, while in the SWIMCAT dataset f(I, x, y) [f(I R , x, y) T f(I G , x, y) T f(I B , x, y) T ] T to the RGB image, where I R , I G , and I B represent the intensity images in three channels, respectively.e parameters used in our experiments are set empirically as follows: β 0.1, λ 0.01, t s 10 −4 , and t p 10 −4 .

Performance of MKSCDL.
e rst experiment is carried out on the zenithal dataset.With diverse choices of the atom number in the dictionary, the performance of MKSCDL varies correspondingly.Figure 4 shows the overall accuracy on the updated dictionary with di erent sizes.It is illustrated that when the dictionary size k equals 14, the overall accuracy achieves up to 96.33%, which outperforms the other cases.
Figure 5 reports the confusion matrix when k 14. e element of row i and column j in the confusion matrix means the percentage of the i-th class recognized as the j-th class.As a result, the diagonal elements correspond to the recognition rates of all categories.It is illustrated all of the stratiform clouds and 99.3% clear sky images are identi ed correctly, which means that these two cloud types possess prominent characteristics to be distinguished.Likewise, the recognition rates of cumuliform, waveform, and cirriform clouds achieve more than 93%.On the whole, MKSCDL reveals an ideal performance in identifying the ground-based grayscale cloud images with a rather high accuracy.

Performance Comparison with the Baselines.
To assess the e ectiveness of MKSCDL further, WLBP [11] and BC [12] are applied for comparison: (i) WLBP [11] is the method fusing the variance of a local patch into LBP.e KNN classi er is employed for cloud classi cation based on the chi-square distance.

Advances in Meteorology
(ii) BC [12] integrates statistical and local texture features and adopts the Bayesian classi er with regularized discriminant analysis.Note that the statistical features have only 8 dimensions because the infrared images are lack of colour information.
Table 1 presents the comparison results of 10-fold cross validation.e performance of MKSCDL exceeds that of the other two methods especially with regard to cumuliform, waveform, and cirriform clouds.MKSCDL achieves the highest overall accuracy of 96.33% among them.at means the dictionary is learned well and the samples are described adequately on the corresponding dictionary instead of different types of dictionary, which makes contribution to the competitive performance.

Performance of MKSCDL.
e second experiment is conducted on the SWIMCAT dataset.Similar to the rst experiment, Figure 6 exhibits the accuracy on the learned dictionary with di erent sizes.As the dictionary size increases, the overall accuracy improves in general.When k is 20, MKSCDL performs best with a result of 98.34%.In consideration of excellent classi cation performance and computational cost of MKSCDL, we con rm that k 20 can satisfy the experimental requirement.
Figure 7 demonstrates the confusion matrix when k 20.Patterned clouds and thick white clouds possess obvious characteristics for discrimination with an accuracy of 100%.Likewise, the results of the clear sky and thick dark clouds achieve over 98%.In addition, the challenging veil clouds resembling clear sky [27] attain a decent result of 92.94%.It is shown that the misclassi cation rate for each class is rather low, which means that on the whole, the proposed method works well in categorizing the groundbased RGB images.

Performance Comparison with the Baselines.
Besides the two baselines mentioned in Section 4.1.2,the texton-based method [27] integrating both colour and texture features is used for comparison as well.Note that di erent from the grayscale images, the statistical features extracted from colour images in the BC method have 12 dimensions.
Table 2 presents the comparison results of 10-fold cross validation.By contrast, MKSCDL performs better than WLBP and BC overall.It is clear that MKSCDL has a strong power for the task of cloud categorization of the SWIMCAT dataset.
Input: SPD matrices X and atom number of the dictionary k Output: updated dictionary D Initialize the dictionary D 0 by k-means on Riemannian manifolds using the Frechet mean [29] Return label(x i t ) ALGORITHM 2: Framework for cloud-type classi cation.

Advances in Meteorology
To compare with the texton-based approach well, we also implement the experiment with the same configuration as that in [27], which chose a training set with 40 images per category while another 45 ones for testing randomly (40/45).Table 3 lists the results.In comparison with Table 2, the overall performance of different methods changes little.e textonbased method achieves perfect classification accuracy for all categories except veil clouds, whose accuracy remains to be improved.Viewing the results of MKSCDL, each cloud type acquires a fair high recognition rate.
Comprehensively, in comparison with the other three methods, MKSCDL is validated to be the most effective one in recognizing the ground-based colour images.

Conclusions
In this paper, a novel cloud classification named "MKSCDL" on manifolds is proposed.e SPD matrix is extracted from each image and acts as the image feature.To maintain the non-Euclidean geometrical characteristics of the SPD matrix, kernel sparse coding and dictionary learning are conducted to obtain a representative dictionary.e testing sample's reconstruction errors on different categories of dictionary are calculated to identify the specific cloud type.By comparing recent methods on grayscale and colour datasets, it is interesting to find that WLBP performs better in grayscale images, while BC performs better in colour images.Comparatively speaking, MKSCDL is suitable for both grayscale and colour images and is equipped with high capacity for automatic ground-based cloud classification.
e proposed method MKSCDL can be applied to the real life, and the results provided by the visual observation are adopted for comparison and evaluating the automatic classification method.With limited images, it may not produce perfect performance in cloud-type recognition.As the dataset becomes more representative and more adequate, it will work better and can satisfy the task of cloud classification well.In the future, multiple improvements should be considered for better automatic cloud-type classification.As for the feature extraction, more features like the gray-level cooccurrence matrix and Gabor filtering could be fused into the feature mapping to describe the image well.In terms of dictionary learning, the interclass difference can be taken into consideration for a more discriminative dictionary.What is more, samples' sparse coefficients can be applied to build the SVM model.In addition, complex sky condition existing in multiple cloud categories is supposed to attract our attention in the next work.
(x, y) is the pixel location; I(x, y) represents the pixel gray value; |I x |, |I y |, |I xx |, and |I yy | denote the first-and second-order derivatives of I(x, y) in the direction of x and y, respectively; and ��������� � |I x | 2 + |I y | 2means the modulus of the gradient.

Figure 3 :
Figure 3: Framework of the proposed cloud classification method.
1. Let X � x 1 , x 2 , . . ., x N   denote N points on S d ++ and D � d 1 , d 2 , . . ., d k   be a dictionary with k atoms.With the Stein kernel, we update the dictionary D by two steps iteratively: kernel sparse coding and kernel dictionary learning.e model is an optimization problem with ℓ 1 constraint: .