ResearchArticle Personal Communication Technologies for Smart Spaces Density-Based Clustering for Content and Color Adaptive Tone Mapping

,


Introduction
e demand for high dynamic range (HDR) images is rapidly increasing with the advent of sensors, display technology, and availability of multiple exposure photography. HDR images provide better visual quality as compared to their counterpart low dynamic range (LDR) images on account of their large ranged brightness levels and better preservation of color variations. is enables to store and visualize bright and dark objects in the same image which would normally require multiple images at different exposures to be in low dynamic range. Visualization of HDR images on standard display devices (made for displaying LDR images) is challenging as they are incapable of displaying the high dynamic range of the image. To address this issue, tone mapping operators are used to convert HDR images to LDR images. However, the technology that directly displays HDR images is becoming more cost-effective. Recently, most of the available display devices have LDR, which is why HDR to LDR conversion still has practical applications.
For visualization of HDR images on LDR devices, Retinex theory has been widely used, which slices the image into its components: base layer and detail layer. Edge preservation filtering techniques are used for the approximation of base layer in Retinex image decomposition. However, these methods have resulted in false coloring and halo artifacts because filtering is unable to capture complex local structure of the image. To overcome this problem, we need a method that can better adopt the local structure and color variation of HDR images in LDR images.
Furthermore, tone mapping operators are generally divided into two main categories: global tone mapping operators and local tone mapping operators. Global tone mapping operators use the same monotonic curve for dynamic range compression of the entire image [1,2]. Initially, the focus of the research was on the design of global tone mapping operators. Drago et al. [3] proposed a tone mapping algorithm that used an adaptive logarithmic base for luminance compression while maintaining the image's details and contrast. Kim and Kautz [4] introduced a tone mapping operator based on the hypothesis that the human visual system adopts Gaussian distribution. Reinhard et al. [5] proposed the use of spontaneous dodging and burning for dynamic range compression. e terms "dodging" and "burning" originate from printing, where holding backlight from a region of print is called dodging and the addition of light to image is referred to as burning. is can be used to increase or decrease the brightness of a captured image effectively. Reinhard and Devlin [6] introduced the light adaptive tone mapping which provides satisfying visual effects. Khan [7] method by considering the impact of ambient light on HVS. Generally, global tone mapping operators are computationally efficient [9]. However, they are unable to preserve the local characteristics and dynamic range variations as they ignore local pixel intensity variations in an image [9,10].
Local tone mapping operators solve the issue of global tone mapping operators by incorporating the ratio between neighboring pixels and compressing each pixel. Gu et al. [11] and Min et al. [12] proposed local tone mapping operators based on layer decomposition, where the input image is divided into base layer and detail layer. e layer-decomposition-based approach has also been utilized by authors in [13][14][15][16]. Durand and Dorsey [17] proposed a method that preserves the detail layer by encoding large-scale variations of the base layer. Debevec and Gibson [18] proposed a method that helped to preserve brightness and details of image by applying a local luminance adaption function for compression of dynamic range, followed by reinjecting details in the low dynamic range image. Image color appearance model (iCAM06) [19] employed tone mapping by using iCAM06 color appearance model by considering the viewing conditions to generate optimal results. However, iCAM06 [19] led to the poor visual quality due to introduction of halo artifact, color saturation, and gradient reversal at the edges. To eliminate these problems, the authors in [20] proposed an iCAM06 based model. In their proposed algorithm guided filter [20], color adaptive transformation matrix and HPE primitives were altered to enhance the effectiveness of model.
Krawczyk et al. [21] introduced an algorithm based on anchoring theory that decomposed image luminance into patches and then calculated the lightness for each patch. Li et al. [22] proposed a symmetrical analysis-synthesis filter for reducing the intensity range. Jia and Zhang [23] used the guided image filter for tone mapping. Meylan et al. [24] suggested a method based on the characteristic of human retina to reduce dynamic range while increasing the local contrast of the image. Another retina inspired range compression algorithm is used [25] to overcome the halo artifacts, loss of details, different visualization across different displays, and color saturation.
Parraga and Otazu [26] developed a tone mapping method based on human color perception by dividing the image intensity into multiresolution contrast. Also, they had used a nonlinear saturation model for dynamic range reduction based on visual cortex behavior. Chen et al. [27] introduced Earth mover's distance to segment HDR image and then applied local tone mapping on each component of the image.
is maintained local texture and balanced perceptional impression. To remove artifacts and contrast enhancement, Liang et al. [28] used nonlinear diffusion. l 2based retime model [29] was used for contrast enhancement. Ahn et al. [30] proposed a Retinex-based adaptive local tone mapping algorithm according to which the use of guided filter reduced halo artifacts. Recently, Liang et al. [31] proposed a hybrid l 1 − l 0 norm-based layer decomposition model. l 1 sparsity term was imposed on the base layer and l 0 on the detail layer. In [32], the authors proposed the use of decomposed multiscale Retinex for information preservation in tone-mapped image. Shu and Wu [33] employed an optimal local tone mapping operator to avoid halo artifacts and double edges. Rana et al. [34] proposed a pixel-wise adaptive tone mapping operator based on support vector regression to overcome the problem of drastic illumination variation.
El Mezeni and Saranovac [35] introduced an enhanced local tone mapping operator (ELTM). ELTM decomposed an image into detail and base layers, where the base layer was compressed into both logarithmic and linear domain. Local tone mapping operator presented in [36] was operationally similar to HVS. Li and Zheng [37] presented a saliencyaware local tone mapping operator that preserved edges using guided filters. Ferradans et al. [38] proposed a twostage algorithm incorporating both global tone mapping and local tone mapping. In the first step, they applied a global tone mapping operator based on the human visual system and followed by local method for contrast enhancement in the second stage. To make the local and global tone mapping algorithm more effective, Ambalathankandy et al. [39] used histogram equalization method implemented on FPGA with efficient resource usage. To resolve the overenhancement, a nonuniform quantization technique is proposed for CT image enhancement [40]. Recently, deep convolution neural network is also used for range compression [41]. e authors in [41] used the output of existing tone mapping operators as training set and therefore inherited the best properties of all tone mapping operators. Hence, they performed robustly well for all testing images compared to current methods of tone mapping.
Li et al. [42] introduced a clustering-based content and color adaptive tone mapping algorithm that preserved the local structure and naturalness of image. ey used K-means algorithm for the clustering of color structure. However, Kmeans is difficult to adopt for a specific problem due to prespecified number of clusters. In addition, the utilization of K-means for tone mapping has led to the loss of complex local detail in LDR images. Furthermore, the number of available clustering algorithms encourages the use of the most effective algorithm for clustering-based tone mapping algorithm. erefore, to preserve the complex local details, we have adopted a better clustering technique.
In this paper, we have proposed a clustering-based tone mapping algorithm to overcome the problems of false coloring, halo artifacts, and loss of complex local structure. For that purpose, we have utilized the fast search and find of density peak for content and color adaptive tone mapping.
is clustering technique automatically recognizes clusters regardless of dimensionality of data. e following study compares various available state-of-the-art clustering algorithms for tone mapping operation. e effectiveness of proposed technique is compared with different clustering algorithms through subjective and objective evaluation techniques. e experimental analysis suggests that the fast search and find of density peak provides visually appealing result without compromising the local structure of an image. e rest of the paper is composed of the following sections. e second section provides a brief introduction to the clustering-based content and color adaptive tone mapping algorithm. e third section describes different applied clustering techniques. Furthermore, the fourth section discusses the experimentation conducted to evaluate the performance of different clustering techniques. Our overall work is concluded in the fifth section.

Clustering-Based Content and Color Adaptive Tone Mapping Algorithm
Li et al. [42] proposed clustering-based content and color adaptive tone mapping algorithm. Most of conventional tone mapping operators split an image into chrominance and luminance channels. Instead, Li et al. divided an image into overlapped color patches. e overall algorithm consists of training and testing phases. e algorithm [42] utilizes training phase to learn PCA transform matrix for each color structure cluster from HDR training images, while the testing phase is used to project the HDR test images and to find the closest match in the training set. In the first step, logarithmic transform is applied on each HDR image to enhance the contrast and brightness of low luminance values in a channel while compressing the higher ones. Afterward, each image is divided into overlapped color patches to avoid artifacts such as local graying out, hue shift, or color fringes [43]. Each patch is further divided into approximately uncorrelated components: color structure, color variation, and patch mean. Patches with varied intensity level may have similar structures [42]. erefore, patches are grouped together into different clusters based on color structure and then PCA transform matrix is obtained for each cluster. In the testing phase, for each patch color structure, similarity measure is calculated with cluster centers extracted from the training data and then relevant PCA matrix is retrieved. For tone mapping, S curve arctan function is applied on PCA projection matrix. e patch mean is compressed by a linear function and the color variation is controlled by an arctan function. e image is reconstructed by processed patches. At the end, postprocessing step is performed for contrast enhancement by clamping image at its maximum and minimum intensity value.

Compared Clustering Techniques
To explore the influence of different clustering algorithms, K-means is a reference clustering algorithm and the results are compared with Gaussian Mixture Model (GMM) [44], DBSCAN [45], and fast search and find of density peak (FSFDP) [46] clustering algorithms. Before presenting the results obtained by using each of these clustering schemes, a brief description of each clustering technique is presented with its strengths and shortcomings.

K-Means. K-means
is an extensively used partitionbased clustering technique proposed by MacQueen. Partitioning can be done based on minimizing the objective function known as square error [47] which can be calculated as follows: where ‖x i − v i ‖, "c," and "c i " represented Euclidean distance, number of cluster centers, and the number of points in i th cluster, respectively. K-means was considered by Li et al. [42] for the clustering of the color structure. K-means is a relatively simple and computationally efficient algorithm. However, it suffers from the restriction of requiring human interaction for selecting the number of clusters, which provides easy implementation and effective results. Results are influenced by the initialization and may affect the performance of the tone mapping algorithm. Furthermore, K-means is sensitive to the cluster with a single data point due to its square distance [48].

Fast Search and Find of Density Peak.
Fast search and find of density peak (FSFDP) [46] assumes that the density of data at the center of a cluster is relatively high and the center of one cluster is far away from the center of another cluster. Based on this assumption, FSFDP calculates the number of clusters and their centers automatically. Moreover, it detects and removes outliers intuitively. Clusters of nonspherical shape are easy to be recognized using FSFDP. In FSFDP, the cutoff distance used for calculating the density of each data point has a great influence on the effectiveness of FSFDP. For a data point i, FSFDP calculates local density, ρ i , and distance, δ i , from its nearest center as follows: where d ij , denotes the distance from data point i to j and d c is cutoff distance used to calculate the density of each data point. For assigning a data point i to the nearest cluster center, distance is calculated as Equations (2) and (3) represent that the cluster centers are points with greater distance "δ" from other cluster centers and high data density "ρ." e fast search and find of density peak is further experimented on two scale implementations [42] and represented as two-scale fast search and find of density peak (TFSFDP).

Gaussian Mixture Model.
Unlike K-means clustering, Gaussian Mixture Model (GMM) [44] allows clusters of different shapes. e cluster orientation is dependent on the Gaussian distribution. GMM sets the k number of Gaussians according to the data distribution. GMM learns the cluster belonging through the probability of each data point using its parameters such as variance Σ k , mean μ k , and weight of the cluster π k . Mathematically, it is computed by using GMM allows cluster overlapping; however, it is computationally extensive. erefore, GMM is not applicable for high-dimensional data. Moreover, GMM is dependent on Gaussian distribution (clusters).

Density-Based Spatial Clustering of Applications with
Noise. Similar to FSFDP, Density-Based Spatial Clustering of Application with Noise (DBSCAN) [45] is a density-based clustering algorithm that does not require the number of clusters as an initial parameter. DBSCAN identifies the clusters of various shapes and sizes based on data connectivity. Furthermore, DBSCAN is widely adopted because of its efficient computation. However, the algorithm of DBSCAN depends on the initial radius "r" of the cluster as follows: If the value of the radius is set to be small, all points that belong to the sparse cluster are considered as noise. On the other hand, if the radius is set to be large, all points lie within a single cluster. erefore, to obtain appropriate results, the algorithm is executed for a number of times with different values of "r."

Experiments and Results
is section describes the details of the dataset and the evaluation criteria used in our experiments. As different clustering methods have different parameters, the details of these are discussed in parameter setting section. e focus is to elaborate the influence of different state-of-the-art clustering algorithms on clustering-based content and color adaptive tone mapping algorithm, which is examined in performance and effectiveness section.

Dataset and Evaluation Criteria.
For the training phase, Kodak's database [49] is used, which consists of a total of 24 images of size of 512 × 768 × 3. All images are of true color (24 bits per pixel) and, for tone mapping, these images are used as standard set suit [49]. In the testing phase, we performed a series of experiments on Fun and Shi [50] HDR dataset to evaluate the influence of different clustering algorithms on clustering-based content and color adaptive tone mapping [42]. is dataset contains both indoor and outdoor HDR images of 105 scenes captured through Nikon D700 digital camera. Sample images from the dataset are shown in Figure 1.
We compared the results of different clustering algorithms both qualitatively and quantitatively. Tone mapping operators attempt to preserve desirable characteristics, including local structure and naturalness, and to avoid the halo artifacts while converting an HDR image into LDR images. e classical evaluation parameters for objective measurements like PSNR cannot be used for the evaluation of tone mapping due to the unavailability of reference LDR images. So, we used two metrics for quantitative evaluation of dynamic range reduction with different clustering techniques: tone-mapped image quality index (TMQI) [51] and feature similarity index for tone-mapped images (FSITM) [52]. TMQI [51] is used to compute the structural resemblance and indexes of naturalness, and FSITM [52] is to measure the local phase similarity between HDR and LDR images.

Parameters Setting.
Appearance regeneration of an image is dependent not only on the intensity relationship but also on many local weighted attributes including brightness, contrast, gray level, and color relationship. Clustering-based content and color adaptive tone mapping [42] is a patch base model in which the size and shape of the patch depend upon the window used for the calculation of mean and color structure. In this study, we have used the default window with size of 7 × 7 as defined by Li et al. [42]. We have used default values of the control parameter for color appearance, local structure, and luminance within the tone mapping process as used by [42].
For clustering algorithms, the control parameters are set with the most optimal values. e details of these parameters for each clustering method are as follows.

For K-means.
For K-means clustering, the number of clusters is initially set to be 100. 4 Mobile Information Systems

For Fast Search and Find of Density Peak.
For fast search and find of density peak, cut-off distance "d c " given in equation (2) is significant to be set. However, varying its default value that is "1.4141" has no difference in objective evaluation as shown in Table 1; therefore we set it as 1.4141.

For Density-Based Spatial
Clustering of Applications with Noise. Furthermore, for DBSCAN, we execute the algorithm several times to get to the appropriate value of "r" as mentioned in equation (5). On the default value of "r," which is 10 for DBSCAN, the algorithm considers sparse datapoints as noise and therefore selected value of "r" to be 0.2.

For Gaussian Mixture Model.
For GMM, we used the default value of Gaussian distribution "k" in equation (4) as (2).    Tables 2 and 3 present the results of different clustering techniques applied with the method of [42] in terms of TMQI and FSITM, respectively. e best results are highlighted in both tables. Green, blue, and yellow numbers represent the first, second, and third best scores, respectively; likewise red number represents the worst score. e effectiveness of tone mapping operators depends upon the contrast, brightness, and local structure of HDR image scenes. So, a tone mapping operator cannot be equally effective for all HDR images [53]. Table 2 depicts that TFSFDP performs best for 9 images; however, considering   the average of each method, K-means and FSFDP can be used alternatively. Table 3 has strengthened the results further, as FSITM shows that TFSFDP performs best, while K-means and FSFDP stood in the second and third positions, respectively. Furthermore, DBSCAN performs worst in terms of both FSITM and TMQI for every image. Based on the obtained results, DBSCAN is not recommended for tone mapping.

Objective Evaluation.
Performance of clustering-based tone mapping techniques is also compared in terms of their execution time. e execution times of different clustering-based tone mapping techniques are presented in Figure 3, which highlights that GMM has high computation cost irrespective of the provided TMQI and FSITM results. erefore, it is not a preferred clustering algorithm to be used for tone mapping operation. In terms of fastest execution time, K-means outperforms the other Mobile Information Systems methods with least execution time of about "6.598 seconds," whereas DBSCAN and FSFDP can also be considered for tone mapping operation as their execution times are about "11.31 seconds" and "12.218 seconds," respectively.

Subjective Evaluation Assessment.
Subjective evaluation is performed through visual assessment and scoring. Subjective evaluation of 12 images is performed by 8 volunteers currently working on image processing and machine learning. We used a monitor with a spatial resolution of 2560 × 1600 to display images, side by side using the image viewer customizable window. We showed the resultant images of all methods to volunteers and asked them to score image from 1 to 10. "1" represents the worst score, while "10" is the highest score. Ratings are recorded by using score sheets provided to each volunteer during the subjective assessment. Figure 6 shows that, for most of the images, FSFDP has the highest averaged score.

Conclusion
Tone mapping is a complex field in which the conversion of HDR image to LDR image without information loss is needed. e intention of this article is to present the best clustering technique among existing techniques for preserving complex local details lost in case of clusteringbased content and color adaptive tone mapping method. For this purpose, the influence of different clustering techniques is examined. e effectiveness is measured in terms of subjective evaluation and FSITM and TMQI values. Experiments show that fast search and find of density peak results in more appealing results for tone mapping with acceptable computational cost. In the future, this model can be extended by using a feature other than color structure for clustering of patches. Besides, any other function in place of S shape arctan curve can be used for range compression.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.