Retrieval Architecture with Classified Query for Content Based Image Recognition

,


Introduction
Image data has strong impact on consumer attention that has kindled the buying intention of consumers [1,2].Advent of globalization has rapidly influenced the customer preferences and demands [3].Consumer satisfaction has led to strong and positive behavioral outcomes connected to sustainable purchase [4].The revenue of digital business process has been adversely affected by predominant dissatisfaction of consumers due to large amount of irrelevant results generation from text based queries [5].The customers were deprived of taking pleasure in a perishable searching environment.This has led to the popularity of image data which has principally reinstated the text based keyword searching.The progression of multimedia technology has encouraged the use of multimedia in business practice and changed the way we use computers [6,7].Image data has revealed increasing importance in the field of contemporary business environment.The authors have proposed a novel methodology for retrieving image data with query classification which has stimulated the performance compared to state-of-the-art techniques for information identification.Statistical measures have been adopted to validate user responses for the fruitfulness of the proposed technique and to establish the significance of the findings.

Related Work
Accessibility of diverse online and offline information for products and services has radically altered the customer preferences [8].Traditional means to locate the product of interest by the customers were based on text queries.However, the method has huge amount of irrelevant results as output.One of the driving factors for inappropriate output has been due to reprehensible selection of keywords as query.Recent approaches of searching have emphasized on the content of the searched object rather than its name as a keyword [9][10][11][12].The content based searching process has been facilitated by the product image which can provide the necessary knowledge for the required product based on its image contents and has been anticipated to filter out the unwanted results with higher probability.Various methods have been carried out for feature extraction that has applied image binarization as a tool to signify the object of interest and its background, respectively [13][14][15].Threshold selection has been vital to enable binarization of image to differentiate the object and its background.It has been stated that uneven illumination and inconsistent gray levels within the image and its backgrounds have adversely affected threshold selection for binarization [16][17][18].Threshold selection can be divided into three different categories, namely, mean threshold selection, local threshold selection, and global threshold selection.Feature extraction with mean threshold for binarization has been discussed in [19,20] and with bit plane slicing in [21].The problem of uneven illumination in images was efficiently addressed by local threshold techniques [9,[22][23][24][25][26].The literatures have used measures of dispersion like standard deviation and variance to calculate the threshold.El Alami, 2011 [27], has worked with color and texture based features by 3D color histogram and Gabor filters.Hiremath and Pujari, 2007 [28], have calculated local descriptors of color and texture from the color moments and moments on Gabor filter responses by dividing the image into nonoverlapping blocks.Imagery significant point features were chosen by Banerjee et al., 2009 [29], for retrieval process of images.Jalab, 2011 [30], has fused color layout descriptor and Gabor texture for better detection of images.Shen and Wu, 2013 [31], have extracted signatures from images by dealing with color, texture, and spatial structure descriptors.Irtaza et al., 2014 [32], have explored wavelet packets and eigen values of Gabor filters to generate feature vectors from images.Rahimi and Moghaddam, 2015 [33], have referred to the intraclass and interclass features for effective extraction of image features of interest.The authors have proposed a novel technique for content based product recognition in an Internet based business model and has compared the same with the existing techniques of object recognition discussed in the literature.The performance of the proposed technique has outclassed the existing methods and has shown statistically significant improvement to foster generation of relevant results for a given content based product query.

Our Approach
The proposed methodology has considered different image categories which have signified diversified nature of product varieties offered by the firms.Uneven illumination of images can be a factor for degradation of product recognition process.It has been addressed by the authors by selection of local threshold for binarization.At the outset, the binarization was carried out using Niblack's method of local threshold selection [9,18].Each color component of an image was considered for derivation of pixel-wise threshold values by sliding a rectangular window over the component.The local mean (, ) and standard deviation (, ) were calculated primarily with a window size of (25 × 25).The threshold calculation was done as (, ) = (, ) +  ⋅ (, ).The value of  was a constant in between 0 and 1 and was considered to be 0.6.The quality of binarization was dependent on the size of the sliding window and the value of .

Feature Vector Generation
The pixels were locally divided into two different intensity values, namely, the higher intensity values and the lower intensity values by comparing with the corresponding threshold values.The mean and the standard deviation of the higher intensity value pixels and the lower intensity value pixels were considered to derive the higher intensity feature vector and the lower intensity feature vector, respectively.The process of feature extraction has been graphically illustrated in Figure 1.Initially, the original image displaying the model with a cell phone has been considered for feature extraction as shown in Figure 1(a).The image was divided into red (R), green (G), and blue (B) color component at the beginning in Figures 1(b)-1(d).The process was followed by binarization of each of the color components by Niblack threshold selection method shown in Figures 1(e)-1(g).The binarized image of individual color components has different shades of black and white as clearly visible.Two feature vectors of higher and lower intensity values for each color component were computed from the binarized image and were stored for image recognition as shown in Figure 1(h).
The algorithm has been given in Algorithm 1.

Begin
(1) Input an image I with three different color components R, G, and B, respectively, of size  *  each.
(2) Calculate the local threshold value  for each pixel in each of the color components R, G, and B using Niblack's method: where  = 0.6.
(3) Compute binary image maps for each pixel for the given image: / *  = R, G and B * /.  (4) Generate image features for the given image for each color component: / *  = R, G and B * /.

Complexity Analysis
The proposed technique has parted the images into three color components, namely, red, green, and blue.If the total number of gray levels for each component was assumed to be  then linear time () was consumed for selection of threshold for all the gray values in each component.Hence, the number of iterations for three color components was (3).Consequently, it was inferred that the time complexity for the feature selection process was linear.Conventional feature extraction techniques have same feature dimension to that of the image from which the feature was extracted.Hence, for an image of size  *  the feature size was  2 .The proposed method has radically reduced the feature size to 12 irrespective of the image dimension.Thus, the space complexity was efficiently addressed with small feature size.

Retrieval Architecture
The process of retrieval was carried out by means of classified query as in Figure 2. Conventional retrieval process comprised searching the entire dataset with a generic user query.On the contrary, retrieval with classified query initially classifies the query image into the nearest category of images.The classification process was followed by retrieval of images only from the class of interest.The rest of the image categories were pruned down for a classified query as they do not belong to the native class of the query.Thus, the process has evidently improved the recognition performance compared to stateof-the-art techniques.However, a wrong query classification resulting from bleak feature extraction process would result in zero image retrieval of relevant class as all images will be retrieved from the misclassified category.In case of generic query the scenario is different and in most of the cases a minimum number of image retrieval can be expected from the class of interest.Higher degree of misclassification of retrieval query would have adverse effect on retrieval performance which may be considered as a disadvantage of the proposed technique with respect to existing methods.Nevertheless, it can be avoided by designing effective techniques for robust feature extraction.

Experimental Verification
Two different datasets, namely, Wang dataset (10 categories, 1000 images) and Oliva and Torralba (OT-Scene) dataset (8 categories, 2688 images) were considered for the evaluation purpose.Figures 3 and 4 have illustrated the sample of the datasets used for the experimentation process.The validation process was carried out with 10-fold cross validation in which 9 subsets were considered as training set and 1 subset was considered as the testing set. 10 trials were conducted for the performance assessment of the classifiers.The final decision was made by combining the 10 results thus obtained after evaluating 10-fold cross validation.The evaluation process was performed using three different classifiers, namely,  Nearest Neighbor (NN), Support Vector Machine (SVM), and Artificial Neural Network (ANN) [36].NN was considered as an instance based classifier which has acted based  on similarity functions of two different instances.SVM has followed the learning process of Self-Organizing Map (SOM) which has presumed that only nearby nodes have affected the behavior of each other.Finally, ANN was based on a feed-forward architecture known as multilayer perceptron (MLP).The retrieval process was conducted using city block distance as a measure to match the query image with the database image.The equation for city block distance has been given in where  is distance,   is query image, and   is database image.Two different metrics were considered for evaluation purpose, namely, precision and recall.Precision was defined as the probability that an object is classified correctly as per the actual value and recall was considered as the probability of a classifier to produce true positive results.Table 1 has given the precision and recall rate for two different datasets, namely, Wang dataset and OT-Scene dataset under three different classifier environments.It was observed that in case of Wang dataset the highest precision was of 0.838 and the highest recall was of 0.837 using ANN classifier.The highest precision rate for OT-Scene dataset was noted to be of 0.753 with a recall rate of 0.754.
The precision and recall rate for classification with proposed technique of feature extraction were further compared with state-of-the-art techniques as shown in Figure 5.
It was clearly observed that classification with proposed technique of feature extraction has outperformed the existing techniques.
Hypothesis 1. Classification with the proposed method of feature extraction has outclassed the existing techniques.Table 2 has shown the significance of  values obtained from precision comparison and hence the null hypothesis of equal precision rate for the proposed algorithm and existing algorithms was rejected.Hence it was inferred that the proposed method has been capable of improving the classification performance radically.Further, the proposed technique of feature extraction was tested for retrieval performance.The performance was assessed by comparing retrieval with classified query and retrieval with generic query.Table 3 has shown the category wise comparison of precision for retrieval for both the retrieval techniques.The comparison in Table 3 has revealed that retrieval with classified query has improved the identification for each category of images in the Wang dataset.Hence the overall performance of retrieval has increased considerably.The average precision for retrieval with classified query has also outperformed the conventional retrieval technique with generic query.Hypothesis 2. Retrieval with classified query cannot give higher precision results compared to retrieval with generic query.4 has shown significance and hence Hypothesis 2 was rejected.Henceforth, the precision results of proposed technique of feature extraction for retrieval with classified query were compared to state-of-the-art techniques of retrieval as in Table 5.

𝑝 value in Table
Hypothesis 3. Retrieval with classified query with proposed feature extraction technique has higher precision compared to the existing techniques for product recognition for diversified product categories.It was evidently revealed in Table 5 that the proposed method has higher performance compared to the existing techniques and has outclassed the category wise precision Analysis in Table 7 has shown significant association in between consumer satisfaction and image data based query (likelihood ratio = 79.270;Phi = 0.734; Cramer's  = 0.424; and  < 0.01) related to product recognition in diversified product categories compared to text data based query (likelihood ratio = 18.577;Phi = 0.296; Cramer's  = 0.210; and  < 0.05).It was observed that image data based query was having greater association with consumer satisfaction related to product recognition with respect to text data based query.Henceforth, the relation between consumer satisfaction and product query based on image data has been analyzed in Table 8.
From the above results shown in Table 8, significant correlation was observed between consumer satisfaction and query based on image data related to product recognition in diversified product categories.On the other hand it was detected that significant negative correlation lies in between consumer satisfaction and query based on text data related to product recognition in diversified product categories.

Conclusion
The model of digital marketplace has enriched the firms with competitive strategy to gain technical payback from their competitors.The authors have investigated the application orientation of innovation information identification by image data.The study has addressed the challenge of relevant result generation for customer query and has efficiently suggested a content based search method for relevant output generation.The novel technique of product identification based on query with image data has envisioned a new direction for innovation and value addition for using information technology in organizational development.The technique has outclassed all the state-of-the-art techniques and has increased customer satisfaction in product recognition from

Figure 2 :
Figure 2: Block diagram of feature extraction and recognition process.
Thepade et al., 2013) Feature extraction from even + odd image using

Figure 5 :
Figure 5: Graphical illustration for precision and recall comparison.

Table 1 :
Precision and recall for two datasets under different classifier environment.

Table 2 :
Paired t-test for significance testing of precision results for classification.

Table 3 :
Category wise comparison for retrieval with classified query and generic query.

Table 4 :
Paired t-test for significance of retrieval results with classified query.

Table 5 :
Category wise comparison of precision results of the proposed technique with the state-of-the art techniques.

Table 6 :
Paired t-test for significance of precision results for retrieval.

Table 6 .
The test was carried out to validate that the difference in precision results was not generated from a population with zero mean.Table6has shown significant  values for the comparison of precision for retrieval with proposed feature extraction technique with respect to the existing techniques.Hence Hypothesis 3 was accepted and the supremacy of the proposed method for content based image retrieval was established.

Table 7 :
Chi-square test for consumer satisfaction analysis related to product recognition in diversified product categories in digital environment.Chi-square tests Raw variable → consumer satisfaction related to product recognition in diversified product categories

Table 8 :
Nonparametric correlation test for consumer satisfaction analysis related to product recognition in diversified product categories in online environment.It has efficiently pruned the inappropriate outcomes for customer queries by replacing text based search process with image query formulation.The work can be extended towards unstructured data analysis and e-commerce and for other analytical activities related to consumer orientated revenue generation for modern business process.