Efficient Localization of Multitype Barcodes in High-Resolution Images

Barcode positioning technology is one of the important components of barcode technology. However, most of the current algorithms are only applicable to a single type of barcode positioning or limited to low-resolution images due to a large amount of calculation, and thereby it is still a challenge to locate multitype barcodes in high-resolution images. In response to the above problems, this paper proposes a reliable multitype barcode localization method for multibarcode localization in high-resolution images where one-dimensional (1D) barcodes, two-dimensional (2D) barcodes, or multitype barcodes are present simultaneously. The method consists of three main steps: ﬁrst, extracting multiple types of barcode features through a joint edge detection algorithm; next, marking target barcode regions with a bidirectional contour labeling method; and ﬁnally, extracting barcode regions by an improved aﬃne transformation. The experimental results show that, in terms of localization accuracy, the proposed method has a better accuracy of 97.83% than existing algorithms in low-resolution images and can locate multitype barcodes in high-resolution images with an accuracy rate of 98.04%. Besides, in terms of time cost, the proposed method eﬀectively reduces the time cost by 50% and improves the barcode localization eﬃciency.


Introduction
Barcode positioning technology is widely used in many industries and fields, such as warehousing, library management, health care, industrial production, and express logistics [1,2]. However, its efficiency is insufficient for multiple types of barcode localization in high-resolution images because most of the existing methods are only applicable to a single type of barcode localization and the localization process uses complex algorithms, which lead to high time costs.
Produced by Gallo and Manduchi, a simple and fast algorithm is used to calculate the vertical and horizontal gradients of each pixel and perform global binarization for the segmentation of the 1D barcode region [3], which, however, is unsuitable for locating slanted barcodes. Yun and Kim [4] proposed a new 1D barcode localization method that detects barcode features based on the similar structure of entropy and edge orientation in 1D barcodes, thus realizing the localization of arbitrarily tilted 1D barcodes. Literature [5] exploited the stacked edge features inside 2D barcodes to develop a histogram-based 2D barcode detection method, but it is vulnerable to the interference of dense text. Rincon et al. [6] locate quick response (QR) codes in complex contexts by detecting the structural features of three rectangles in QR codes, effectively addressing the impact of dense text. e above algorithms have high accuracy for single type barcode positioning, but none of them consider the localization of multiple types of barcodes in one image. D. T. Lin and C. L. Lin [7] proposed a multitype barcode localization framework, which utilizes an adaptive thresholding method to extract black and white linear features inside barcodes to achieve multitype barcode localization. However, this method only performs multitype barcode positioning tests in low-resolution images with a resolution of 300 × 400 pixels. Katona and Nyúl [8] performed edge feature detection of barcodes by improving the Sobel and Feldman algorithm [9], which enables the framework to perform edge detection of multiple types of barcodes in barcode images with a resolution of 720 × 480 pixels, compared to the previous method, but its algorithm complexity is high. ese algorithms are limited to feature detection of a single type of barcode, or their algorithms have high complexity and face the disadvantages of high time cost and computational inefficiency in high-resolution images.
In the image processing-based barcode localization, barcode region marking is also of great interest. D. T. Lin and C. L. Lin [7] developed a two-channel connected domain analysis algorithm, which can improve the efficiency of barcode region labeling to replace the traditional method. Another important contribution of this algorithm is the ability to label multiple types of barcode regions. Chang et al. [10] studied a contour-detection-based marking method that has a wide range of applications by using the contourdetection technique to track and label the external and internal contours of each target location. By improving this algorithm, Chen et al. [11] proposed a two-stage method for marking the external contours of barcode regions, namely, orientation-based region contour tracking and contourbased region marking. is algorithm can not only mark multiple types of barcode regions but also do it more efficiently because global marking is not required. Liu et al. [12] successfully applied the above algorithm to multiple 2D barcode regions, and the experimental results also showed that the contour-detection-based barcode region marking method is better than the connected domain analysis method, but its marking time is affected by the resolution size of a barcode image.
Effective extraction of barcode areas is a prerequisite for obtaining barcode information. Lin and Fuh [13] used the position detection patterns of QR codes to extract QR codes that could recognize different types of QR code images. In the literature [14], a QR code extraction algorithm based on the Hough transform [15] is proposed. e algorithm achieves QR code extraction by forming a bitmap with edge detection and Hough space and preserving the intersection of vertical line segments. Chen [16] provided an algorithm to effectively extract the tilted 1D barcode region. It first obtains the 1D barcode edge information in the image by Canny edge detection [17], then detects the straight lines inside the barcode using the Hough transform, obtains the tilt angle of the 1D barcode region, and finally rotates the extracted barcode to a horizontal state. e above barcode region extraction algorithm can effectively extract tilted single-type barcodes, but it does not consider the extraction of multitype barcodes, and the process requires high time cost due to complex preprocessing, especially in high-resolution images. e development of deep learning and machine learning has inspired many new barcode positioning algorithms. Zamberletti et al. [18] introduced a machine learning-based barcode detection method in which the algorithm effectively recognizes 1D barcodes. Katona et al. [19] used a vector machine to learn the characteristics of QR codes and use them to extract QR barcodes. Unfortunately, none of the above algorithms take into account multitype barcodes. Ren et al. [20] provided a fast region-based convolutional neural network (CNN) model that can quickly detect and classify multiple targets. Based on this model, Tian et al. [21] developed a barcode detector called BAN, which can quickly detect multiple types of barcodes in barcode images. Unfortunately, it has only been tested in images with resolution of 1024 × 768 pixels. Jia et al. [22] developed a multitype barcode recognition model based on an improved CNN and trained it using images with resolution of 1920 × 1080 pixels. Additionally, another focus of this model is that the accurate position and distorted barcode shape can be determined and corrected. e performance of these localization algorithms is tied to the quality and quantity of training data and the training time of the model. Based on the above research, the problems identified are high computational cost, restricted to low-resolution images, limited to barcode types, etc. In order to address these problems, a reliable localization method for multitype barcodes in high-resolution images is proposed. e main contributions of these works can be summarized as follows: (1) Effectively reduces time costs and further improves barcode positioning accuracy. (2) e proposed algorithm can perform barcode localization in high-resolution images and has excellent performance. (3) Efficiently extracts arbitrarily tilted barcodes and provides good horizontal barcodes for decoders. (4) e proposed method can locate 1D barcodes, 2D barcodes, and multitype barcodes.

Methods
Since the majority of current barcode localization algorithms are confined to a single kind of barcode and the high computational complexity is limited to low-resolution pictures, there is still room for improvement in their localization performance and efficiency. Aiming at the problems mentioned above, this paper proposes a barcode positioning method for high-resolution images. e method includes three main modules: feature extraction of the barcode edge, labeling of the barcode region, and extraction of the barcode region. e barcode positioning method is shown in Figure 1.

Feature Extraction of the Barcode Edge.
Effective extraction of edge features in the barcode region is a prerequisite for barcode localization. Nevertheless, existing techniques are only appropriate for extracting a single type of barcode feature or are limited to low-resolution images due to the high complexity of the algorithm. erefore, this work proposes a simple and quick algorithm, namely joint edge detection.

Grayscale Conversion.
e input image is a highresolution image with three red, green, and blue (RGB) channels. It not only fails to reflect the features of the barcode region but also leads to data redundancy. To reduce data redundancy and improve calculation efficiency, images are first converted to single-channel grayscale images by the following equation: where r, g, and b represent the pixel value of red, green and blue channels of the image at the (x, y) position, respectively. rough this operation, the high-resolution image is converted into a grayscale image, which effectively keeps the edge feature information of the barcode and reduces data redundancy.

Joint Edge Detection.
e joint edge detection algorithm proposed in this paper mainly detects the edge features of barcodes in all directions through horizontal, vertical, and double diagonal directions. Detection directions are shown in Figure 2. e suggested method is divided into three stages, as follows: In the beginning, the horizontal and vertical gradients I x (n) and I y (n) for each pixel in the greyscale image G are computed in the horizontal and vertical directions. en, they are combined in a nonlinear manner, and their absolute values are calculated. Finally, a horizontalvertical edge gradient map I e (n) is generated by equation (2).
is approach fully detects the barcode with horizontal (vertical) edge features and improves the robustness of the barcode horizontal (vertical) edge feature extraction direction.
en, the barcodes with horizontal (vertical) edge features are extracted in the gradient map I e (n), and all the extracted barcode regions D i are removed by equation (3) in the grayscale image G, forming a new grayscale image G new , where i is the number of extracted barcode regions.
Finally, 45-degree and 135-degree gradients I 45 (n) and I 135 (n) for each pixel n in the new greyscale image G new are computed in double diagonal directions. en, they are combined in a nonlinear manner, and their absolute value is calculated. Finally, a double diagonal edge gradient map I d (n) is generated by equation (4). rough this step, the edge features of inclined barcodes and 2D barcodes can be effectively detected, and the robustness of the extraction direction of the edge features of inclined barcodes and 2D barcodes can also be improved.
e edge feature detection results of the high-resolution image through the joint edge detection algorithm are shown in Figure 3, where it can be seen from Figures 3(b) and 3(c) that the edge features of horizontal (vertical) barcodes, inclined barcodes, and 2D barcodes are effectively detected by this method. e higher the energy, the better the effect of barcode edge feature detection.

Labelling of the Barcode Region.
e labeling of the barcode region is an important process for accurate barcode localization. e traditional connected-component marking approach requires that all pixels in the target region be marked. e efficiency is low, and the cost of marking time is heavily influenced by image resolution. erefore, a new connected-component method has been developed to improve efficiency.

Preprocessing Operations.
In order to reduce the influence of small background clutter and weak target areas in gradient maps I e (n) and I d (n), a block filter of size 35 × 35 over gradient maps I e (n) and I d (n) is used to reduce noise and improve the weak barcode region. e size of the filter was chosen based on the range of the resolution of the input images by our method. It is worth noting that block filtering can be implemented efficiently so that only few operations per pixel are required. In addition, the thresholds are determined by taking the adaptive thresholding approach of Otsu [23]. e binary images B e (n) and B d (n) are obtained by equation (5) after the thresholds T e and T d are obtained, and morphological techniques are utilized to fill in the gaps in the barcode feature region.
In this formula, I e (n) is the horizontal-vertical gradient map, I d (n) is the diagonal gradient map. e preprocessing results of I e (n) and I d (n) are shown in Figures 4(b) and 4(c), respectively. e traditional approach is abandoned in this section, and a bidirectional labeling method based on contour information is proposed. is method tracks and labels the outer contour of each barcode region clockwise and counterclockwise, as realized by parallel computing [24] on the CPU. e bidirectional contour labeling method consists of five steps used iteratively to examine all the pixels in the image. In Step 1, the current pixel P 0 is the starting point of the outer contour of the new barcode region if it is a foreground point and the prior pixel P −1 is a background point, as illustrated in Figure 5(a). After the contour beginning point is found, the beginning and middle points P S and P C can be set as P S � P −1 and P C � P 0 , respectively, for initialized contour tracking.
In Step 2, the next contour point P N is tracked clockwise and marked, with P S as the starting point of clockwise tracking and P C as the center point of eight-neighborhoods, as shown in Figure 5(b).
In Step 3, the next contour point P N is tracked counterclockwise and noted, with P S as the starting point of counterclockwise tracking and P C as the center point of eight-neighborhoods, as presented in Figure 5(c). In Step 4, the starting and center points P S and P C can be set as P S � P C and P C � P N , respectively, for clockwise tracking (counterclockwise tracking) when a new contour point P N is tracked clockwise (counterclockwise) and marked.
In Step 5, the parallel computing technique is adopted to achieve simultaneous contour tracking and marking both clockwise and counterclockwise. e method proceeds to determine the starting point of the next contour if clockwise and counterclockwise tracking to the pixel point P N has been marked, indicating that the contour tracking and labeling of the current barcode region have been finished. Figure 5(d) shows the outcome of the bidirectional contour marking approach in the target region.

Extraction of the Barcode Region.
In order to improve decoding efficiency, the marked barcode areas need to be extracted. Since the existing barcode extraction technology is restricted to a particular type of barcode extraction, a quick and simple extraction algorithm is proposed in this section. e procedure consists of two steps: calculating the rotation angle of the inclined barcode and rotating the inclined barcode region to a horizontal state.

Rotation Angle of Barcode Region.
Based on the rectangular shape of the barcode region, a maximum outer rectangle is fitted to the marked barcode region, and the fitted barcode region is extracted from the binary image B(n). e rotation operation with the extracted barcode region instead of the entire image can effectively reduce the data volume and improve the computation speed. e minimum outer rectangle is fitted to the extracted barcode region, and the tilt angle θ of the barcode region is calculated based on the fitted minimum outer rectangle. e rotation angle for rotating the tilted barcode area to a horizontal state is calculated by the following two steps: In Step 1, based on the width W and the height H of the minimum outer rectangle, the width-to-height ratio R of the minimum outer rectangle is calculated by the following equation: In Step 2, the rotation angle β is determined by the relationship between the ratio R and the threshold values R 1 and R 2 , which are empirical values herein, as shown in the following equation: In this formula, R 1 and R 2 are set to 1.2 and 0.9, respectively, based on experience, and the tilt angle θ is the angle of the first side of the minimum outer rectangle rotating clockwise and touching the X-axis of the coordinate system. e above side is the width W of the smallest outer rectangle, and the adjacent side is the height H, as shown in Figure 6. is method fully guarantees that the long side of the barcode is horizontal and the short side of the barcode is vertical after rotating β with any tilt barcode.

Improved Affine Transformation.
e tilted barcode area is rotated to a horizontal state around the center point of the barcode area by the affine transformation, where the rotation matrix A and translation vector B required for the affine transformation are determined by the rotation angle β and the center of rotation. e rotation relationship is as follows: In this formula, A � cos β sin β −sin β cos β , C x and C y are the horizontal and vertical coordinates of the center point of the barcode area, (x, y) are the horizontal and vertical coordinates of the pixel point to be rotated, and (x ′ , y ′ ) are the horizontal and vertical coordinates of the pixel point after rotation. However, as the size and the center of the barcode area are changed after rotation, the traditional affine transformation for rotation will result in missing or incomplete edges of the barcode area. To address this shortcoming, the algorithm is improved in this paper to recalculate the barcode area size and the new translation vector after the barcode area rotation by the following equations: where W and H are the width and the height of the barcode area before rotation and W′ and H′ are the width and the height of the barcode area after rotation. e results of barcode region extraction are shown in Figure 7, where the red and green boxes in Figure 7(a) are the results of fitting the maximum outer rectangle and the minimum outer rectangle, respectively. e results show that the proposed algorithm can extract arbitrarily skewed multitype barcode regions in high-resolution images and rotate them to a horizontal state, providing good barcode images for decoders.

Experimental Environment and Datasets.
In this paper, the proposed algorithm is simulated using the OpenCV framework and C++, and the barcode images are captured using a high-resolution camera. e barcode images are divided into three data sets as follows:  Table 1.
To verify the performance of the proposed algorithm, this paper compares the accuracy and time cost of barcode localization with those of the traditional localization algorithms.

Accuracy of Positioning.
e accuracy of barcode positioning includes single type barcodes and multiple type barcodes in both low-resolution images and high-resolution images. e details are as follows: First, the localization accuracy of the proposed algorithm for barcodes in low-resolution images is tested in Dataset A. Tables 2-4 show the accuracy of the proposed algorithm  [7,8,11,20,21], respectively. e experimental results show that the proposed algorithm has an average accuracy of 98.07% in 1D barcode localization, 97.95% in 2D barcode localization, and 97.82% in multitype barcode localization. From the data in the tables, it can be seen that the proposed algorithm can effectively locate both single type and multitype barcodes in low-resolution barcode images and that its   [19] 1024 × 768 3-5 550 233   localization performance is better than that of the existing barcode localization algorithms. Second, the localization accuracy of the proposed algorithm for barcodes in high-resolution images is tested in Dataset B. Tables 5-7 show the accuracy of the proposed algorithm for 1D, 2D, and multitype barcode localization. From the data in the table, it can be seen that the proposed algorithm can effectively locate both single type and multiple type barcodes in high-resolution barcode images with high accuracy. Specifically, the average positioning accuracy of 1D, 2D, and multitype barcodes was 99.06%, 99.70%, and 98.04%, respectively.

Time Cost of Positioning.
Effectively reducing the time cost of barcode positioning is another focus of the proposed algorithm.
e time cost of barcode positioning includes multiple types of barcodes in low-resolution images and highresolution images, respectively. e details are as follows: Firstly, the time cost of the proposed algorithm for multiple types of barcode localization in low-resolution images is tested in Dataset C, where the barcode image resolution and the number of barcodes correspond to those provided by Lin, Katona, and Chen. As shown in Table 1, the time costs of the proposed algorithm are 98 ms, 165 ms, 176 ms, and 233 ms, respectively, which are lower than the time costs of the conventional localization algorithms, i.e., 230 ms, 256 ms, 310 ms, and 550 ms, respectively. It can be seen from the data in the table that the proposed algorithm can effectively reduce the time cost of multitype barcode positioning in low-resolution images by 36%-57% compared to the traditional method.
Secondly, the multitype barcode images in Dataset B are converted to five different resolutions, which are 526 × 390 pixels, 1052 × 780 pixels 2104 × 1560 pixels, 3208 × 2120 pixels, and 4208 × 3120 pixels, respectively. Meanwhile, the time cost of the proposed algorithm to locate the multitype barcodes in the high-resolution images is being tested. e test results are shown in Figure 8. It is clear that the time cost of barcode positioning is greatly influenced by the image resolution and the number of barcodes to be positioned, but the increase in the time cost of the proposed algorithm is lower than that of existing methods. On the other hand, the time cost of existing algorithms is higher than that of the proposed algorithm at different resolutions. e experimental results show that the proposed algorithm can effectively reduce the time cost of barcode positioning and improve the efficiency of barcode positioning, especially in high-resolution images.
rough the above experimental analysis, the accuracy of the barcode localization algorithm proposed in this paper in locating 1D barcodes, 2D barcodes, and multitype barcodes in     low-resolution images is higher than that of existing algorithms. Secondly, it can effectively locate single-type barcodes and multitype barcodes in high-resolution images, and some of the localization effects are shown in Figure 9. Finally, the time cost of the proposed algorithm is lower than that of traditional algorithms for multitype barcode localization in both lowresolution images and high-resolution images, which can effectively reduce time cost and improve localization efficiency.

Conclusions
In this study, a new reliable localization method is proposed to extract arbitrary tilting real barcodes from one image, especially when 1D, 2D, or multitype barcodes are present in high-resolution images. e proposed method mainly comprises three steps: first, detection of barcode edge features; second, marking of barcode regions; and finally, extraction of barcode regions. In the proposed method, the edge features of arbitrary barcodes are quickly extracted by a joint edge detection algorithm, and potential barcode regions are efficiently marked by using a bidirectional contour marking method. Finally, the problem of tilted barcode extraction is solved by an improved affine transformation.
e experimental results show that the proposed method can effectively locate multitype barcodes with higher accuracy in both low-resolution and high-resolution images. On the other hand, the proposed method can effectively reduce the time cost of barcode localization and further improve its efficiency, especially for multitype barcodes in high-resolution images.

Data Availability
e datasets used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e author declares that there are no conflicts of interest.