Research on Printing Defects Inspection of Solder Paste Images

Solder paste printing is the first part of the surfacemount process flow; its postprinting defect inspection is particularly important. In this paper, we focus on studying the printing defects inspection algorithm for solder paste on PCB (Printed Circuit Board) images.+e work proposes a number of methods to enhance the defects inspection performance of solder paste printing: a regional multidirectional data fusion image interpolation method, which can achieve fast and high precision image interpolation; a method for detecting solder paste areas with better accuracy, efficiency, and robustness; an improved connected domain labeling method to reduce time complexity; and defects detection and types classification method, which extracts features and centroid of every solder paste region and completes the inspection by comparing with a standard image. +e experiments show that the defects inspection algorithm can detect the most common types of defects with low time consumption, high inspection accuracy, and classification accuracy.


Introduction
(a) is an image of a PCB printed with solder paste and taken by a camera. Figure 1(b) is the corresponding standard image obtained by parsing the standard Gerber file of the PCB. As the image taken by the camera and the standard image do not match exactly in size and position, the registration operations for them need to be performed. One of the important steps in registration is image interpolation. After the registration is completed, the camera-taken image needs to be binarized to obtain a binary image with the solder paste area in white and the rest in black. In order to compare each solder paste region, a pass-through tagging operation is performed to assign a unique tag to each solder paste region in the binary image. Once the connected field is marked, the corresponding comparison inspection can be performed.
Existing image registration methods are divided into two main categories: feature-based registration and grayscalebased registration [1]. Firstly, we extract the features of the image, find out the transformation model and parameters, and finally transform the image as a whole [2][3][4]. e latter is based on the similarity of pixel grayscale to complete the matching, without extracting other feature information, but when the image is rotated and scaled, it has a huge impact on the algorithm [5]. Image interpolation is an important step in the image registration process. Existing image interpolation methods are broadly classified as adaptive and nonadaptive. Nonadaptive algorithms apply a fixed pattern to each pixel, mainly nearest neighbor, bilinear, and bicubic image interpolation; adaptive techniques require consideration of image edge information, texture, and pixel intensity [6,7].
Image detection methods have their specific targets, and the corresponding detection methods are designed according to the application and image characteristics. For grayscale images, existing image detection methods are mainly based on two aspects: grayscale value discontinuity or grayscale value similarity. e former mainly relies on image edge detection and edge connectivity for implementation, such as improved Canny algorithm [8,9], improved Prewitt operator [10], and Soble algorithm [11]. e latter, on the other hand, categorizes the image pixels according to a certain predefined rule, and typical algorithms include region generation and erosion algorithms and binarization algorithms.
Existing defect classification and inspection algorithms can be broadly classified into two categories: reference comparison method and reference-free verification method. e former is to compare the board to be tested with the standard board for pixel comparison and determine the defects by the difference between the two images. is type of method can inspect more types of defects but requires a high standard image and the registration accuracy of the two images. e latter does not require a standard image but generally requires the extraction of a large amount of feature information and has a high algorithm complexity. Depending on the type of inspected defects, researchers have proposed different inspection schemes [12,13].
In this paper, the four aspects of image interpolation, solder paste area detection, connected domain labeling, and printing defect inspection are improved. e solution of this problem can enrich the connotation of machine vision in solder paste image and printing defect inspection and solve the urgent problem of SMT (Surface Mount Technology) production.

Multidirection Data Fusion
Image Interpolation Based on Regional Features. In order to meet the strict requirements of high-level industrial inspection in terms of accuracy and operation time, an image interpolation method with adaptive fusion calculation characteristics is proposed by referring to the idea of data fusion techniques, which is applied to edge regions in the images.
Let the corresponding point of the point to be interpolated in the source image be point P with coordinates (u, v), i and j are the largest integers not larger than u and v, respectively, and the grayscale values of the four pixel points in the 2 × 2 neighborhood of this corresponding point are f(i, j), f(i + 1, j), f(i, j + 1), and f(i + 1, j + 1), respectively. e mean grayscale value E is According to the image characteristics, set the threshold T. If Var < T, it is judged as a region of low grayscale variation, and the bilinear interpolation algorithm is used to interpolate the value. Otherwise, it is judged as an edge region with large grayscale variations, and interpolation is performed according to the following idea.
First, the estimated values of point P in the four interpolation directions of 45°, 135°, horizontal, and vertical are calculated and denoted as y 1 , y 2 , y 3 , and y 4 , respectively. e equations are as follows, where S x,y is the distance of point P from the source image Euclidean distance in pixels from the pixel with coordinates (x, y) in the image, where x � i, i − 1, i + 1, i + 2; y � j, j − 1, j + 1, j + 2.
Next, the distances from the point P to the four interpolation directions, denoted as r 1 , r 2 , r 3 , and r 4 , are calculated as follows: e normalized formula for r 1 , r 2 , r 3 , and r 4 is as follows: en, the mean values of the grayscale gradient in the four interpolation directions are calculated, which are noted as g 1 , g 2 , g 3 , and g 4 , respectively, with the following equations: Wireless Communications and Mobile Computing 3 Normalized to g 1 , g 2 , g 3 , g 4 , the formula is as follows: e method to determine the fusion coefficient is as follows: where λ is a constant, 0 < λ < 1, used to adjust the interpolation distance and the weight value of the grayscale gradient in the fusion coefficient. e formula for normalizing Ψ k is as follows: After data fusion, the grayscale value of the point P to be interpolated is as follows: At this point, the interpolation of point P is completed.

Solder Paste Area Detection.
e PCB images used for inspection are taken with a monochrome industrial camera under red light irradiation and blue light irradiation, referred to as red monochrome images and blue monochrome images, respectively. e images taken under both color light irradiation are of the same size and correspond to the same area on the PCB. ey are used to enhance the accuracy, efficiency, and robustness of the detection.
Let the original image f be a grayscale image with N rows and M columns, and the grayscale value of the pixel at the coordinates (x, y) is denoted as f(x, y). e average grayscale value T ave of the whole image is calculated by the following formula: Calculate the average grayscale value T ave1 between T ave and 255.
where num is the total number of pixels in the image with grayscale values between T ave and 255 and S is the set of pixels with grayscale values between T ave and 255. A Gaussian model is created for the red monochrome image and blue monochrome image with pixel grayscale values in the interval T ave1 to 255, respectively, its grayscale mean value μ and standard deviation σ are found, and the binarization model is set as follows: Binarize the red monochrome images and the blue monochrome images according to the model shown in (14), respectively, and then according to formula (15) for the binary AND operation to obtain the binarized result image.
where g r (x, y) and g b (x, y) are the grayscale values of the pixel at (x, y) in the binarized red and blue monochrome images, respectively. g o (x, y) is the grayscale value of the pixel at (x, y) in the resulting image. When grayscale values of pixels are 0, they are background pixels and will not be processed. If grayscale values of pixels are 255, the pixels are target pixels. e solder paste areas are obtained.

Connected Domain
Labeling. e labeling of solder paste regions is a prerequisite for printing defect inspection of solder pastes. An improved method is proposed here to reduce the high time complexity in existing connected domain labeling algorithms.
For a target pixel, the grayscale values of pixels in a specific neighborhood are detected according to the location of the target pixel, and the label is set according to specific rules. All the different location types of pixels in the binary image are shown in Figure 2.
Set a two-dimensional array label[y][x], which is used to store the label of the pixel at the location of (x, y) coordinates in the image, and set a one-dimensional array belong[label[y][x]], which is used to store the label of the connected domain to which the pixel at the location of (x, y) coordinates belongs, and set the coordinates of the current pixel as (x, y), in the following three cases: (

Defects Detection and Types Classification.
e types of defects studied here are Excessive Solder, Insufficient Solder, Solder Offset, Solder Bridge, Missing Printing, and Spattering Solder, as shown in Figure 3. e paper studies the characteristics of different defects and compares the connected domains between the labeled result image and standard image to propose the inspection rules to find the defects and identify their types.
Calculate the area and centroid of the solder paste area A with the following equation: where (x, y) are the coordinates of the pixels in the solder area, S is the area of the solder area, N is the total number of pixels in the solder area, and (x 0 , y 0 ) is the centroid of the solder area. e connected domain with an area less than a certain threshold T 0 is treated as noise and is not used for defect inspection.
Step 1. By means of the centroid coordinates, find the label of the corresponding connected domain in the standard image of all the solder-paste connected domains in the resulting image. If the label is 0, it corresponds to the background area and is not processed. If the label is not 0, the following two cases can be distinguished: (1) If there are duplicate labels, it means that Solder Bridge has occurred. (2) For a nonrepeat label, it indicates that Solder Bridge has not occurred and continues with other defect type inspections. T 1 and T 2 are preset thresholds, T 1 < 1, T 2 > 1. Take one out of all the nonrepeated labels, suppose its connected component area is S f , and the corresponding connected component area in the standard image is S b .
① If d ≤ T 3 , the connected domains are judged to be normal. ② If d > T 3 , the connected domains are judged as Solder Offset.
Step 2. rough centroid coordinates, find out the connected domain labels in the standard image corresponding to all the unprocessed solder paste connected domains in the resulting image, and record all the labels: (1) If the label is 0, the connected area of solder paste is judged as Spattering Solder. (2) If there is a nonzero repeated label in the label record, it means that multiple solder paste connected domains in the resulting image correspond to one solder paste connected domain in the standard image, and data merging is required. After data merging, the operation is similar to Step 1. (3) For nonzero and nonrepetitive labels, there is no need to merge data. e operation is exactly the same as the processing of nonrepetitive labels in Step 1.
Step 3. After the processing of the above steps, if there are unprocessed connected domains, it indicates that a Missing Printing has occurred.

Image Interpolation Experiment.
e original image is shown in Figure 4(a), which is sampled three times to reduce its length and width to 50%, 70%, and 30% of the original one, respectively. en, the bilinear, bicubic, and the proposed methods in this paper are interpolated, respectively, and due to space limitation, only 30% of the resulting images are given as shown in Figures 4(b), 4(c), and 4(d).
Calculate the peak signal-to-noise ratio with the following equation: PSNR � 10 log 10 where M and N are the horizontal and vertical sampling points of the image, respectively, f (m, n) is the original image, and f (m, n) is the processed image. e larger the PSNR value, the higher the accuracy of the interpolation algorithm. Due to space limitations, only the experimental results are given when the image aspect is reduced to 30%, as shown in Table 1. From the table, we can see that the algorithm in this paper has the highest PSNR, indicating that the algorithm has high interpolation accuracy.

Solder Paste Area Detection Experiment.
ree sets of red and blue monochrome images were taken under normal, strong, and weak illumination, respectively, and the binarization method proposed in this paper was used to detect the solder paste area. Here, the detection and result images under weak illumination are given as an example which are shown in Figure 5, where Figures 5(a) and 5(b) are the red and blue monochrome images to be detected, respectively, and Figure 5(c) is the detection result image.
Ten types of solder paste areas were used for detection, as shown in Table 2; due to space limitations, only the results of solder paste detection under weak illumination are given. e error rate for each type of solder paste area detection is calculated according to the following formula: where N denotes the number of solder areas of the type, n i denotes the number of pixels detected in the ith solder area of the type, and n io denotes the true number of pixels in the ith solder area of the type. e smaller the ER, the higher the detection accuracy. From the table, it can be seen that the adaptive threshold binarization method proposed in this paper has a low ER and can achieve high accuracy detection of solder paste areas.

Connected Domain Labeling Experiment.
Six images were labeled with connected domains using the algorithm of this paper, the region growth method [14], and the tourbased coding method [15], respectively. ey were executed 10 times consecutively, and the average time was recorded as shown in Table 3. As can be seen from the table, compared with the other two algorithms, the algorithm of this paper has the advantage of a shorter running time.
One of the six images is shown in Figure 6(a). e image labeled using the improved algorithm in this paper is shown in Figure 6(b). Each connected domain in the resulting image is assigned a different grayscale value to show the distinction in labeling. the Gerber file.

Defects Detection and Types Classification Experiment.
e inspection results are shown in Figure 7(c), and the meanings of each letter are as follows: D for Excessive Solder, S for Insufficient Solder, P for Solder Offset, Q for Solder Bridge, L for Missing Printing, and J for Spattering Solder.
Using the results of manual visual inspection as the standard, the inspection accuracy, classification accuracy, and false alarm rate of the defect inspection method studied in this paper were calculated, and the results are shown in Table 4. As can be seen from the table, the

Conclusion
In this paper, the techniques related to the inspection of solder paste images and printing defects are studied and some substantial results are achieved. e proposed regional multidirectional data fusion image interpolation method not only has a fast interpolation speed but also has a high interpolation accuracy, can protect the edge details of the image well, and is applicable to any level of interpolation transformation, and the interpolation method is applied to PCB image registration, which improves the speed and accuracy of registration. e proposed solder paste region detection scheme can accurately identify solder paste regions and can adapt well to changes in light source brightness with low time complexity. e proposed solder paste printing defect inspection method is able to detect six types of defects with low time consumption and has good performances in inspection and classification. However, the research work in this paper is aimed at external defects inspection and cannot deal with internal cavity defects detection. Moreover, the method is studied for the inspection work in two-dimensional space, which cannot detect those defects when needing height or depth information, for example, scars inspection, pits inspection, and so on. ree-dimensional defect inspection is worth studying, where highly accurate height or depth detection is the key technology.
Data Availability e datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest
e authors declare no conflicts of interest.