BSV System with MSC and Its Application in the Measurement of Surface Cracks

. In this paper, we have proposed a fexible noncontact crack-size measurement method that can realize binocular stereo vision measurement with only a single camera. On the premise that the camera’s intrinsic parameters have been accurately calibrated, we use a camera to collect the image of the crack from two directions. Ten, we calculate the motion parameters using the collected images from the camera in diferent positions. In addition, Canny algorithm is used to extract the edge pixels of crack images. Finally, we establish the binocular stereo vision model for crack measurement according to the camera parameters, the motion parameters, and the edge information of crack images. Tus, we can measure the crack size through this model. Experimental results show that the measurement error is less than 5% under a distance of 2meters, which can efectively prove the precision of the proposed method. In addition, our method only uses a single camera. Compared with the traditional binocular stereo vision method, this method is not only fexible but also more economical.


Introduction
In this work, we are devoted to seeking a more fexible and economical crack measurement method based on computer vision under the premise of ensuring the measurement precision.Crack measurement plays a fundamental role in monitoring of structure stability and safety.Te cracking process might also be expected during the serviceability of concrete structures, but when the fracture magnitude (width and linear extension) signifcantly increases, this could lead to a deterioration in steel reinforcement rods or it could be a presignal of forthcoming collapse.Until now, crack measurement has remained a common problem in railway engineering [1,2], structural engineering [3], road engineering [4], and water conservancy projects [5,6].
Traditional measurement methods for structural surface cracks include the microscopic method, ultrasonic method, and displacement sensor method.Te microscopic method measures the crack by using a crack microscope [7].When we use a crack microscope, we place the lens parallel to the crack.Tis will lead to inefciency and inconvenient operation.Te crack depth can be obtained by ultrasonic methods [8].Similar to the microscopic method, there are still problems of inconvenient operation and low efciency.Te sensor method places a displacement sensor inside the material in advance so that the realtime data of surface cracking can be obtained [9].Although the method is of high precision, it is often expensive and can only be used for one crack, which is generally suitable for real-time crack monitoring in key structures.In recent years, the crack measurement method based on multisensor fusion has been gradually applied to engineering practice.For instance, two methods for fusion-based damage diagnostics for stifened composite panels have been proposed in literature [10], which demonstrates that the intrinsic capacity of the two techniques can be utilized, leading to synergistic efects for damage diagnostics.Literature [11] proposes a methodology to fuse signals from multiple ultrasonic sensors and detect cracks in the reinforced concrete reference structure.In essence, literature [10,11] still belongs to ultrasonic methods which have the disadvantage of inconvenient operation.
Te abovementioned methods have problems of inconvenient operation (microscopic methods and ultrasonic methods) or high hardware costs (displacement sensor methods).At present, with the development of digital image processing technology, surface crack detection technology based on computer vision has been widely used [12].It refers to using the camera to collect the images of the crack, and then, the geometric dimension information of the crack can be obtained according to the image.Tis is a noncontact measurement method, which is fast and highly automated.It is often mounted on drones or vehicles to achieve rapid and accurate measurement of cracks.
However, in recent years, crack detection methods based on computer vision have mainly focused on how to identify cracks [13,14], and there is little research on how to measure the size of cracks.Literature [15,16] studied how to use digital images for size measurement.However, this method needs to strictly ensure that the camera is parallel to the plane of cracks; otherwise, the measurement error will be uncontrollable.Fortunately, binocular stereo vision (BSV) technology can be used to solve this problem.For the BSV method, it usually requires two cameras.In addition, the relative positions of the two cameras shall be kept strictly unchanged during the measurement process.Tis will adversely afect the fexibility and economy of measurement.
To further improve the fexibility and economy of BSV, we propose a crack measurement method using BSV with a moving single camera (MSC) in this paper.First, we use a camera to take crack images from two diferent directions.Ten, we use the feature-matching method, camera motion estimation method, and image edge extraction method to realize the high precision and automatic measurement of crack width.Tis method uses only one camera, which signifcantly improves the fexibility of measurement.Compared with the traditional BSV, our method only needs one camera, so the fexibility is greatly increased.Besides, if the type of the camera is the same, our method can be half cheaper for imaging devices; compared with other monocular vision methods, our method does not need to keep the camera parallel to the crack plane and can achieve high shooting accuracy at diferent shooting angles.As to the ultrasonic method, acoustic information fusion method, microscopic method, and displacement sensor method, our method is a noncontact measurement method with a higher degree of automation.

Measurement Purpose.
In general, we use the minimum distance method to calculate the crack width.On the premise that the spatial positions of the each point on the two edges of the crack are known, for each point on one edge, we calculate the minimum distance from the point to the other edge by In Equation (1), X i denotes a point on the edge of one side of the crack and Y i denotes a point on the edge of the other side.Te maximum value of all calculated dis i is the maximum width of the crack, which is the purpose of crack measurement.
After we have cleared the purpose of measurement, we need to know the spatial positions of each point on the two edges of the crack.In the following section, we introduce how to use BSV to calculate the 3D coordinates of spatial points.

Basic Principle of BSV with MSC.
Essentially, BSV with MSC is a generalization of BSV.In this section, we take BSV as an example to explain this in detail.
As shown in Figure 1, if O 1 and O 2 are two cameras, we call it the BSV system.If they are just two diferent positions of the same camera, we call it the BSV system with MSC.In this paper, we only discuss the latter case.
Te projection points of the point X on the imaging plane of camera position 1 and position 2 are x 1 and x 2 , respectively, and x 1 and x 2 are the so-called corresponding points.
Te optical centers of the camera at the two positions are O 1 and O 2 , and the projection matrices of the camera at these two positions are M 1 and M 2 , respectively.
We call the three-dimensional space, where point X is located, the world coordinate system, and the two-dimensional plane, where point x 1 and x 2 are located, is called the image coordinate system.In addition, there is a camera coordinate system in the BSV system, and O 1 and O 2 are the origin of the camera coordinate system at the corresponding position, as shown in Figure 1.
We write the coordinates of x 1 , x 2 , and X as homogeneous coordinates; that is, x 1 = (u 1 , v 1 , 1), x 2 = (u 2 , v 2 , 1), and X=(x, y, z, 1).Te relationship between them can be described as In Equations ( 2) and (3), s 1 and s 2 can be eliminated as intermediate variables.Essentially, Equations ( 2) and (3) are the equations of lines O 1 x 1 and O 2 x 2 , respectively, and X is the intersection of the two lines.In this way, we can calculate the spatial three-dimensional coordinates of the point X through the projection points of the spatial points in the two positions of the camera.Tis is the basic principle of BSV, and the specifc derivation process can be referred to literature [17].
Te projection matrix M 1 can be decomposed as and so is M 2 .K denotes the intrinsic parameter 2 Mathematical Problems in Engineering matrix of the camera.R 1 denotes the rotation matrix of camera position 1, and t 1 denotes the corresponding translation vector.Te matrix is defned as the rotation from the camera coordinate system to the world coordinate system, and the translation vector is defned as the translation from the camera coordinate system.We set the camera coordinate system of position 1 to coincide with the world coordinate system, and then, we can get M 1 � K [I, 0], where I is the identity matrix.In this case, we set the rotation and translation from position 2 to position 1 as R and t, respectively; then, we can get M 2 � K [R, t].R and t are called motion parameters; the process of solving motion parameters is called motion estimation [18].

Measurement Method.
In Section 2.2, the basic principle of BSV with MSC has been explained, and in this section, we describe how to measure cracks in this method.
Te schematic diagram of surface crack measurement based on BSV with the MSC system is shown in Figure 2. When we use the camera to capture the same one crack image from position 1 and position 2, respectively, we can obtain the three-dimensional coordinates of a point on the edge of the crack.Ulteriorly, we can complete the measurement of the maximum width according to Equation (1).
It has been previously described how to use BSV with the MSC system to measure cracks in detail.However, there are still two key problems: one is how to solve the intrinsic parameter matrix K and motion parameters [R, t], and the other is how to efectively recognize the crack edge from the image.In the following sections, we discuss these two problems in detail.

Solution of System Parameters.
According to the analysis in Section 2.2, the parameters mainly include the camera's intrinsic parameter matrix K and motion parameters R and t.As for the matrix K, its solution method has been quite mature.In this paper, Zhang's calibration method is adopted to solve it with a plane checkerboard [19].In the following, we focus on how to solve the motion parameters of the camera.
Equation ( 4) describes the so-called "polar equation," where F is the fundamental matrix, and it has the following properties: We set t � [t 1 , t 2 , t 3 ] T , and in Equation ( 5), We set where E is called the essential matrix.We set We can get Te essential matrix is only related to the motion parameters of the camera.Terefore, the motion parameters of the camera can be solved by the essential matrix.
Typical methods for solving the essential matrix include the "5-point algorithm" [20] and the "8-point algorithm" [21].In other words, if we can fnd at least 5 pairs of corresponding points, we can calculate the essential matrix.In addition, considering that data redundancy can reduce errors, we can use more corresponding points to calculate the essential matrix.
Once we have calculated the essential matrix, we can decompose this matrix by the method of SVD to get [t] × and R [22].
However, the translation vector obtained by the abovementioned method can only obtain the direction of translation but cannot obtain the size of the translation X (x,y,z) Mathematical Problems in Engineering vector.Te translation vector and the real translation vector are proportional, and the ratio is called the "scale factor."Usually, the scale factor is deduced according to some objects with known size in the image, such as a circle with known radius and a square with a known side length.Te method of obtaining the scale factor can be referred to literature [23].

Image Edge Detection.
In this paper, edge detection refers to the detection of edge pixels of crack images.Essentially, image edge is a discontinuous and abrupt form of image local features.Terefore, edge detection is actually performed to fnd the position of local feature mutation in the image and obtain the two-dimensional coordinates of this position for the image.
A gray level is a basic attribute of images, and the gradient of images describes the change rate of the gray level.Generally speaking, the gray level of the edge region changes faster.Terefore, we can design edge detection algorithms according to this characteristic.In this paper, we adopt the Canny algorithm to detect the crack edge in the image [24,25].
Te core steps of the Canny algorithm are image smoothing, gradient calculation, nonmaximum suppression, and dual-threshold detection.For a typical crack image of a surface, the Canny algorithm is used for crack edge detection, and the detection results are shown in Figure 3.

Crack Measurement Experiment
In this section, we describe that how we adopt BSV with the MSC system to measure the maximum width of a crack.Te camera parameters used in the system are shown in Table 1.
In the following, we frst describe the experimental process and then verify the precision of our proposed measurement method.

Experimental Process.
We use the camera to collect an artifcially constructed crack image from two diferent directions.In addition, in order to estimate the scale factor, a black-and-white checkerboard is placed near the crack, and the size of the individual squares on the checkerboard is 10 mm × 10 mm.Te crack image we collected is shown in Figure 4.
Te camera shooting angle in Figure 4(a) is 80 °and that in Figure 4(b) is 70 °.Here, we need to pay attention to the camera shooting angle, and Figure 4 shows only a schematic diagram, which shows shooting from diferent angles; we do not have strict requirements for shooting angles.
To obtain the corresponding points required to estimate the essential matrix E, we adopt the SIFT algorithm to match the feature points of Figures 4(a) and 4(b).Te results are shown in Figure 5.
Te intrinsic parameter matrix of the camera is K. Ten, when the corresponding points and the grid size in the image are known, we can calculate the rotation matrix R and the translation vector t of the camera from position 2 to position 1 by the method described in Section 3.1.
In addition, we can use Equation (10) to calculate the homography matrix H from Figures 4(b) to 4(a) through K, R, and t.
In Equation (10), n denotes the unit normal vector of the imaging plane of camera position 2 in the coordinate system of camera position 1, d denotes the distance between the coordinate system origin of camera position 1 and the imaging plane of camera position 2, and n d � n/d.For any pair of corresponding points, for instance, x 1 and x 2 , H has the following properties.4 Mathematical Problems in Engineering After the camera motion estimation is completed, the Canny algorithm is used to detect the edges of the two images, respectively.Te results of edge detection are shown in Figure 6.
We set x 2i (u 2i , v 2i ) to be the projection point of X i (x i , y i , z i ) in Figure 4(b), where X i is a point on the edge of a crack in the world coordinate system.Ten, the corresponding point of x 1i in Figure 4(a) can be calculated by x 2i H, that is, Here, we need to pay attention that since the estimation results of camera motion parameters are generally slightly diferent from the real values, the coordinates of the corresponding points of the point x 2i in Figure 4(a) calculated by the homography matrix are often inconsistent with the real coordinates of the corresponding points.Ten, we can take the edge point closest to the Euclidean distance of the point x 2i H in Figure 4(a) as the corresponding point of x 2i , as shown in Figure 7. Te coordinates of the true corresponding point are recorded as x 1i � (u 1i , v 1i ).Now, we have obtained x 1i (u 1i , v 1i ) and its corresponding projection matrix M 1 � K [I, 0] and x 2i (u 2i , v 2i ) and its corresponding projection matrix M 2 � K [R, t].Ten, we can calculate the spatial three-dimensional coordinates of X i according to the method described in Section 2.2.
After the three-dimensional coordinate calculation of all points on the two edges of the crack is completed, the method in Section 2.1 can be used to calculate the maximum width of the crack.

Experimental Results.
Te solution results of the camera's intrinsic parameter matrix K and motion parameters R and t are shown in Table 2.
For the crack shown in Figure 4, its known maximum width (7.032 mm) is used as a reference value.Ten, under the conditions of a measurement distance of 0.5 m, 1.0 m, Mathematical Problems in Engineering 1.5 m, and 2.0 m, the crack width is measured according to the experimental process described in Section 4.1, and the shooting angles of the two cameras at the two positions for the four distances are roughly 60 °and 80 °, respectively.Te measurement results and measurement error are shown in Table 3.
It can be seen in Table 3 that, under diferent measurement distances, the BSV with the MSC system can achieve the measurement error within 5%.Tis shows that the proposed measurement method has good reliability.In addition, this method does not need contact measurement, so it has good convenience.
Besides, to verify the advantages of our proposed method over similar computer vision-based crack measurement methods, we compare the measurement precision of our method with the methods in literature [15,16].We set the measurement distance to 1 m and compare the measurement error under diferent camera shooting angles, and the cracks used for measurement are still shown in Figure 4. Te experimental results are shown in Table 4.
As shown in Table 4, under the condition that the camera imaging plane is parallel to the plane of the crack, that is, the camera shooting angle is 90 °; the measurement precision of the three methods is very close.However, when we change the shooting angle, only our method can maintain relatively reliable measurement precision.Tis fully shows that the proposed method has good reliability while ensuring fexibility.

Conclusion
In this paper, we have proposed a crack measurement method based on BSV with MSC using computer vision.Tis is an intelligent, fast, and highly automated measurement method.Te main contributions in this article are as follows: (1) We only use one camera to realize the function of traditional BSV.Compared with the traditional BSV with two cameras, this method is more economical.Compared with the ultrasonic method, microscopic method, sensor method, and traditional monocular vision method, the fexibility of this method is greatly improved.(2) We construct a complete crack measurement system and realize the noncontact automatic measurement of cracks in a large space.Te experimental results show that our method can ensure the measurement precision and has higher fexibility.(3) In this measurement system, we integrate the feature matching method, motion estimation method, and edge extraction method.It can directly calculate the crack size according to the collected image, which makes the measurement results more objective and less interfered by human factors.
Te measurement precision of our method is independent of the camera's shooting angle.In addition, compared with monocular vision measurement systems using the same imaging device, our method is more economical.Terefore, this method is economical, fexible, and has high precision.Furthermore, it can be combined with an unmanned aerial vehicle (UAV) so that it can measure cracks in complex or dangerous situations.

Future Work
In the measurement experiment, we use a relatively ideal example; that is, there is no interference of other adverse features in the measurement scene.In fact, when using the computer vision method to solve such problems, we must consider how to accurately segment the crack region in the image with other regions.
In future work, we plan to combine crack detection with crack measurement [13,14,26].We will frst use the current depth learning method to accurately fnd the location of the crack from the collected image and extract it.Ten, for the extracted crack image, we use the method in this paper to measure its size.In this way, the system will be more intelligent and more widely used.

Figure 2 :
Figure 2: Schematic diagram of surface crack measurement using BSV with the MSC system.

Figure 3 :
Figure 3: Typical material surface crack and its edge detection result.(a) Image of surface cracks.(b) Edge detection result of the crack.

Figure 4 :Figure 5 :
Figure 4: Crack images collected using the camera at the two positions: (a) position 1; (b) position 2.

Table 1 :
Parameters of the camera for the experiment.

Table 2 :
Solution results of parameters for the BSV with the MSC system.

Table 3 :
Measurement results and relative errors of crack width under diferent measurement distances.