Computer Vision-Based Pier Settlement Displacement Measurement of a Multispan Continuous Concrete Highway Bridge under Complex Construction Environments

. Various concrete bridges have been built across oceans, valleys, and mountains; however, the settlement displacement of bridge piers caused by environmental changes or self-weight during construction phases often leads to uneven stresses, cracking, and eventual collapse. To address the labor-intensive and high-cost issues of pier displacement monitoring using contact-type sensors, this paper proposes an automatic vision-based method for measuring pier settlement displacement under complex construction environments, such as complex image backgrounds, varying ambient light, and camera movement. In the proposed method, a deep learning network was frst employed to eliminate the adverse efect of complex image backgrounds and varying ambient light on the accuracy of target detection; then, an adaptive displacement extraction algorithm without a human-computer interaction process was developed to automatically extract the center coordinates of targets attaching to the bridge piers and reference platform; fnally, the pier settlement displacement was calculated by using the relative displacements obtained by a dual camera system to eliminate the measurement error caused by camera translation and rotation movements. Laboratory tests of a cantilever beam and feld tests of a continuous multispan concrete girder highway bridge under construction have successfully validated the efectiveness and robustness of the developed methodology. Te results obtained in this paper can provide some insights for engineers in applying computer vision technology for the real-time monitoring of bridge displacements.


Introduction
Various concrete, steel, and composite bridges have been built all around the world to span oceans, mountains, and valleys.Concrete bridges are the most common type of bridge in engineering.Bridge piers are necessary components for transferring trafc loads from the bridge deck to the foundations [1,2].Te settlement displacement of bridge piers caused by environmental changes or selfweight during construction phases is ubiquitous, which can lead to uneven stresses, cracking, and eventual collapse.Terefore, pier settlement displacement is one of the most important metrics for assessing the construction quality and safety of bridges, and it is essential to be monitored by advanced sensing technologies [3][4][5][6][7].Current displacement measurement technologies mainly depend on contact sensors, such as displacement transducers and fbre Bragg grating (FBG) sensors [8].However, sensor installation and data transmission are difcult when the bridge pier is difcult to access.To overcome the challenging problems of contact-type sensing technologies, researchers have developed various noncontact equipment for bridge displacement monitoring, such as total station [9], global position system (GPS) [10], and microwave radar [10][11][12][13].Although these noncontact technologies have been applied to measure structural displacements and pier settlements, they still have some limitations in practical applications.
With the maturity of optical cameras and artifcial intelligence, computer vision technology has been recognized as an inexpensive displacement measurement method with the capability of multipoint displacement measurement, high precision, and remote sensing [14][15][16][17][18][19].Te current computer vision-based displacement extraction algorithms can be roughly divided into image intensity-based methods and phase-based methods.Most of the computer visionbased displacement extraction methods use the image intensity-based method, such as target detection algorithms, edge detection algorithms, feature tracking algorithms, and template matching algorithms.For example, Ye et al. developed a continuous edge detection algorithm for structural deformation measurement by close-range digital photogrammetry system [20]; Feng et al. used a single camera to measure the multipoint dynamic displacement of a simply supported beam by template matching algorithm [21]; Tian et al. used the gradient-based Hough transform (GHT) method to measure the multipoint dynamic displacement of a cantilever beam under static load and impact load [22]; Havaran et al. used the random Hough transform algorithm to track the movement of elliptical markers attached to the structural surface to measure the multipoint displacement of the structure [23]; Tian et al. developed a line segment detection (LSD) and matching algorithm to calculate the dynamic displacements of bridge cables [24]; Shao et al. developed a novel monocular vision system for 3D vibration displacement measurement by using deep neural networks to learn the depth of scenes from captured images [25].In addition, researchers have developed various phase-based algorithms for structural displacement measurement.Chen et al. developed a phase-based motion magnifcation algorithm for structural displacement measurement and modal identifcation of simple structures [26]; Cha et al. developed a phase-based optical fow method for structural displacement measurement and bolt loosening detection using the unscented Kalman flters [27]; Valente et al. quantifed the amount of physical motion with the degree of magnifcation in phase-based displacement extraction method [28]; Shao et al. developed a target-free three-dimensional (3D) tiny displacement measurement method by deep learning and motion magnifcation technique [29,30]; Luo et al. proposed a broadband phase-based motion magnifcation and line tracking algorithm for cable displacement and cable tension force estimation [31].Although the computer vision-based method has been widely investigated for 2D and 3D displacement measurement of civil infrastructure, there are two challenging problems that need to be addressed when applying this method to complex construction sites.
Te frst problem is how to remove the negative efect of complex image backgrounds on the accuracy of target detection for subsequent displacement calculation.Due to the complex environment on the bridge construction site, the camera-captured images contain many invalid backgrounds, such as trees, buildings, construction machinery, workers, and so on.Tese complex backgrounds usually lead to false matching in the target detection and feature tracking processes for displacement calculation.In recent years, several deep learning algorithms have been developed to remove complex backgrounds from captured images to enhance the robustness of the vision-based displacement measurement methods.For instance, Zhang et al. used a pretrained fully convolutional network (FCN) model to remove the complex background information (i.e., pedestrian movement, buildings, trees) contained in the drone-captured video of an urban footbridge for dynamic displacement extraction of the bridge cables with an improved line segment detection algorithm [32]; Cheng et al. used the Yolov4 target detection network to remove the invalid background contained in the captured image of bridge piers; and the displacement trajectory of a bridge pier in the lifting process was extracted by tracking elliptical targets [33].However, no studies have comprehensively investigated the efect of various complex image backgrounds and varying ambient light on the accuracy of target detection from captured images of bridge piers in complex construction sites with deep learning methods.
Te second problem is how to compensate for the measurement error caused by camera movement and avoid the human-computer interaction process for real-time monitoring applications.On the bridge construction site, many uncontrollable factors interfere with the camera, such as the motion caused by the construction machinery, accidental hand touch, and wind-induced vibration, resulting in large errors in the subsequent displacement extraction process.To overcome this problem, researchers have attempted to use stationary objects in the background, such as nearby mountains or buildings, to compensate for the measurement errors caused by camera movement [34][35][36][37][38].However, it is difcult to fnd fxed reference points on the bridge construction site in the captured image due to the limited lens and resolution of the camera.On the other hand, computer vision algorithms, such as target detection, template matching, and feature detection algorithms, are usually employed to extract static or dynamic displacements from camera-captured images.In those algorithms, the GHT target detection algorithm has been widely investigated in civil engineering for circular target detection and displacement measurement because of its unique advantage of high accuracy [39].To cope with the efect of the perspective view on target detection results, a randomized Hough detection algorithm was used to track the movement of ellipse markers attached to the structure for displacement measurement [23].However, the computational efciency is seriously afected by the requirement to predefne the radius value when extracting the coordinates of the target center, which makes it difcult to automatically extract the displacement of bridge piers for real-time applications.
Aiming to address the above problems faced by computer vision-based displacement monitoring in complex construction sites, this paper proposes a computer visionbased automatic and highly robust method for the pier displacement measurement of a high bridge pier under construction.Te structure of this paper is organized as follows.Section 2 describes the theoretical framework of the proposed method, including the deep learning-based image background removal for accurate target detection, the automatic pier settlement displacement extraction algorithm, 2 Structural Control and Health Monitoring and the camera motion compensation method.Ten, laboratory validation of the developed method is presented in Section 3. Subsequently, the robustness and efectiveness of the developed method are further verifed by full-scale tests of a long-span bridge under construction in Section 4. Finally, the conclusions are presented in Section 5.

Proposed Methodology
2.1.Framework.Te framework of the proposed method for displacement measurement of bridge piers under construction from captured video by optical cameras is shown in Figure 1.In the proposed method, a dual-camera system was designed to capture the video of both the bridge pier and the reference target.Ten, a deep learning network was trained to remove the unnecessary complex background and varying ambient light contained in the captured video, from which circular targets attached to the bridge pier and reference points were accurately detected.Subsequently, the region of interest (ROI) was determined and mapped to the original image for displacement calculation, in which an adaptive threshold-based GHT algorithm was developed to extract multipoint displacements of the bridge pier and reference points.Finally, the pier settlement displacement is obtained by combining the raw displacements extracted from captured images of the main camera with camera motion calculated from the collected images of the reference target.

Deep Learning-Based Image Segmentation for Accurate
Target Detection.In construction sites, the video recorded by optical cameras usually contains the physical targets, trees, pedestrian movement, and construction machinery, which leads to the difculty of accurate target detection and displacement extraction.In addition, the varying ambient light afects the intensity distribution of camera-captured images in long-term measurement.Terefore, the U 2 -net deep learning architecture was employed to remove the invalid background contained in the recorded video (Figure 2).U 2 -Net is a two-level nested U-structure that is designed for object detection without the use of any pretrained backbones from image classifcation [40].Te U 2 -net network consists of a six-stage encoder, a fve-stage decoder, a saliency map fusion module that is attached to the decoder stages, and the fnal encoder stage [40].Te encoder and decoder structures include fve residual network structures, namely, RSU-7, RSU-6, RSU-5, RSU-4, and RSU-4F, where the RSU-4F uses dilated convolutions to replace the upsampling and the downsampling.Te fusion module then fuses the saliency maps produced by each layer to produce the fnal predicted probability map.Te loss function in the training process is defned as follows: where l (m) side and l fuse are the loss of the side output salience map and the loss of the fnal fusion output salience map, respectively, and w (m)  side and w fuse are the weights of loss of l (m) side and l fuse , respectively; Te standard binary cross-entropy is used to calculate each loss term l: where (H, W) and (r, c) are the pixel coordinates and the height and width size of the captured image, and P G(r,c) and P S(r,c) are the pixel values of the ground truth and the predicted saliency probability map, respectively.Te evaluation metrics max F β and MAE [41,42] are used in the U 2 -net network training process to assess whether the model has converged.F β is used to evaluate both the precision and recall.
where β is a number from 0 to 1, and it is set to 0.3 in this training process.Te max F β is chosen as the evaluation index, and the higher the value obtained, the better the training accuracy obtained.Te mean absolute error (MAE) is calculated to evaluate the average per-pixel diference between a predicted saliency map and its ground truth.
where P(r, c) is the predicted probability map and G(r, c) is the corresponding ground truth.Te lower the value of MAE, the better the training performance.

Automatic Displacement Extraction and Camera Motion
Compensation.After removing the complex image background and varying ambient light efect in the captured video, target-tracking algorithms can be further employed to extract structural displacements.An adaptive thresholdbased GHT algorithm was developed to automatically detect the radius of circular targets contained in the captured video and calculate structural displacements, as shown in Figure 3. Te basic idea of the developed algorithm is described as follows.
First, the radius of circular targets contained in the captured images is estimated for subsequent automatic displacement calculation, as shown in Figure 3(a).In this step, the Gaussian flter was employed to eliminate various noises contained in the captured images, which is expressed as follows:

Structural Control and Health Monitoring
where k and σ are the dimension size and standard deviation of the Gaussian flter, respectively, and i and j are the coordinates in the X and Y direction.
After applying the Gaussian flter to the raw images, the image gradient in the X and Y direction can be calculated as follows: where G x and G y are the image gradient in the X and Y directions, S x and S y stands for the Sobel operator, and I is the intensity matrix of the recorded image.By combining the image gradients expressed in equation ( 6), the image gradient at pixel (i, j) is calculated

􏽱
. Ten, the nonmaximum suppression (NMS) algorithm was adopted to eliminate the errors caused by edge detection.Te linear interpolation of two adjacent gradient values was calculated as follows: where t � |G y (i, j)|/G x (i, j) is the proportional ratio for gradient value calculation.
If the current gradient value is larger than the computed gradient value in the positive and negative directions, the current pixel point is considered the edge point.Ten, the binary images of attached circular targets can be obtained by a defned threshold.After that, the number of pixels with grey values of 1 in the binary image was counted, and the pixel radius value of the circle in each image can be estimated as follows: where r i is the radius value (pixel) of the i-th frame of the recorded images, n is the number of circular targets to be detected, N is the number of pixels with a grey value of 1 in the binary image, and k is the number of images.S (1) side S (2) side S (3) side S (4) side S (5) side S (6) side

Structural Control and Health Monitoring
After obtaining the radius value of each image, the averaged radius value of all images r �  k 1 r i /k was calculated as the fnal results for the subsequent calculation.An optimized range with the calculated radius value was used to automatically detect possible circular targets in the gradient Hough transform algorithm, as shown in Figure 3(b).In this step, a discrete characteristic curve is defned on the image gradient feld to determine the circle center coordinates in the image.
Assume that the image sequences corresponding to the time t 0 , t 1 , • • • t n are collected, and the GHT algorithm with an optimized radius threshold is applied to all the captured images to automatically detect the circle center.Ten, the two-dimensional displacements of the bridge piers and reference targets in pixel coordinates can be calculated by subtracting the circle coordinates of subsequent images from those of the reference image, as shown in Figure 3(c).
where d t i x and d t i y are the vertical and horizontal displacements of the target at the time t i ; X t i c,p and X t 0 c,p are the circle center of the target p at the time t i and t 0 in the horizontal direction respectively; Y t i c,p and Y t 0 c,p are the circle center of the target p at the time t i and t 0 in vertical direction, respectively; p is the number of artifcial targets attached to the bridge surface; and m is the total number of the attached targets.
Finally, the pixel displacements need to be converted into physical displacements using the pixel-to-displacement conversion factor.In this study, the known target dimensions in the captured video are used to calculate the conversion factor, S � d physical /d pixel .
Te above equation can extract the raw displacements of the bridge piers and the reference target from the captured video, but the extracted displacement contains the measurement error caused by the camera movement, such as the vertical translation motion and rotation motion in the vertical and horizontal direction, respectively.To address this issue, a dual-camera system was used to compensate for the efect of camera motion on extracted displacements, as shown in Figure 4.If the support platform has a vertical translation movement (shown in Figure 4(a)), the true displacement of the pier settlement is calculated by subtracting the raw vertical displacement of the main target attaching to the bridge pier from that of the reference target.Te calculation formula is expressed as follows: where d t i y is the relative displacement obtained by the dual camera system; d t i y,C 1 and d t i y,C 2 denote the vertical raw displacement of the bridge pier and reference target, respectively; S C 1 and S C 2 are the conversion factors of camera 1 and camera 2, respectively; and C 1 and C 2 stand for the main camera and secondary camera for video recording of bridge piers and reference targets, respectively.
If the support platform has a rotation movement in the vertical direction (as shown in Figure 4(b)), the true displacement of the pier settlement can be calculated by the following equation: If the support platform has a rotation movement in the horizontal direction (shown in Figure 4(c)), the false displacements induced by the two cameras are the same; therefore, the true pier settlement is calculated as follows: 6 Structural Control and Health Monitoring Substituting the displacement calculation formula in equation (9b) into ( 10)-( 12) results in the fnal expression for pier settlement calculation by considering three types of camera movements, namely, where m and n denote the total number of attached targets on the bridge pier and reference platform, respectively.It should be noted that equations (13a)-(13c) are used to eliminate the vertical translation movement and rotation movement in the vertical and horizontal direction, respectively.

Laboratory Testing of a Cantilever Beam
3.1.Experimental Setup.Te efectiveness of the proposed methodology is frst verifed by laboratory testing of a cantilever beam.Te experimental setup of the cantilever beam under investigation is shown in Figure 5. Te cantilever beam has a length of 2 m, and the height of the cross-section is 0.4 m.A total of six circular targets were attached to the cantilever beam for displacement extraction, and six displacement gauges were also installed at the corresponding position to verify the accuracy of the proposed method.In this experiment, a high-speed camera with a resolution of 1024 × 1024 pixels and a frame rate of 1000 Hz was used to capture image sequences of the circular targets when the cantilever beam was excited by an impact hammer.

Experimental Results.
After acquiring the images of the investigated beam under impact loading, the U 2 -net was used for image background removal.Ten, the developed displacement extraction algorithm was used to detect circular targets from the preprocessed images.Figure 6(a) shows the circular target detection results of the cantilever beam without complex background segmentation with a radius range of [0.8r, 1.2r] in the conventional gradient Camera translation motion (Vertical) Reference target Main target Camera rotation motion (Vertical)

Δα
Δα Camera 1 C am er a 1 C am er a 2 Camera 2 Camera rotation motion (Horizontal) In comparison, there are no false circular target detection results when the raw image is processed by the proposed method (Figure 6(b)).However, the detected   Structural Control and Health Monitoring circular targets are missing when the circle radius is to a smaller range [0.9r, 1.1r], as shown in Figure 6(c).Specifcally, one, three, and two circular targets are lost in the frst, third, and tenth frames of the recorded images of the studied cantilever beam, making it impossible to calculate displacements at these targets.However, in the preprocessed images using the proposed method, there are no missing targets (Figure 5(d)).It is concluded that incorrect target detection results and target-missing phenomena occur in the process of circular target detection by the conventional method due to the complex image backgrounds.In addition, the change in the radius range afects the circular target detection results due to the invalid image background.
After the image background segmentation, the developed automatic displacement extraction algorithm is used to extract the dynamic displacements of the six circular targets attached to the cantilever beam.Te extracted displacements are compared with the directly measured displacements collected by the displacement meters to verify the accuracy of the developed algorithm (Figure 7).It can be seen that the extracted displacements by the developed algorithm are in good agreement with the directly measured displacements by the displacement meter, thus verifying the correctness of the developed algorithm.
Furthermore, three metrics-the correlation coefcient (ρ), the root mean square error (RMSE), and the coefcient of determination (R 2 )-are used to quantitatively evaluate the measurement accuracy of the developed algorithm.
where x v and x c are displacements obtained by the developed method and the displacement meters, respectively, n represents the total number of captured images, μ v and μ c are the average values of calculated displacement time histories, ρ stands for the correlation degree between two kinds of displacements, and R 2 represents the matching degree of two curves.Te specifc values of the three metrics between the displacements calculated by the developed algorithm and the results measured by the displacement meters are given in Table 1.It can be seen that the displacements calculated by the developed method are in good agreement with the displacements measured by the displacement meters, and the maximum RMSE of the two displacement curves is 0.0613, the minimum ρ and R 2 are 0.9198 and 0.8954, respectively, validating the accuracy of the developed algorithm for displacement extraction.In addition, the target missing rate by using the developed method and the traditional method was also compared.It is seen that measurement points 1 to 6 have diferent proportions of the target missing rate in the traditional method, whereas there is no target missing in the proposed method, verifying the correctness of the developed method.

Field Testing of a Long-Span Highway Bridge
To verify the robustness and stability of the proposed method in complex environments, feld tests were carried out on a continuous multispan long-span highway bridge under complex construction environments.Tis section provides details of the feld tests and the results of the pier settlement displacement of the investigated bridge.

Bridge Description.
Te bridge under investigation, called the Lugou River Bridge, is a continuous rigid frame bridge, as shown in Figure 8. Te superstructure is a prestressed continuous rigid-frame with a cast-in-place cantilever construction method; the substructure adopts doublelegged solid piers and hollow thin-walled piers; and the foundation is a group pile foundation.Furthermore, the bridge is a separate two-line bridge with a main bridge span arrangement of (96 + 5 × 180 + 96) m, a deck width of 16.25 m, and a maximum bridge height of 209.57m.Te cross-section of the main girder is a single box girder, and the height of the box girder is 4.0 m at the center of the span and 11.5 m at the top of the pier.

Overview of Field Tests.
To minimize the measurement error caused by the camera movement during the pier settlement monitoring, a dual camera system was designed for the measurement, using two optical cameras (MV-CA050-10GC) mounted on a tripod with a resolution of 2448 × 2048 pixels, as shown in Figure 9(a).Image sequences of the bridge pier (main target) and the reference point (subtarget) were acquired simultaneously, with the reference point being a fxed platform chosen to suit the site conditions.Te main target at the bridge pier and the subtarget at the reference point are shown in Figures 9(b) and 9(c), respectively.Te physical radius of the outer circle of the circular target used in the feld test is 100 mm, and the pixel size is approximately 60. Te camera is roughly located in the middle of the main target and subtarget, about 30 meters away from both sides.
In the feld test, three test conditions were designed, as shown in Table 2. Specifcally, case 1 is set up to validate the efectiveness of the camera motion compensation, and cases 2 and 3 are used to validate the feasibility of the proposed method in monitoring pier settlement displacement.

Complex Background Removal for Accurate Target
Detection.Before extracting the pier settlement displacement, the deep learning-based image background removal method is used to remove the invalid background contained in the image.Te training database contains images with circular targets and common backgrounds (i.e., pedestrians, buildings, trees, etc.).In the training phase, the initial Structural Control and Health Monitoring learning rate, the number of training epochs, and the batch size set to 0.001, 50, and 2, respectively.Te segmentation results of circular targets after model training are shown in Figure 10.It can be seen that the U 2 -net model can efectively detect circular targets in various complex backgrounds.
Te loss function curve during model training is shown in Figure 11(a), which shows that the training loss and tar loss decrease rapidly with training epochs and then fuctuate around 0.08 and 0.01, respectively.In addition, the evaluation metrics are also shown in Figure 11(b), where max F β gradually increases and stabilizes at 0.96, and MAE gradually decreases and stabilizes at 0.005.
Te circular target detection results of the bridge pier without and with complex image background segmentation are shown in Figure 12.It can be seen that false circular target detection results occur in the captured images, as shown in Figures 12(a) and 12(c).Several false circular targets are detected in the original image when the radius     12 Structural Control and Health Monitoring not only improves the robustness of the displacement extraction but also avoids the human-computer interaction process, which greatly improves the computational efciency.

Pier Settlement Displacement Monitoring under Complex Environments
(1) Case 1: Pier Settlement Monitoring under Disturbance.To verify the efectiveness of the camera motion compensation method, three groups of experiments were designed in Case 1, where the camera was artifcially disturbed 1, 2, and 3 times, respectively, during the measurement process.Te experimental setup and the recorded images are shown in Figure 13.It can be seen that the brightness of the captured images varies with the measurement time except for the artifcial disturbances.Ten, the developed automatic displacement extraction algorithm is applied to the segmented circular targets.Te extracted displacements of the bridge pier and the true pier displacement are shown in Figure 14.Tere is an error of about − 10 mm after a single artifcial disturbance, and then the error returns to 4 mm.Te proposed camera motion compensation algorithm was applied to process the extracted raw displacements, and the maximum measurement error was reduced to 0.09 mm.
Te results of pier settlement monitoring under two artifcial disturbances are shown in Figure 14(b).It can be seen that the camera system produces measurement errors of approximately 3 mm and 9 mm under the frst and second disturbances.After applying the camera motion compensation algorithm, the maximum measurement error is reduced to 0.15 mm. Figure 14(c) shows the measurement error of the camera under three times of disturbances, the maximum measurement error is up to 11 mm but is reduced to 0.13 mm by the proposed method.Te averaged absolute displacement values under one, two, and three times of disturbance by the proposed method are 0.015 mm, 0.2750 mm, and 0.088 mm, respectively.In contrast, the averaged absolute displacement values in the three cases without background removal are 0.595 mm, 0.791 mm, and 0.559 mm, respectively, which is much larger than the results obtained by the proposed method.It demonstrates the high accuracy of the developed method for displacement measurement.It can be seen that the measured displacements have obvious deviations under the camera disturbance, but the proposed method can reduce the measurement error.It should be noted that the accuracy of the measured displacements is afected by the pixel/physical size scale and camera noise.Te systematic investigation of the displacement measurement errors will be conducted in future work.
(2) Case 2: Pier Settlement Monitoring during Lifting of Hanging Baskets.Te pier settlement was monitored using the developed dual camera system when the main beam basket was lifted, and the captured images are shown in Figure 15.It is seen that the brightness of the captured image varies with the natural lighting conditions and even contains shadows in the captured image of the reference target at the time of 6400 s.Terefore, the complex image background needs to be removed by the deep learning method.Te circular targets mounted on the bridge pier and the reference target are accurately detected regardless of various complex image backgrounds, as shown in Figures 15(a Te extracted displacements of the bridge pier in Case 2 are shown in Figure 16.Te results show that the camera was slightly disturbed during the lifting process of the hanging basket (Figures 16(a) and 16(b)), resulting in measurement errors.Te true pier settlement displacement by the camera motion compensation method is shown in Figure 16(c).It can be seen that the pier settlement displacement fuctuates around 0 mm, and there is no obvious settlement tendency.As the bridge foundation is a deep pile foundation with a length of 40 m, there will be no settlement of the piers when the hanging basket is lifted.
(3) Case 3: Pier Settlement Monitoring during Concrete Pouring.Te extracted displacement of the bridge pier during the concrete concreting process of segment 1 of the main beam is shown in Figure 17.It is seen that the extracted raw displacement contains large spikes without the deep learning-based image segmentation method (Figures 17(a) and 17(b)).Te pier settlement displacement during the test period is divided into two stages, which gradually increase from 0 mm and stabilize at 0.15 mm, as shown in Figure 17(c).During the pouring process of segment 1, the pier settlement increases by about 0.15 mm due to the gradual increase of the concrete wet load.Tis is because the ratio of the weight of the hanging basket to the weight of the poured concrete segment is generally 0.3 to 0.5 in the actual project.Terefore, it is reasonable that the settlement value of the pier in Case 3 is relatively large.Te test results of both Cases 2 and 3 have verifed the robustness of the proposed method for pier settlement monitoring of long-span bridges.Structural Control and Health Monitoring

Conclusions and Future Works
Te complex image background, varying ambient light, and camera movement afect the robustness of the computer vision-based method for displacement monitoring, and the human-computer interaction process in the conventional method prevents automatic and real-time displacement extraction.To overcome the above problems, a computer vision-based automatic and highly robust pier settlement measurement method was developed using deep learning technologies.Te detailed results are as follows: (1) A deep learning-based complex background removal method was used to eliminate the efect of invalid backgrounds (i.e., pedestrian movement, trees, construction machinery, etc.) and the varying ambient light on the accuracy of target detection.Ten, an adaptive threshold-based GHT algorithm was developed to accurately and automatically calculate pier settlement displacement.Te relative displacement of the bridge pier concerning the reference target was considered as the true pier displacement to compensate for the camera movement induced by construction machinery on the construction site and accidental hand contact by workers.(2) Laboratory tests on a cantilever beam were carried out to verify the accuracy and robustness of the developed algorithm.Te test results show that the displacements extracted by the proposed method are in good agreement with those directly measured by displacement gauges.Te RMSE of the two types of displacement curves is 0.0613, and the minimum correlation coefcient (ρ) and determination coefcient (R 2 ) are 0.9198 and 0.8954, respectively, verifying the correctness of the developed method.(3) To validate the efectiveness and robustness of the proposed method, feld tests were carried out on a long-span, high-pier highway bridge under construction.Te results show that the proposed method can efectively reduce the measurement error caused by the camera movement and eliminate the adverse efects of complex image backgrounds and varying ambient light on the measured displacements.Te pier settlement displacements obtained by the proposed method under three types of experimental conditions are consistent with the actual engineering project.
In conclusion, the proposed method has great potential for use in noncontact measurement of pier settlement in harsh environments.Future work will focus on the development of a portable computer vision system for real-time monitoring of bridge displacement.In addition, a stereo vision system and an advanced monocular vision system will be developed to measure the three-dimensional displacements with high accuracy.Te efect of the pixel/physical size scale, camera noise, and ambient temperature on the accuracy of the displacement measurement in feld tests also needs to be systematically investigated.

Figure 1 :Figure 3 :
Figure 1: Framework of the proposed method.

Figure 2 :
Figure 2: Framework of deep learning-based image segmentation for target detection.

Figure 4 :
Figure 4: Schematic diagram of diferent camera motion: (a) translation motion; (b) rotation motion in vertical direction; (c) rotation motion in horizontal direction.

Figure 6 :Figure 5 :
Figure 6: Target detection results of the studied cantilever beam.(a) Target detection without background removal with a radius of [0.8r, 1.2r]; (b) target detection with background removal with a radius of [0.8r, 1.2r]; (c) target detection without background removal with a radius of [0.9r, 1.1r];(d) target detection with background removal with a radius of [0.9r, 1.1r].

3 Figure 7 :Figure 8 :
Figure 7: Displacement calculation of the cantilever beam with the developed automatic algorithm.

Figure 9 :
Figure 9: Field testing setup: (a) image acquisition system; (b) main target at the pier; (c) subtarget at the reference point.

Figure 12 :Figure 13 :
Figure 12: Target detection results of the studied bridge pier: (a) target detection without background removal with a radius of [0.8r, 1.2r]; (b) target detection with background removal with a radius of [0.8r, 1.2r]; (c) target detection without background removal with a radius of [0.9r, 1.1r];(d) target detection with background removal with a radius of [0.9r, 1.1r].

Figure 17 :
Figure 17: Displacement results of case 3. (a) Raw displacement of the bridge pier; (b) raw displacement of the reference point; (c) true pier settlement value.

Table 1 :
Performance metrics of the ATGHT algorithm.

Table 2 :
Field test conditions.