Machine Learning-Based Distraction-Free Method for Measuring the Optical Displacement of Long-Span Bridge Structures

,


Introduction
Te structural properties of bridges continuously deteriorate under the action of environmental erosion, material aging, and vehicle overload.Defection monitoring, as a key part of bridge structural health monitoring, can help clarify the working state of the bridge and identify abnormal changes promptly, thereby facilitating the prevention of sudden disasters.Photogrammetry-based defection measurement methods can enable multipoint synchronization, real-time dynamic or static monitoring, and fne target positioning and have thus been widely used in structure displacement monitoring [1][2][3].
However, photogrammetry is susceptible to complex external factors such as environmental vibrations [4][5][6], illumination changes [7,8], rain and fog [9], occlusion [10,11], temperature changes [12,13], and atmospheric disturbances [14,15], which can lead to inaccurate image displacement extraction.Atmospheric disturbances decrease the accuracy of precision optical measurement results such as morphology, displacement, and velocity [16].Image restoration techniques are typically applied for precision optical measurement [17,18].Notably, the correction efect of signal fltering [19,20] on the displacement results is limited as it does not involve optical principles.Moreover, correction methods based on the design of camera systems [20] cannot be applied to singlecamera-based measurement.Turbulent image processing methods [21], widely used in the military feld, are aimed at target detection and cannot be directly applied for displacement measurement.In this context, the precise tracking of the target displacement must be realized to increase the displacement measurement accuracy based on the use of a single camera in scenarios involving atmospheric disturbances.In summary, the most efective strategy for single-camera-based displacement measurement is to enhance the subpixel accuracy of target tracking.
Center positioning algorithms such as the centroid method, ftting method, and least-squares ftting method [22] are widely used in real bridge applications.Te basic principle is to perform statistical analysis or ftting of grayscale images in the efective calculation window (ECW).Te existing studies on enhancing the center detection accuracy [23,24] have focused on human-made targets with uniform gray levels, and the problem of atmospheric disturbances has not been extensively considered.Mahrt et al. [25] highlighted that atmospheric disturbances may lead to blurring and distortion of the target image, resulting in target center positioning errors.Te key problems can be summarized as follows: (1) Imaging blurring and distortion lead to inaccurate determination of the ECW.When selecting the target area window, the appropriate window size must be determined, and the pixels in the ECW directly participate in the central calculation.Te traditional threshold-based window determination method is susceptible to environmental interferences.To measure the infrared target radiation intensity, Yang et al. [26] proposed an adaptive ECW determination method and a real imaging region identifcation method [27,28] based on the principle of normal distribution.Notably, these methods are suitable only for light targets with regular shapes, and any possible imaging distortion is ignored.Consequently, this paper proposes an adaptive ECW selection method based on energy accumulation to alleviate the infuence of environmental noise.(2) Imaging blurring and distortion, especially imaging distortion, make it difcult to guarantee the positioning precision based on a single algorithm [26].To solve this problem, several scholars have proposed weighted positioning methods [29,30].However, the determination of weights is challenging.In general, the internal mechanism associated with the distortion of the target center by atmospheric disturbances and its features is complex.Terefore, Wang et al. [31] attempted to determine the weights through a back propagation neural network (BPNN).However, BPNNs operate based on the empirical risk minimization criterion, which is prone to overftting and getting trapped in local optima.In comparison, the least-squares support vector machine (LS-SVM) [28] has a higher training speed and prediction accuracy.Notably, the ECW determination in the study of Wang et al. [31] was based on a fxed threshold value.Considering this research background, in this study, an adaptive ECW selection strategy based on energy accumulation is used to establish a weighted center location method using LS-SVM.
Te abovementioned environmental infuences can be corrected by image preprocessing.However, traditional image processing methods may be inefective in cases involving extreme interference problems such as dramatic illumination changes, shadows, occlusion, or unexpected camera shaking.Recently, deep learning (DL) techniques have been used to address these complex phenomena [32].For example, Xu et al. proposed a novel distraction-free target tracking approach by integrating a DL-based Siamese tracker [33] with traditional correlation-based template matching.However, DL methods are also inefective for extreme cases, for instance, those involving severe occlusion.
In addition, bridge structures typically have few surface features and measurement points cannot always be substituted over the measured section, resulting in data loss.
Other data anomalies such as omission and loss, jump points, drifting, and trend mutation are also commonly encountered in structure health monitoring (SHM).To solve these issues, machine learning algorithms have been applied in the feld of intelligent diagnosis, especially to identify abnormal data and structural damage [34][35][36].Among the existing machine learning algorithms, relevance vector machine (RVM) can minimize the regression error and exhibits a high generalization and antinoise disturbance abilities.Moreover, the RVM can adapt to the characteristics of nonlinear time sequences of bridge health monitoring system data and exploit the correlation between the data and selected training samples to predict the missing data or correct the abnormal data [37].Optical methods can be used to synchronously monitor multiple measurement points and provide data support for the model training of RVM.Terefore, in this study, such methods are used to address extreme interference problems.
Te remaining paper is organized as follows: Section 2 introduces the basic principles and the validation tests of the proposed method, including the center location algorithm considering the atmospheric disturbance and the data selfdiagnosis based on the RVM for adverse scenarios.Section 3 describes the feld-monitoring test performed over a long-span cable-stayed bridge.Section 4 presents the concluding remarks.

Methodology
Figure 1 shows the framework of the proposed singlecamera-based structural displacement measurement method, which includes target tracking, displacement conversion, and data correction.A center detection algorithm is used to locate the target regions in the image plane.Next, the extracted pixel displacement is converted to the physical displacement through the ftted object distance using an existing method [36].Although several variants of target tracking methods are available, their performance is inefective in a feld-monitoring campaign involving environmental variations or other nondetectable obstructions.To overcome this limitation, a novel target tracking approach is developed in this study by integrating the distraction-free center localization algorithm and machine learning-based data diagnosis technology.Te key principles are introduced in Sections 2.1 and 2.2.First, a rough calculation window sized M × N pixels is determined in the initial reference image, and the energy E of the region is calculated using the following equation: Structural Control and Health Monitoring where g (x, y) represents the gray level at pixel coordinate (x, y).Because the edge may be fuzzy owing to the environment vibration, the pixels in this rough window cannot be directly used to calculate the center coordinates.An energy concentration area that satisfes equations ( 2) and ( 3) is determined as the ECW.

Target Location Method against Atmospheric Disturbance
Specifcally, when the gray level g (x, y) is larger than the threshold g T , the energy of the energy concentration area must be larger than η times the total energy, where η (η � 85%) characterizes the energy concentration of the target.Otherwise, the value must be adjusted until both conditions are satisfed.Tis step must be implemented only at the initial moment.In the subsequent moments, only the threshold must be slightly adjusted, assuming that the concentrated energy of the target imaging remains nearly unchanged.Subsequently, the central coordinates are calculated using the gray information of this ECW.When calculating the centroid using equation ( 4), the gray value is the binarized gray g 1 (x, y).
Diferent from equation ( 4), to determine the squaredgray-based centroid location using equation ( 5), the original gray value is needed.Similarly, to reduce the infuence of calculation area selection on central extraction results, this study only allows the pixels in ECW to participate in the calculation.However, the gray values of pixels in ECW cannot be directly used.Because the light intensity of both the active target and the external environment is not stable, this can also lead to fuctuations in positioning results.To alleviate this infuence, this study proposes to optimize the grayscale using the threshold value g T .Terefore, the expression is shown as follows: Te weighted relationship between the centroid (x 1 , y 1 ), squared gray centroid (x 2 , y 2 ), and real center (x W , y W ) can be expressed as Te determination of weight η i is critical.However, owing to the complexity of atmospheric disturbances, the internal mechanism between the disturbed detection result and the real center position cannot be determined intuitively.In this study, the ftting function of the LS-SVM algorithm is used to address this problem.Te nonlinear regression model of LS-SVM can be expressed as where α i is the Lagrange multiplier constituting a vector α � T and b is the amount of deviation and K(x i , x) is the kernel function.Te radial basis kernel function with high data antinoising ability is used in this study equation as follows: In practical applications, λ 1 and λ 2 can be learned from a static experiment.Te experimental environment here should be consistent with the real test environment.Tere is no relative displacement between the target and the camera, but camera noise, environmental noise and algorithm errors lead to the displacement of the tracked target.So to obtain the displacement of higher precision, the end condition of model training is that the displacement (noise) variance equation (10) in the time domain is less than the threshold.Tese weights can be further used for the following practical measurement tasks.
In general, indoor or close measurement conditions easily provide sufcient conditions for the above process.However, for remote outdoor measurement, such as bridge displacement measurement concerned in this study, the piers or bearings are ideal stable points for static experiments.Although these reference points are not completely stable, this is not inconsistent with the end conditions equation (10).But there is a new problem that the measuring point of bridge displacement, such as the mid-span point, is not close to the stable reference point, which means that the measuring environment of the static experiment and the displacement test are not the same.However, the turbulence characteristics within the scope of the engineering site are uniform and will not change signifcantly in a short time.So in the case of long-term monitoring, the camera must be adjusted to ensure that both the reference point and measurement points are being captured.Structural Control and Health Monitoring

Outdoor Static Validation Test.
To evaluate the antiatmospheric disturbance performance of the proposed center detection algorithm, an outdoor test was performed (Figure 2(a)) in the summer.Te test site is near the river.High temperatures and air humidity create conditions for atmospheric fow on the ground.To reduce the efects of daytime light, the experiment began in the evening.An infrared LED lamp (wavelength of 850 nm) controlled by a high-precision electric displacement table was used as the target (Figure 2(b)).Te degraded target is shown in Figure 2(c).Te range of the electric displacement table was 500 mm, and the accuracy was 0.1 mm.Te object distance was 50 m.
First, the target remained stationary, and the image sequence was obtained with an acquisition rate of 2 frames/s.Te gray square centroid method, binary gray centroid method, and proposed method were used to extract the center pixel coordinates, as shown in Figure 3(a).Te discreteness of the location results obtained using the proposed method was smaller than those of the other two methods, corresponding to a higher resistance to disturbance.Subsequently, the target was moved in steps of 1 mm through the displacement table.Te displacement measurement results are shown in Figure 3(b).To evaluate the measurement accuracy, the root mean squared error (RMSE) was calculated using the data of the displacement table as the reference values.By contrast, the displacement measured using the weighted positioning algorithm exhibited the highest accuracy and stability.

Te Principles of Data Diagnosis Based on a Relevance
Vector Machine.When the vision-based displacement measurement method was applied to real structures, two types of data anomalies are typically caused by environmental vibrations: (1) missing data owing to the failure of target positioning, attributable to drastic illumination changes, shadows, or severe occlusion and (2) abnormal data (data jump or shift) caused by unexpected camera shaking or other unknown factors.Considering the similarity of the vibration responses of diferent sections of long-span bridges, the trained RVM was proposed to be used to supplement the missing data and correct the abnormal data.Figure 4 shows the process fow of the RVM regression model.Due to limited space, detailed principles will not be introduced.
Te implementation process of RVM-based abnormal data identifcation and correction is as follows: (a) Select the normal displacement data of diferent sections as the training sample (b) Initialize the kernel functions and the hyperparameter α.Gaussian kernel function was adopted in this study because of its high regression accuracy and operation speed.Te insensitive loss parameter is set as u � 0.02, penalty coefcient C is 10, and error accuracy σ is 0.0001 (c) Te maximum a posteriori probability (MAP) method was used to solve the weight coefcient w, and then the covariance matrix  was calculated (d) Update the hyperparameter α according to w and .
Repeat step (c) until data residuals satisfy the accuracy requirement, where f data is the training sample data and f RVM is the predicted value (e) Get the predicted samples data pred � x pred,1 ,  x pred,2 , • • • , x pred,n } of the abnormal monitoring samples data origin � x 1 , x 2 , • • • , x n   using the nonlinear model of the trained RVM (f ) Locate the abnormal data using the generalized 3delta method (g) Replace the abnormal data or missing data with predicted data

Verifcation Test of an Impacted Steel Beam.
To evaluate the reliability of the abnormal data diagnosis method, the displacement measurement data of an impact test based on a high-speed camera was analyzed, as shown in Figure 5(a).Te beam was 1.5 m long, and six measurement points were evenly arranged on it.Figure 5(b) shows the vibration-response time history curves for all measurement points, generated under the impact of a force hammer.Te vibration displacements at P2, P3, and P4 from frames 1∼800 were used as the training samples of RVM.Ten the trained model was adopted to correct the results from other frames.Te case for abnormal data was established by falsifying the data of measurement point P4.Similarly, parts of the data of measurement points P3 and P4 were deliberately erased to establish the case for missing data.Te measured displacement at P2 remained unchanged, and then combining the trained model, the predicted displacement at P3 and P4 was obtained.Finally, the 3-delta method was used to locate the abnormal data.Te predicted value is compared with the real measured value, as shown in Figure 5(c).Te missing data identifcation and complementation results are shown in Figure 5(d).It is found that the diference between the predicted and measured values was less than 5%.

Application to Vortex-Induced Bridge Vibration Response Measurement
Te measured bridge is a sea-crossing cable-stayed bridge with a main span of 888 m, as shown in Figure 6(a).An unexpected vortex-induced vibration (VIV) event was observed on this bridge, potentially caused by the temporary cover placed on the bridge deck during vertical hanger replacement.In general, the use of traditional contact-type sensing technologies on this bridge when VIV occurs is dangerous and time-consuming.In contrast, the proposed camera-based displacement method can satisfy the sudden and urgent measurement requirements.During the day, the camera was set up under the bridge to track the drainage holes evenly distributed under the main girder to measure the defection, as shown in Figure 6(b).To overcome the problems associated with poor illumination at night, the Structural Control and Health Monitoring camera was placed on the shore to track the evenly distributed LED lights, as shown in Figure 6(c).Te details of the feld measurement at diferent times are described in the following sections.

Reliability Evaluation Using a Microwave Radar.
As shown in Figure 7(a), to verify the measurement accuracy of the proposed optical method, a high-precision microwave radar was used to measure the defection at 1/8 span.In this test, the pitching angle of the camera was approximately 23 °.
As shown in Figure 7(b), the width of the bridge bottom was used for the scale factor calibration of the section of interest, and the evenly distributed drainage holes were tracked.Structural Control and Health Monitoring respectively.It can be seen that the results obtained by the proposed centroid tracking algorithm were closer to the data from the radar.However, it cannot be concluded that the proposed method is more accurate.Because the measurement results of the radar correspond to the average displacement for a cross-section and the cross-section position determined by the radar is not necessarily consistent with that tracked by the camera.But to a certain extent, the reliability of the proposed method was proved.8 Structural Control and Health Monitoring of missing data were considered: (1) the left and right measurement points were not simultaneously blocked, as in the cases of 5/8 and 6/8 span, and (2) both points were simultaneously blocked, as in the case of 7/8 span.Te defections of the other sections and diferent measurement points in the same section could be used to compensate for missing data.In general, a higher correlation between the measurement points can help enhance the accuracy of the prediction results.Terefore, a data correlation analysis of the 4/8 and 5/8 spans was performed using the multiple regression analysis method, as shown in Figure 8(c).Te correlation of defection data in the same section was higher than that between adjacent sections.Terefore, it was preferable to use the data from the same section for the data prediction.Nevertheless, it was preferable to use the data of adjacent measurement points when both measurement points in the same section were occluded.Since the moment of occlusion occurrence was known in this test, the displacement from frames 0∼3000 was used as the training samples of RVM.Te data completion results are shown in Figure 8(d).In another implementation, the camera underwent an accidental collision, resulting in data jumps and data drifts, as shown in Figure 9. Instead of recognizing the time of mutation and subtracting a constant from the subsequent data, these two types of phenomena are uniformly processed as data anomalies.Tis is because some of the abnormal data is the superposition of these two phenomena before the camera comes to rest.According to the proposed method, after the 3-delta method identifes the abnormal data and RVM predicts the displacement at the corresponding time and fnally replaces the abnormal value with the predicted value.

Synchronous Monitoring of the Complete Bridge
Defection.Te proposed method was used to measure the vibration response of seven measurement points uniformly distributed over the complete bridge.During the nighttime, a moving load test was performed under a VIV event of the bridge.Te camera was mounted by the riverside and focused on the LED targets of the bridge, as shown in Figure 10(a).Te camera elevation was approximately α � 7 °.A lens with a focal length of 75 mm was used.Te distance between adjacent lights was 36 m.For the evenly distributed target points, the object distance L could be calculated using a curve-ftting method [38] based on the center detection results of the LEDs (Figure 10        is related to each other in both space and time, an RVM-based data self-diagnosis method for SHM was introduced.Te efect of the proposed method was verifed by manipulating the data of a vibration test artifcially.
Te proposed displacement method was applied to a long-span bridge.Te measurement accuracy is verifed by comparing it with a high-precision microwave radar.Especially, the environmental interference such as targets being obscured by ships and cameras being accidently shaken is solved well in the real bridge application.Tis study is of great signifcance for long-term vibration monitoring of long-span bridges.

2. 1 . 1 .
Te Principles of Weighted Center Detection Algorithm Based on Machine Learning.Te center location algorithm is optimized considering the efects of atmospheric turbulence on the target imaging.In general, turbulence leads to the blurring of the target boundary in the image, thereby changing the ECW.To address this problem, an adaptive window selection strategy based on energy accumulation is proposed.Furthermore, the deterioration of the image 2 Structural Control and Health Monitoring quality, especially the distortion, decreases the center location accuracy of a single algorithm.To address this problem, a weighted center location method based on two typical centroid algorithms is developed.Te weight values are predicted through LS-SVM.Te details of these methods are presented in the following text:

2 Figure 1 :
Figure 1: Framework of structure displacement measurement method for scenarios involving severe environmental interference.

Figure 3 :
Figure 3: Measurement results for the outdoor test.(a) Center location results of the static target obtained using diferent algorithms.(b) Displacement measurement results of the moving target obtained using diferent algorithms.

3. 2 .
Abnormal Data Correction.As shown in Figure8(a), when a camera was set up under a bridge, the feld of view may be blocked by passing ships.Terefore, test was performed to evaluate the abnormal data correction capabilities of the proposed method.Both the left and right drainage holes in each section were monitored.Only the 4/8 span was not disturbed by occlusion, and fection time history is shown in Figure8(b).Te other sections were disturbed, leading to missing data.Two cases

Figure 5 :Figure 6 :
Figure 5: Measurement results for the impact test.(a) Test confguration.(b) Original defection data of all measurement points.(c) Abnormal data identifcation and correction for measurement point P4 (the confdence is 95%).(d) Comparison of the predicted and measured data for points P3 and P4.
(b)).Te object distance L and scale factor of the control sections are

Figure 7 :Figure 8 :
Figure 7: Results of the accuracy evaluation test for the proposed optical method.(a) Set up of radar and camera.(b) Captured image and centroid positioning of the target.(c) Comparison of displacement measured by the radar and optical methods.

Figure 9 :
Figure 9: Abnormal data caused by camera shaking and machined learning-based correction results.

Figure 10 :
Figure 10: Measurement at night.(a) Captured image.(b) Centroid of the LED target.

Figure 11 .
Te visonbased measuring results were processed through the meanshift function.Terefore, the ice-blue line indicates the coupling response of vortex-induced vibration and moving trucks, and the red curve represents the defection infuence line caused by moving trucks.Finally, the dominant frequency of the bridge during this period was 0.228 Hz, corresponding to the third vibration mode of the bridge.

Table 1 :
Scale factor calibration results for seven target points.

Table 1 .
Owing to the limited space, part of the measurement results is shown in