A Multifrequency Heterodyne Phase Error Compensation Method for 3D Reconstruction

. In view of the problem of “ jumping points ” in the phase unwrapping of the multifrequency heterodyne principle, this paper proposes a novel method to improve the multifrequency heterodyne. By solving the root-mean-square error of the original frequency function, it includes the relationship between the error and the adjacent phase in the condition of constrained phase unwrapping, and it compensates phase ±2 π of the skip point. To ensure the accuracy of phase unwrapping, the function with a “ jump point ” after each phase unwrapping and the absolute phase curve of the principal value function are used to establish the threshold judgment model of the least square method, and the initial phase unwrapping of the principal value function with di ﬀ erent frequencies is carried out continuously. The simulation analysis of phase compensation with the four-step phase-shifting method shows that the error is reduced to 36% under the set environment. The experimental result of 3D reconstruction by measuring the ﬂ atness of the plate shows that the error decreases by 41% after phase compensation compared with before phase compensation. The three-dimensional reconstruction experiment of pitch measurement with a nut shows that the nut after phase compensation is smooth without noise, and the pitch error is 0.033mm, which veri ﬁ ed the method is workable and e ﬀ ective.


Introduction
Based on triangulation, the structured light threedimensional measurement method has been widely used in various fields due to its high-precision and noncontact measurement [1][2][3]. For example, the measurement of threedimensional morphology is applied to reverse engineering, biomedical treatment, heritage acquisition, architecture and computer animation, etc. [4][5][6]. To get a high-precision three-dimensional profile, we need to unwrap the phase accurately. The four-step phase-shifting method is adopted in the 3D reconstruction process. The arctangent function is involved in the derivation process, so the phase is wrapped between ½−π, π. Therefore, the wrapped phase needs to be unwrapped continuously in the global field [7].
At present, the principle of multifrequency heterodyne is one of the widely used methods. Multifrequency heterodyne is a time phase unwrapping method [8,9]. The calculation of phase unwrapping at different frequencies is independent in the process of phase unwrapping. Therefore, the algorithm can better suppress interference. However, there are some problems in phase unwrapping of multifrequency heterodyne, such as jumping error [10]. Jiang et al. [11] proposed a series of constraints to compensate the phase of multifrequency heterodyne. Zhang et al. proposed a look-up table method to build a preestablished storage table between the input gray and the output gray to compensate for the nonlinear phase error in the projection and acquisition system, but this method took a lot of time to calculate [12]. Cai et al. used the Hilbert algorithm to compensate the reverse error, but this method was unable to measure the profile of complex objects, and the measurement speed was slow [13].
In order to solve the problem of jumping error in the principle of multifrequency heterodyne, this paper proposes a novel method to improve multifrequency heterodyne [14][15][16]. The innovation of this paper is described below. The root-mean-square error of the principal value function with different frequencies is solved, and the phase unwrapping is carried out with this error as the corresponding constraint condition. Finally, the least square threshold judgment model is established for the function with a "jump point" after each phase unwrapping and the absolute phase curve of the principal value function. According to the threshold value, whether to enter the next iteration is judged. If the error is greater than the set value, then the error is taken as the constraint condition for phase unwrapping again until the threshold condition is met.
In this way, the jumping error is eliminated when the phase of the initial principal value function of different frequencies is unwrapped. Through the simulation of phase unwrapping of sinusoidal gratings with three different frequencies of noise, there is a "jump point" in phase unwrapping before phase compensation, and a continuous phase is got after phase compensation; in the experimental part, the method is applied to 3D profile reconstruction, and the experimental results show that the 3D profile after phase compensation is more smooth without a jump point.
In this paper, the contributions of the phase error compensation method are as follows. Firstly, the constraints of the phase error compensation method in this paper are more concise and easy to calculate. Secondly, this paper compensates the wrapping phase of various frequencies, respectively, and the phase becomes more continuous after phase unwrapping. Thirdly, compared with single-frequency wrapping phase error compensation, the multifrequency heterodyne method is more fault-tolerant and accurate in 3D reconstruction of objects because of the independent phase unwrapping process of various frequencies. The second part of this paper involves the description of the principle of multifrequency heterodyne phase unwrapping; the third part is a novel method of phase compensation; we divide the fourth part into simulation verification of this method and experimental verification of the feasibility of this method; the fifth part is the summary.

The Principle of the Multifrequency Heterodyne Method
The principle of the heterodyne method is to stack the principal values of sine grating functions with different frequencies and finally get the entire image with the frequency of 1 Hz, where absolute phase ϕðx, yÞ of its expression is [17] where kðx, yÞ is an integer. φðx, yÞ is the principal value function of the phase of the sine grating functions. The key to phase unwrapping is to find the value of kðx, yÞ. Different frequencies of λ 1 and λ 2 correspond to phase function φ 1 ðx, yÞ and φ 2 ðx, yÞ, respectively. The wrapped phase maps of φ 1 and φ 2 ðx, yÞ will be unwrapped, where the expression of absolute phase is [18] ϕ 1 = 2πk 1 + φ 1 , The schematic diagram of multifrequency heterodyne shows phase unwrapping in Figure 1.
The fringe series n satisfy λ 1 n 1 = λ 2 n 2 , The integer fringe numbers of the fringe patterns of different frequencies on the same point satisfy The heterodyne phase value is From Equations (2)-(5), we get The frequency value after superposition of different frequencies is From Equations (6) and (7), the absolute phase value of fringe gratings with different frequencies is From Equations (2) and (8), the key value kðx, yÞ is Finally, the absolute phase value ϕ 1 can be got [19]: In fact, there will be some "jumping points" in the process of multifrequency heterodyne phase unwrapping. Therefore, it is necessary to compensate the errors to improve the precision of phase unwrapping.

Multifrequency Heterodyne Phase Error Compensation
There are three kinds of gratings with different frequencies.
The projector projects the gratings with fringe periods of 70, 64, and 59. After collecting by a CCD camera, they carry the phase unwrapping processing out. Finally, the noise problem shown in the red circle area in Figure 2 will appear in the result of global phase unwrapping. After analysis, it is a jump error. Therefore, it is necessary to compensate the phase error before 3D reconstruction of the object, so as to avoid "jumping points" in the reconstructed model. With the multifrequency heterodyne method, there are some problems such as jumping error. First, calculate the root-mean-square error (RMSE) after the phase unwrapping of sine grating functions with different frequencies. According to the principle of multifrequency heterodyne in the upper section and the formula below, the root-mean-square error (RMSE) can be got: where X defined as where ϕ i is the absolute phase value of a certain frequency grating, i.e., the value after phase unwrapping of fringes; λ i is the frequency of the fringes; and i is the total frequency. The wrapped phase error is analyzed, and the error is a Gaussian distribution, and the root-mean-square error (RMSE) is calculated as σ φ , where σ φ1 , σ φ2 , σ φ3 are the principal value function φ 1 , φ 2 , and φ 3 Gaussian standard deviations. To deal with the " jump point" error, add the constraint conditions to the original algorithm. The constraints are where φ 12 ′ ðx, yÞ is the wrapped phase after dual-frequency correction, computing σ φ1 and σ φ2 over φ 1 ðx, yÞ and φ 2 ðx, yÞ, and select σ φ = max fσ φ1 , σ φ2 g. In this way, avoid the jumping error caused by the noise in the wrapped phase. The phase unwrapping function after correction is For the heterodyne function φ 23 ðx, yÞ, the constraints are 3 Journal of Sensors where φ 23 ′ ðx, yÞ is the wrapped phase after dual-frequency correction. Two wrapped phase maps of φ 2 ðx, yÞ and φ 3 ðx, yÞ have Gaussian standard deviation σ φ2 and σ φ3 , respectively, computing σ φ2 and σ φ3 over φ 2 ðx, yÞ and φ 3 ðx, yÞ, and select σ φ = max fσ φ2 , σ φ3 g. φ 123 ′ ðx, yÞ is using a similar method. Compensation for jumping errors in the wrapped phase map φ 123 ðx, yÞ is computed by where φ 123 ′ ðx, yÞ is the wrapped phase after compensation. At this moment, the value of σ φ is determined by where ϕ 12 ′ ðx, yÞ and ϕ 23 ′ ðx, yÞ are the phase unwrapping function after correction. After phase compensation is completed, the model relationship between the compensated phase unwrapping mean X ′ * λ and the compensated frequency phase unwinding phase function value ϕ i ′ is established. The model relation is given by formula (18). After phase compensation, there may be a "jump point" function and its absolute phase curve of its principal value function to establish the least square method threshold judgment model and determine whether the error is less than the set threshold value.
If not, return the error to Equations (13)-(17) as the condition of constrained phase unwrapping and then carry out phase compensation until the threshold condition is met. The judgment equation is The multifrequency heterodyne completes phase compensation.

Simulation and Experimental Analysis
4.1. Simulation Analysis of 3D Reconstruction before and after Phase Error Compensation. Three sinusoidal gratings with different frequencies are selected for phase compensation. The three frequencies are 70, 64, and 59, respectively. The simulation environment is a nonideal sinusoidal grating with a resolution of 1024 * 768. In Figure 3, it can be seen that the environment noise before compensation will cause the "jump points" of the last phase unwrapping. First, the root-meansquare error (RMSE) of the noise is calculated as σ φ , which is the constraint condition of Equations (13) and (15) in this after the phase unwrapping. It can be seen that the function shows "jumping points" because of noise, and the pixel jump reaches 30-40 pixels at the beginning and end. After the algorithm proposed in this paper is used for compensation, as shown in Figures 3(c) and 3(d), the pixel jump drops to 5 pixels after the phase of the blue curve is unwrapped, indicating that the phase is compensated. Figures 4(a), 4(b), and 4(c), respectively, correspond to the phase compensation front curve of functions φ 1 , φ 2 , and φ 3 . The initial noise also exists in these three principal value functions. It is transferred to the principal value function after superposition frequency by the heterodyne method. The basic steps of the heterodyne method are as follows: step 1: get the phase value of the principal value function first; step 2: get an entire image with the phase frequency of 1 Hz by using the principle of the heterodyne method; and step 3: reverse ϕ i = 2πk i + φ i . According to the heterodyne method, the frequency is got by superposition, and the sequence of phase unwrapping is the principal value functions φ 12 , φ 23 , φ 1 , φ 2 , and φ 3 , while the final purpose is to obtain the last three different frequency principal value functions, the continuous phase unwrapping of φ 1 , φ 2 , and φ 3 . In the process of phase unwrapping, when the errors of   Journal of Sensors the conditions of −2σ φ < φ 1 ðx, yÞ − φ 2 ðx, yÞ < 0 or 0 < φ 1 ðx, yÞ − φ 2 ðx, yÞ < 2σ φ , which is not enough to affect the phase unwrapping of φ 1 , φ 2 , and φ 3 , so the phase unwrapping no longer has a jump. In this paper, a decision condition of the least square method is added to the method. The error is still large, but after many iterations, it is found that the phase unwrapping of φ 1 , φ 2 , and φ 3 in Figures 4(d), 4(e), 4(f), respectively, does not have jumping points.
The experimental results show that the errors of the principal value function before phase compensation (phase wrapping) and after phase compensation (phase unwrapping) can be ignored, and the errors of the principal value function of heterodyne can be reduced by 36%. Table 1 shows the calculated standard deviation before and after the phase compensation, which shows that the algorithm in this paper has a significant improvement on the accuracy.   Journal of Sensors In order to further prove the accuracy and better visualization effect of the method proposed in this paper, combined with MATLAB, the hemispherical R = 50 radius is simulated by the four-step phase-shifting method, and the grating resolution ratio is 256 × 256. Gaussian noise with standard deviation of 0.01 is added. Figures 5(a-f) display the threedimensional and two-dimensional images of the phase unwrapping, respectively. Through the comparison of the two sets of data, we can see the recovery morphology of the hemisphere after the phase compensation, which shows that the compensation method proposed in this paper is accurate and workable.

Experimental
Analysis of 3D Reconstruction before and after Phase Error Compensation. In the experiment, the flat plate and nut are scanned for verification. Figure 6  scanning and collecting the plate data. The experimental platform mainly includes a screw rod, a two-dimensional storage platform, a projector and camera installed on the screw rod, and a two-dimensional storage platform controlled by a step motor, which can move along the horizontal X-axis and Y-axis. Figure 6(b) is to collect the nut data. The place where the nut of the experimental platform is placed is a two-freedom-degree rotating platform, which can swing the arm forward and backward, and the fixed axis can rotate in a circle. In the experiment, the projector resolution ratio is 1024 × 768 pixels, the camera resolution ratio is 3078 × 2048 pixels, and the number of fringe periods is 64, 60, and 57. The upper computer of the two platforms is a Windows 10 system, and the CPU is Intel i5. The camera collects the data of the flat plate and the nut, respectively, gets the data, and compares the effect of phase compensation and noncompensation for the data. First, collect the plate data, and the number of effective points collected is64 × 716. In this paper, 3D reconstruction is carried out by the four-step phase-shifting method. Figure 7(a) shows the three-dimensional image before phase compensation, in which there are jump points when the pixel is close to 64 pixels. Figure 7(b) carries out phase compensation in this method, and it can be seen that it eliminates the "jump points." To show the more obvious change, Figure 7(c) shows the variation of the crosssection data, data before phase compensation (red), and data after phase compensation (blue); the front and back  Journal of Sensors error is reduced from the maximum error of 0.5 mm to 0.1 mm; and the error is reduced by 40%, verifying the feasibility of the phase compensation method.
To analyze the accuracy of the phase error compensation method proposed in this paper, the nuts which are widely used in industry are used for reconstruction. The pitch standard is easy to query and can be analyzed quantitatively. Therefore, the pitch value is taken as another standard to evaluate the accuracy of the phase error compensation method in this paper. The four-step phase-shifting method is used to collect nut data for 3D reconstruction. Figure 8 shows the reconstruction model of the nut before and after phase compensation.
By calculating the thread pitch of the nut (unit: mm) as the measurement standard, analyze the distance between ten adjacent threads of the screw, among which the specification of the screw is M12, and the standard pitch is 1.75 mm. For the acquired data, the distance between adjacent ten threads is recorded as shown in Table 2.
According to the table, Figures 8(a) and 8(b) show that the surface of the nut after compensation is flat and there is no jumping error. It shows that the novel method of multifrequency heterodyne phase error compensation proposed in this paper can effectively eliminate the "jump point" in the actual 3D reconstruction and ensure the high reconstruction accuracy.

Conclusion
To eliminate the jumping error in the process of phase unwrapping, this paper proposes a novel method of multifrequency heterodyne phase compensation, which solves the problem of "jumping points" in the actual situation. In this paper, the root-mean-square error (RMSE) of the principal value function with different frequencies after phase unwrap-ping is calculated first, and they include it in the constraint condition of phase compensation, to suppress the jump error. The least square threshold judgment condition is added to verify whether there is still a "jump point" with large error after compensation, which affects the global unwrapping of the last phase. If it is greater than the threshold value, it will carry a simple reiteration to meet the threshold requirements. In this paper, the simulation analysis shows that the error is reduced by 36% in the set environment, and the phase unwrapping after the error compensation can effectively eliminate the "jumping points" and get the continuous phase. The experimental verification of flatness measurement shows that the error after phase compensation is reduced by 41% compared with that before compensation, and the three-dimensional reconstruction of the nut can also achieve high accuracy. After the error compensation, the 3D reconstruction object surface becomes smoother and more continuous. Simulation and experiment show that the improved multifrequency heterodyne phase compensation method is workable.
The future direction of the work in the paper can be divided into the following points. First, it can be seen from the experimental data before and after the compensation of the plate that there is still room for correction of the error of the plate, and it is not clear whether the formula proposed in this paper is the best for the reconstruction error compensation. Second, this paper is to carry out 3D reconstruction of objects without mirror reflection. How to achieve highprecision 3D reconstruction of mirror reflection objects is still a problem to be solved.

Data Availability
The data used to support the findings of this study are available from the corresponding authors upon request.