Reproducibility Experimentation among Computer-Aided Inspection Software from a Single Point Cloud

The ISO GPS and ASME Y14.5 standards have deﬁned dimensional and geometrical tolerance as a way to express the limits of surface part variations with respect to nominal model surfaces. A quality-control process using a measuring device veriﬁes the conformity of the parts to these tolerances. To convert the control measurement points as captured by a device such as a coordinate measurement machine (CMM) or noncontact scan, it is necessary to select the appropriate algorithm (e


Introduction
ese days, 3D computer-aided design software provides various options to fit the measured points to a XYZ coordinate system.Most software features a selection of different "fitting algorithms" that adjust the measured points in various methods.Metrologies have the flexibility to fit a part in multiple ways using postprocessing software.e performance of a measurement system is an estimate of a combination of various measurement errors (random and systematic) that include hardware (equipment error), software (algorithmic error), and operators.Both the software and operator can still strongly correlate.In this paper, algorithmic errors were focused.A novel benchmark model is proposed in this study to compare the fitting algorithms' performance in different CAI software programs.According to a measurement system analysis (MSA) approach, it is strictly a reproducibility study.e purpose of inspecting is to verify compliance with the design requirements.
To keep track of the process, it is helpful to understand the error levels on the important features.In addition, the inspection report must be useful and easily interpretable to make the necessary corrections if the required tolerances are not respected.Form testing machines allow to measure points to form the profile and estimate circularity errors.Usually, all these machines provided many options for evaluating the circularity errors.erefore, it is needed to make a decision about the reference form to use for a particular application.is paper aims to analyze algorithmic errors.A quick review of the available literature on this subject finds that the impact of a measurement strategy (amount of data) and software used (including algorithmic and filtering outliers) is a surprisingly underexplored topic.Recent work in test of fitting algorithms (maximum inscribed, least square, minimum circumscribed, and Chebyshev) indicates that serious problems can exist in the current commercial software packages.

Background
Computational metrology includes the implementation of robust algorithms to adjust and achieve many computations on discrete data collected by measuring machines.e computational coordinate metrology is the most pertinent to validate standards that conform to tolerance specifications (see Figure 1).Now many journals consider computational metrology as a separate topic of interest.Computationally, fitting problem is an optimization problem: where n is the number of points and d i is the shortest distance between measurements data and the reference form.If p � 2 the fitting problem is the least square fitting, and if p ⟶ ∞, l∞ fitting is the Chebyshev or Minimax fitting.
Generally, the fitting problem consists to select an appropriate algorithm to adjust a geometric form (e.g., plane, cylinder, and circle) to data points collected from the inspection of a manufactured part.e perfect form estimation is obtained by a fitting is called a reference feature or substitute feature.e recent publication of ISO 14405-1: 2016 determines size as a fundamental geometric descriptor.It also defines a new set of modifier tools for size [1,2]; Jbira et al. [3] proposed a methodology to evaluate the algorithmic error of modifiers in the ISO 14405-1 [4] standard.ey demonstrated that the noise measurement affects the ISO 14405-1 modifier.
Various methods have focused on the problem of fitting surfaces applied to many fields.us, in the absence of recommendations by the ISO about the method adopted to terminate roundness [4,5], the estimation of circularity errors remains a challenge for the improvement of algorithms to calculate the best center.
In this section, some of the algorithms cited in the stateof-the-art  will be briefly described, particularly in the case of geometric elements of circular, cylindrical, and spherical forms.In [6], a reference circle is calculated from the input data in order to minimize the deviation between actual and reference circles.One of the methods to find this circle is a Voronoi diagram used to determine the minimum circumscribed circle (MCC) or maximum inscribed circle (MIC).To obtain a MCC, the points of the diagram that lie farthest from the center of the diagram are used.e MIC has a center on the Voronoi vertex or on the Voronoi edge.
e distance between the circle center and the convex vertex represents the radius.e Delaunay triangulation is used to calculate the convex and Voronoi diagram.In [9], the roundness error from the input 3D points is evaluated using four methods: minimum zone circle (MZC), least squares circle (LSC), MCC, and MIC.For the MZC method, the roundness error is measured using two concentric circles.For the LSC method, it is estimated inside the profile by minimizing the sum of radial squares between the circle and the profile.Using the center of the LSC, a circumscribed circle and an inscribed circle are obtained.e circularity error (out of roundness) value is the radial separation.e MIC method is used to fit the largest circle inside the profile.In the case of a MCC, a center is calculated by finding the circle that has the smallest radius containing the points of the circle.e circularity error is the difference between the radiuses of the inscribed circle drawn using the found center and the circumscribed circle.Tran et al. [12] proposed an algorithm to fit cylinders and to approximate the parameters given by 3D point cloud.e first step is to compute the normal vector of the points.Using curvature information, potential points that could form the cylinder are identified.
ese points should be updated in a fitting process to check all remaining points that belong to a cylinder.en, the mean shift clustering method is applied to find the approximated parameters of the valid cylinder.is method is validated using different models featuring various noise and outlier levels.In 2013, Srinivasan et al. [13] proposed a solution to the problems of weighted total least squares fitting of lines, planes, and parallel planes.ey demonstrated the need for these algorithms, which is relevant to newer tolerancing standards and instrumentation.Journal of Control Science and Engineering In 2017, Saval-Calvoa et al. [23] presented a novel method based on a random sample consensus (RANSAC).
is method estimates multiple planes from a point cloud with noise.e proposed method based on two steps: the first step is to divide the data into planar faces, and in the second step, the plane models are approximated using the multiconstraint RANSAC method.ey tested the proposed method with existing methods.
In [14], the authors compared different algorithms to facilitate the choice of the adequate execution method for MCC, MIC, and MZC in order to calculate roundness errors.In addition, they used a new geometric concept based on reflecting a mapping technique to assess roundness errors.ey proposed a selected benchmark of algorithms in the literature in order to provide the optimal execution method.It was concluded that no single algorithm provides the best solution.Geometric primitive reconstruction [6,[15][16][17] is an important problem in the field of computer-aided design (CAD).
Goch and Lübke [16] proposed a new algorithm to approximate geometry elements using Gauss and Chebyshev criteria.Furthermore, Chaperon and Goulette [18] proposed an algorithm to extract a cylinder from unorganized 3D points.e two main steps of their method were the extraction of the constrained plane in the Gaussian image to define the direction of a possible cylinder and extraction of the cylinder of known directions from the set of 3D points.ey validated their approach by testing the extraction of pipe objects in industrial environments.In the context of pipeline plant detection and reconstruction, the huge number of points makes the problem extremely difficult.Traditional methods of detection cannot be applied directly because of the high complexity.Moreover, in [19], the authors developed a robust algorithm based on a principal component analysis (PCA) approach to fit cylinders given a set of 3D points.ey validated their method using artificial and real point clouds.Most cylinder approximation methods focus on full data.However, the point cloud data that they obtained through laser scanning was incomplete and contained outliers.Nurunnabi et al. [20] described the problem of circle fitting for complete and incomplete data sets with outliers.ey proposed a robust approach for circle fitting which had to merge two algorithms: PCA and robust regression.
e experimental results confirmed the robustness of the proposed approach with a different percentage of tolerance of clustered outliers.ey compared the proposed approach with another exciting method.Guo and Yang [21] proposed a new procedure for circle fitting.ey used Taubin's approach to compute the center and radius, and then they identified and removed the outliers by calculating the geometric distances given point cloud to form the adequate circle.eir experiments demonstrated that the iterative procedure could resist against the effect of outliers.More specifically in the case of geometric elements of plane form, Deschaud and Goulette [22] proposed an accurate algorithm to extract planes in noisy point clouds using filtered normal and voxel growing.e first step is the estimation of the better normal at the data points.e second step consists of computing a score of local planes, and then they apply the growing voxels.Finally, they evaluated the proposed algorithm on different number of points and compared it with existing algorithms.e presented method has a linear algorithmic complexity and it is able to detect large and small planes in very large data sets.In 2017, Nguyen et al. [24] presented a comparative study of the least square plane fitting algorithms with different segmentation methods (e.g., RANSAC, RGPL, Cabo, and RDPCA).ey validated the study by two real point clouds collected by a Dynascan S250 scan system.e results demonstrated that the RGPL method gives the best results for planar surface extraction in moving least squares (MLS).
In 2018, Marriott et al. [25] presented an unsupervised extraction planar method.ey proposed to adjust the data with a piecewise-linear Gaussian mixture regression model whose components were skewed over planes.In [28], the problem of fitting full and half-geometrical primitives (e.g., circular, spherical, and cylindrical) is addressed.e authors use the Levenberg-Marquardt (LM) method to approximate these geometries.
ey also proved that using the chaos optimization method improves the initial algorithm estimation.In fact, the chaos-LM algorithm provides efficient results even when the input data points are incomplete and noisy.
e literature review demonstrates the importance of geometric elements of circular, cylindrical, plane, and spherical forms fitting problem to industrial designers and inspectors.
erefore, the quality of the deformed geometrical is evaluated by the fitting algorithms.e literature supplied with different algorithms for the cylinder fitting is used to quantify roundness error.Despite different algorithms being proposed, the optimal solution has not been produced.e designers should carefully select the appropriated algorithm.rough this literature review of current software programs and algorithms, many inaccuracies were found.

Experimental Protocol
is paper is interested in the reproducibility estimation between three computer-aided inspection software programs that originate from the same point clouds.According to the industrial guide put out by the Automotive Industry Action Group [AIAG], reproducibility traditionally refers to the variation in average measurements made by different operators using the same gauge and parts.
Typically, the term is defined as the average measurement made by different appraisers using the same measuring instrument when measuring identical characteristics on the same part.In our study, one part was used (same drawings and same specifications) with the same measurement point clouds and appraiser but three inspections software programs were used.For each software, all proposed options by each software are tested (e.g., Best-fit and Minimax).erefore, variability expresses the algorithmic differences between software programs.Voluntarily, we have not publicized the names to maintain certain confidentiality.e study was not conducted to provide a complete judgment on software.It has the focus only to estimate the reproducibility Journal of Control Science and Engineering of the used algorithms.eir choice of these three was dictated by the following point: they are commercial software widely used in the industry (>100 costumers for each software).
A piece is designed using basic feature geometries (e.g., planes and cylinders).Dimensional and geometrical requirements are presented in Figure 2. One part is made out of aluminum and the piece is digitized by a HandySCAN portable 3D scanner (with a resolution of 0.05 mm and an accuracy of up to 0.04 mm). is part is to be used for tting performance evaluation and inspection algorithms while considering dimensional and geometrical tolerances.In this paper, the standard ASME GD&T Y14.5-2009 [27] was used.In total, ve di erent features with dimensional and geometrical tolerances were selected on the test pieces (plane, circle, and oblong hole).Only one operator was involved in the study.ree software programs were tested with all of their options.
e reproducibility in this study (expressed as a standard uncertainty uAV) represents variations due to the di erence between algorithms and how the software deals with outliers and registration t.
Alignment tools can de ne a part reference frame if you know the nominal location of point-reducible features on the part (e.g., points, circles, and spheres).Before performing a datum alignment, the operator has to provide a close initial part reference frame.e operator can use any available part alignment tool (e.g., CAD-cloud point alignment with the ICP algorithm [20] or 3-2-1 alignment) to perform this initial part alignment.e geometric plane line point (PLP) alignment enables us to align a part to the CAD model using a plane (primary datum), a line (secondary datum), and a point (tertiary datum) to create the nominal coordinate reference system.In this paper, for each software, the operator has systematically carried out the PLP alignment.

Methodology Analysis
In the case of circle features, the results are the measured diameter, the localization of the center, and the roundness error.In the case of plane features, the result is only the atness error (Figure 3).As mentioned, three software programs have been tested with all options (Table 1).
e alignment performed is a PLP-type alignment using the measured ABC datum using the least squares algorithm.For the CPU time, and for all tested options, below one second in all cases on a PC with a 2.4 GHz Intel Core i3 processor and a physical memory of 4.0 GB manual intervention time: friendly and very similar in the three cases.

Results
In the measured part, two circles called circle #1 and circle #2 have been extracted, and an oblong hole for evaluating each option for each one of the three software programs.In Table 1: Options for three software programs.
Described options Software Best-fit: an option button that specifies the use of the usual "Best-Fit" fit algorithm that performs a least squares approach.Min: an option that specifies the use of the minimum fit algorithm.A primitive Min is the largest primitive that does not encompass any element.
Max: an option that specifies the use of the maximum fit algorithm.A Max primitive is the smallest primitive that contains all the elements.
Least square: to create a circle that best fits the probed points.e circle is calculated by minimizing the sum of the squares of the gaps between the circle and at each point probed.
Maximum inscribed: to create the largest circle that fits in the probed points.
Circumscribed minimum: to create the smallest circle that contains all the probed points.
Minimax: to create the circle by averaging between the maximum inscribed circle and the circumscribed minimum, which have the same center (minimizing a circularity error).
Best-fit: find geometry using the least squares method from contours or surfaces.Min: find the maximum circumscribed geometry using the minimum separation method from the contours or surfaces.Used to find the maximum circumscribed.
Max: find the minimum circumscribed geometry using the minimum separation method from the contours or surfaces.Used to find the minimum circumscribed geometry that encounters the furthest point to the positive direction of the normal.
addition, two surface planes have been extracted called plane A and plane D to calculate the flatness found in each software.

Analysis of Circle #1.
In the first test, the deviation of the center was computed on the circle in the x and y directions (δ x , δ y ) and the measured diameter (Ø) was calculated, the diameter deviation (δ Ø � 20− Ø) and the roundness error of the circle #1 (Table 2).Results with the least square option are shown in bold, results with the Min option are shown with an underscore, and results with the max option are shown with a double underscore and, if applied, results with the Minmax option are shown in italics.According to Table 2, CAI software #1 and #3 offer three adjustment algorithms for circle #1 (least square circle (LSC), minimum circumscribed circle (MCC), and maximum inscribed circle (MIC)).
Four algorithms are available in the CAI software #2 (LSC, MIC, MCC, and Minimax).
e LSC algorithm provides the same value of measured diameters for the three software programs (19.877 mm).It is an expected result, as the LSC is a deterministic algorithm.Software #1 and #2 with the MCC algorithm provide the same values of measured diameters (20.054 mm), but software #3 gives a bit of a different value (20.059 mm).e range (r) between results is equal to 0.005 mm.Again, the MIC algorithm gives the same values for the software #1 and #2 (19.755 mm), but software #3 gives a little different value (19.745 mm), r � 0.01 mm.
Finally, the Minimax algorithm is available only by software #2; the value of diameter is 19.903 mm.According to Figure 4(a), software #1 and #3 provide the same circularity values for all the methods, and software #2 demonstrates different results for each method used.In the case of size in Figure 5(b), the LSC shows the same size for the three software programs despite the fact that the Max and Min algorithms display different results.3, results and conclusions are very similar to Table 1 (circle #1).e LSC algorithm provides the same value of measured diameters for software #1 and #2 (19.874 mm).In addition, software #1 and #2 with an MCC algorithm provide the same values of measured diameters (20.054 mm), but software #3 shows a bit of a different value (20.054 mm).e range (r) between results is equal to 0.005 mm.Again, the MIC algorithm shows the same values for software #1 and #2 (19.760 mm), but software #3 provides a slightly different value (19.721 mm) with r � 0.039 mm.Finally, with the Minimax algorithm, available only in software #2, the diameter is equal to 19.907 mm.According to Figure 4(a), software #1 and #3 provide the same circularity values for all methods (Max, Min, Best-fit, and Minimax), and software #2 provides different results for each method.Regarding the size in Figure 4(b), the LSC gives the same size for the three software programs, despite the Max and Min algorithms displaying different results.

Analysis of Circle #2. According to Table
According to the circle #1 and circle #2 results, in the case of diameter measurements, there is a small variation.However, in the case of more complex GD&T, there are greater variations (circularity cases).

Analysis of the Oblong Hole.
In the second test, the deviation was computed on the hole location on the axis x (δ x ), and the measured length and width (β, c), the length deviation (δ β � 22.666 − β), and the width deviation (δ c �13.333 − c) were calculated (Table 4).According to Table 4, CAI software #1 and #3 offers three adjustment algorithms for oblong holes (LSC, MIC, and MCC).Four algorithms are available in CAI software #2 (LSC, MIC, MCC, and Minimax).e LSC algorithm displays different values for the measured width and length on software #1, #2, and #3 (see Figures 6(a) and 6(b)) (13.139 mm, 13.152 mm, and 13.151 mm). is is abnormal, since the LSC is a deterministic algorithm that normally should provide the same value of measured width.According to the Figure 6(b) software #1 and #3 with a MCC algorithm displays the same value of measured width (13.320 mm), but software #2 shows a different value (13.316 mm).
e range between results is equal to 0.004 mm.Again, the MIC algorithm gives different values of measured width for software #1, #2, and #3 (12.989 mm, 13.048 mm, and 13.320 mm).Finally, the Minimax algorithm is available only by software #2, where the value of the width is equal to 13.164 mm.

Analysis of Plane A.
In the third test, the flatness was computed of two planes, Plane A and Plane D (Table 5).Results with the least square option are shown in bold; results with the Min option are shown with an underscore, and the max option is displayed using a double underscore (Figure 7).
According to Table 5, CAI software #1 and #3 offer three adjustment algorithms for Plane A, the least square plane (LSP), minimum plane (MP), and maximum plane (MIP).
e CAI software #2 only offers one algorithm (LSP) to fit Plane A. Only the best fit is available.e software #1 gives the same flatness value (0.107 mm) for the three algorithms (LSP, MC, and MIP).In the case of software #2, only the best fit is available (0.119 mm).6, the results are similar to Plane A. software #1 gives the same flatness value (0.070 mm) for the three algorithms (LSP, MC, and MIP).In the case of software #2, only the best fit is available (0.0312 mm), but software #3 gives different flatness values for each of the three algorithms (LSP, MC, and MIP) (Figure 8).

Analysis of Plane D. According to Table
In this paper, an investigation of reproducibility estimation was presented.Similar point clouds, operators, CAD, and many fitting algorithms with different shapes (circle, plane, and oblong hole) and features (size, circularity, and flatness) were evaluated.
e achieved experiences demonstrated that there exists a different variation between the three CAI software programs.Using the example of a LS 6 Journal of Control Science and Engineering algorithm, the deterministic algorithm and the analytical solution are well known.e aforementioned results demonstrate that there a small (and great) variation between CAI software programs.ese deviations can be due to filtering and smoothing operations, a reduction in the point density, the treatment of outliers, a calibration of the point cloud to the CAD model, and optimization operations when the tolerances allow 10 for degrees of freedom.All these variation sources have an effect on variations that we call "algorithmic errors."Journal of Control Science and Engineering

Conclusion
e adjustment of measured point cloud is an essential step in measuring machine software.A correct application of adjustment algorithms can be employed to a comparative tool not just a control tool.Some fitting algorithms optimize the conformance of geometric tolerances and can be applied in the manufacturing analysis.It is important that the correct algorithm be applied in the specific goal; otherwise, an error will occur and the results will not be optimal.An    experimental investigation among three inspection software programs to evaluate algorithmic errors was proposed by the same operator, the same CAD, and the point cloud.is experience shows that there are indeed variations (large and small) among the three software programs.e proposed methodology is set to be an important tool to help industrial designers and inspectors select the appropriate algorithm for size and form evaluation.As such, it is the responsibility of the designer and inspector to carefully select the corresponding algorithm in order to avoid errors.erefore, each specification method has its own field application (functional requirement).Once the specification method for a special application has been assigned, it becomes important to select the most optimal specifications.e benchmark and experimental case studies demonstrate the influence of the algorithm choice.
Capture a cloud of points according to any reference systemFilter and reduceProcess to extract the simulated geometriesCompare with the nominal model to extract the deviations i, j, and k according to the degrees of blocking

Figure 2 :Figure 3 :
Figure 2: GD&T of the sample part with numbering items.

Figure 8 :
Figure 8: Result of flatness analysis in Plane D.

Table 4 :
Results for the inspection of the oblong hole (width � 13.333 mm and length � 22.666 mm).

Table 5 :
Results for the inspection of Plane A.

Table 6 :
Results of inspection of Plane D.