SSF-Align: Point Cloud Registration Based on Statistical Shape Features with Manifold Metric

. As an important topic in 3D vision, the point cloud registration has been widely used in various applications, including location, reconstruction, and shape recognition. In this paper, we propose a new registration method for this topic, which utilizes statistical shape features (SSFs) and manifold metrics to estimate the transformation matrix. The SSFs are extracted to establish a compact representation for the original point cloud. Then, the representation is mapped into a manifold to reduce the in ﬂ uences of different scales and translations. Finally, the manifold metric is used to minimize the distance based on the compact representation and the pose can be estimated. The advantages of our method include robustness to nonuniform densities, insensitivity to missing parts, and better performance to handle large difference of poses. Experimental results show that our method achieves signi ﬁ cant improvements compared to the state of the art methods.


Introduction
Following the development of 3D scanning technology, 3D point clouds and related analysis have been widely used in practical scenarios.Comparing to the 2D images, 3D point clouds take complete geometric information with texture that supports more accurate quantitative analysis.As one important topic in this field, point cloud registration methods have been studied for decade which can be regarded as a fundamental issue in a series of 3D vision tasks.Related achievements have been successfully used in numerous commercial applications, including digital entertainment, intelligent healthcare, and remote sensing data processing.
The mainstream technological routes can be roughly divided into two categories: the global correspondence and the local matching.The global correspondence-based route is to search a transformation that satisfies minimize state under certain metric in a global view.It is robust to local random variations produced by the noise and nonuniform densities.However, the performance is limited by the selected metric.The situation of local optimization cannot be completely avoided.In addition, the missing parts in point clouds have significant impacts that cannot be ignored.The local matching route prefers to establish local region-based correspondence directly.It reflects the relationships of significant geometric features and does not require the complex searching operation in the global view.Therefore, it is not sensitive to the missing parts.Obviously, the drawback of the route is the dependence on the quality of local feature analysis.The noisy points and random distributions take unstable influences.
Recently, a new solution (KSS-ICP) [1] is proposed which can be regarded as a global correspondence scheme.The novelty of the solution is that the manifold metric is used to improve the accurate of the metric.It significantly reduces the influences of scaling, translation, and rotation by alignment while improving the robustness to defective parts.However, it still has some limitations: the lower computational efficiency, the lack of local features analysis, and the restricted metric.The KSS-ICP requires many times of comparisons to achieve final alignment.Even the GPU-based parallel acceleration is used, the efficiency is still lower than others (>1 min for a point cloud with more than 50k points).Due to the method does not consider the local features, it is still sensitive to the missing parts to a certain degree, which can not be solved by the Hausdorff distancebased metric.
In this paper, we propose a new registration scheme that combines statistical shape feature (SSF) analysis and manifold metric to align point clouds.Inspired by the KSS-ICP, we establish a compact representation for original point cloud, which reduces the influences of scaling and translation.At the same time, we extract the SSFs from the point cloud to achieve further simplification for the compact representation.The salient geometric features are kept.Based on the representation, we designed a new manifold metric to align point clouds.It is used to improve the performance of the local feature analysis.Finally, we implement the registration scheme that achieves a balance between global correspondence and local matching.The pipeline is shown in Figure 1.The contribution can be concluded as follows: (1) We present a compact representation for the raw point cloud, which reduces the influences of scaling and translation while considering SSFs.It takes less space and carries more salient geometric features, which is more suitable for the manifold metric.(2) We propose a new manifold metric for the alignment.Comparing to the traditional metrics, the proposed manifold metric has the property of shape feature invariant.It improves the performance of global correspondence by considering local features.(3) We report a comprehensive analysis for the proposed method in two classical datasets.It provides quantitative proofs to show the performance of our method.
The rest of the paper is organized as follows.In Section 2, we introduce related works for the point cloud registration.In Section 3, we introduce some fundamental details of the shape space and SSF.In Section 4, we explain the construction of compact representation for point clouds.In Section 5, we show the details of manifold metric for alignment.The experiments in Section 6 show the performance of our method.

Related Works
Global correspondence scheme estimates the transformation matrix based on the global-view metric.The representative methods are ICP [2] and its variants [1,[3][4][5].The ICP-based registration is to align point clouds according to the potential point-based distance.During the iterative correction, correspondences between points are estimated and the transformation matrix is achieved.However, the registration process is sensitive to the initial poses of point clouds.It makes the registration drop into the local optima with high probability.The variants are proposed to reduce the probability.Fast-ICP [5] utilized the Welsh function and Lie algebra form to represent the registration error and the transformation matrix, which greatly improved the accuracy and convergence speed of ICP algorithm.
As a well-known solution, GO-ICP [6] established a global searching strategy to search the potential solution with a BnB scheme [7].It can be regarded as a controllable exhaustive searching process.KSS-ICP [1] is a new solution that combines with ICP scheme and manifold metric on Kendall shape space.Influences produced by noisy points and different scales are reduced significantly.Following the development of deep learning frameworks, some researchers employ the related technologies to implement the registration.PointNetLK [8] creatively combined the classic Lucas-Kanade (LK) algorithm for the 2D images registration task and the well-known PointNet for 3D point clouds, which opened up a new path for the application of deep learning into 3D point cloud registration.Similarly, many solutions improve the efficiency of the registration with effective feature coding [9,10].
Local matching scheme is to search the point-based correspondence directly.The advantage is that the scheme can  [11][12][13].As a representative method, the fast point feature histograms (FPFH) [12] provide more robustness local shape feature that is benefited from the normal vector-based statistical analysis.Due to the dependency on local features, such methods are sensitive to noisy points and regions with nonuniform densities.Recent works attempt to use the deep neural networks to improve the robustness for feature analysis [14,15].Benefited from the prior knowledge learning, such frameworks can extract semantic feature to guide pointbased alignment.However, such feature coding process suffers for point clouds with complex similarity transformations, especially for the scale difference and nonuniform densities.
Recent researches become more focused on specific issues.For instance, a method based on geometric attention network was proposed to solve the partial point cloud registration [16].Qin et al. [17] proposed a solution which is both keypoint-free and RANSAC-free.Yang et al. [18] proposed a mutual voting method for ranking 3D correspondences, which can be directly utilized in the 3D point cloud registration.
Our registration method inherits advantages of the global correspondence scheme and combines the local feature analysis at the same time.Inspired by the KSS-ICP, we introduce the manifold metric to align point clouds with an efficient way.In the following parts, we discuss the implementation details.

Fundamental
3.1.Shape Space.For quantitative shape analysis between point cloud models, influences of different similarity transformations (shape-preserving transformations) should be removed.Following the basic requirement, Kendall [19] provided the prior work to measure 3D models on Kendall shape space.The Kendall shape space is a manifold space constructed by discrete shape form (point sequence).It is also a quotient space that removes influences of similarity transformations.For point cloud-based measurement, the Kendall shape space provides a manifold metric that is not affected by representation on Euclidean space with similarity transformations, including translations, scaling, and rotations.Such property can be used to align point clouds.If the metric on Kendall shape space is considered as optimization object, then the shape space can be regarded as a solution space to search optimal similarity transformations as the registration result.We provide a mathematical representation for Kendall shape space: where M is a manifold space that is constructed by discrete shape form with n 3D points, G represents similarity transformation group with translation T, scaling S, and rotation O. K s represents the Kendall shape space constructed by M=G, which is still a manifold and, at the same time, a quotient group space.For point clouds, once we define and implement the mapping from Euclidean space to Kendall shape space, the influence of similarity transformation can be removed.The mapping is represented as follows: where a is a discrete point sequence with point number nand T:S:O represent related normalization to remove influences of translation, scaling, and rotation.After the normalization, the similarity transformation cannot change the representation of a (for instance, TðaÞ ¼ Tðt:aÞ; t is to remove the position of a).The distance on Kendall shape space is a manifold metric that reflects the shape similarity between point clouds.The registration result can be achieved from the reverse operation of T:S:O.In Section 3, we will discuss an implementation of the normalization based on the Kendall theory.

Statistical Shape
Feature.The manifold metric on Kendall shape space represents the global matching between point clouds.It can be used to describe the alignment as the global view.Naturally, the drawback of the metric is that some local geometric features are lost.It takes a serious impact when the convex hull of the point cloud model has a significant symmetry structure, such as indoor scenes, transportation, and furniture.At this time, the local geometric information of the model will become important features that cannot be ignored.Therefore, the local geometric information should be considered in the registration.
To formulate the information, some shape operators are proposed to provide a quantitative representation as the local shape feature.Such operators describe local geometric details based on normal vectors or curvature values statistically.Representative operators include normal distributions transform and FPFH [12].However, such methods depend on the quality of normal estimation and the performance are reduced for noisy point cloud inevitably.Therefore, a reasonable SSF should not depend on normal vectors and robust to random disturbances of points.Mathematically, it can be expressed as follows: where p i is a point with normal vector n i ; SSFðp i Þ represents the required SSF, α and β are random disturbances, and ε is an acceptable error.The SSF represents the mapping distance based on the fitting plane in a local region.The interference caused by random noise on the mapping distance of a point is significantly reduced when constrained by the fitting plane.In the following parts, we provide the implementation details according to the requirements of SSF.
Journal of Sensors

Compact Representation with SSF
Generally, the scanned point cloud takes redundant points and random outliers that reduce the computation efficiency.
To improve the performance for alignment, such points should be processed first.We employ the simplification processing [20] and outlier deletion [21] as a preprocessing step.
After preprocessing, the density of point cloud is optimized and point number is uniformed.By default, the simplification number is specified to 50k.
To implement manifold metric, the simplified point cloud needs to be normalized in order to reduce the influence of scaling and translation, which can be regarded as functions T and S mentioned before.The normalization can be formulated as follows: where P is the simplified point cloud with specified number n and p i is a point of P. In KSS-ICP, such representation is directly used in alignment.Local features are ignored completely which affects the accuracy of the further alignment to a certain extent.To solve the problem, we introduce a SSF to combine the local feature analysis into alignment.Some feature points are labeled according to the SSF that provide a reference for the global correspondence.
To establish a functional SSF, a mature solution is to establish a local shape description based on normal vectors or curvatures, such as FPFH [12].It represents shape features by statistically analyzing the normal vector-based angles of a local region.Naturally, it is sensitive to noisy points.To solve the problem, we provide a tangent space-based statistical measure to represent SSF.It can be formulated as follows: where Nðp i Þ is the neighbor set of p i ; T is the tangent plane that is defined by the largest eigenvector of Nðp i Þ; dis represents the mapping distance between the point and the plane.It is clear that if a point is located in a region with sharp curvature changes, the value of SSFðp i Þ is larger.On the contrary, if the point lies on a plane, even if there are noisy points in its neighbor region, the related SSF value is still small.According to the SSF value, we label the top 10% of points (500) as feature points.An instance is shown in Figure 2. Finally, we achieve the compact representation of original point cloud with SSF-based labels.

SSF-Based Manifold Metric
Based on the compact representation with SSF, we propose the SSF-based manifold metric for the registration task.It is used to evaluate the quality of alignment between point clouds.As a classical measurement, Hausdorff distance has been widely used to measure geometric consistency between two shapes [20,22].However, it just provided an upper limit estimation and ignore internal feature correspondence.For point clouds with symmetrical structure, the accuracy of the measurement is reduced significantly.According to the KSS-ICP, the Hausdorff distance is used to simulate the manifold metric on Kendall shape space.We change the metric with SSF to enhance the local feature analysis.Then, a new manifold metric is proposed for shape similarity measurement on Kendall shape space, represented as follows: 4 Journal of Sensors where E represents the new manifold metric between two compact representations of point clouds P 1 and P 2 ; dis m is the mean distance between the representations, qi is the point with minimum distance to pi ; and α and β are weights to control the influences between global measurement and local feature alignment.It is clear that the new metric considers global correspondence and local feature analysis at the same time.The registration task is transferred into an optimization problem.
To optimize the manifold metric, we design a discrete solution-searching scheme based on a parallel structure.The optimized target is the rotation that is represented as follows: where rotation O r is the target result that corresponds to the global minimum of E. To implement the optimization, we generate a candidate rotation set fOg with certain steps for each coordinate axis: where R x ; R y ; R z are rotation angles according to the axis.Each one corresponds to a discrete range of angles with certain steps, like R x 2 f0°; 30°; …; 300°; 330°g.The certain step is 12 and the scale of fOg is 12 3 .To improve the efficiency, we use the GPU-based parallel structure to accelerate the searching for O r .The performance approaches real-time and the achieved rotation is the discrete global result.Figure 3 shows the GPU-based rotation searching scheme.

Experiments
The performance of our method is evaluated in this section.
The experimental machine is equipped with Intel i7 3.2 GHz, 16 GB RAM, RTX2080, and with windows 10 as its running system and Visual Studio 2019 as the development platform.The test dataset is collected from ModelNet40, RGB-D scenes, and the multisource sampling data.The experiments include the following parts: first, we introduce some details of the test dataset and related metrics we used; second, we compare different methods on the test dataset with similarity transformations and noisy point clouds; third, we provide a comprehensive analysis based on the experimental data; and finally, we test our method in the practical application.In addition, we utilize the 3D point clouds in the practical application to verify the effectiveness of our method in real data.
6.1.Dataset and Metrics.In this part, the test dataset is collected from ModelNet40 [23] and RGB-D scenes [24].We randomly select 500 models from ModelNet40 and all models from RGB-D scenes to implement experiments.Journal of Sensors 6.2.Comparisons 6.2.1.Similarity Transformations.Some representative methods are selected as the comparisons, including ICP [2], FPFH [12], Go-ICP [6], PointNetLK [8], Fast-ICP [5], and KSS-ICP [1].The ICP and FPFH are implemented by PCL library.Other methods provide related codes (Github) that can be achieved in their paper.In Figures 4 and 5, we show some registration results by different methods.It proves that our method achieves better results, especially for complex 3D scene with symmetric structures.The related metrics are reported in Tables 1 and 2 as a further explanation.
6.2.2.Noisy Robustness.As mentioned before, the noisy datasets are used to evaluate the noisy robustness for different methods.In Figure 6, we show instances of noisy point clouds with different Gaussian distributions.Related metrics are reported in Table 3.It is clear that our method can also achieve better results on these noisy data.Compared to the FPFH, our SSF is extracted from the tangent space without   6 Journal of Sensors   Journal of Sensors 7 point-based normal vector analysis, which improves the noisy robustness.Due to the SSFs, the symmetrical structures are well-handled while maintaining noise robustness.Our method efficiently solves the problem of traditional manifold metrics provided by KSS-ICP.
6.3.Analysis.The proposed method combines SSF and manifold metrics to implement registration task.It inherits advantages of global shape analysis scheme while enhancing the local shape feature correspondence.The traditional ICP scheme depends on the initial poses that increase the local optimum with high probability.The FPFH proposes a local shape feature to implement point-based correspondence.The feature is established by normal vectors that are sensitive to the noisy points.The SSF of our method is extracted in tangent space.It measures deviation degrees between a point and its neighborhood-based fitting plane.Even if there are noisy points that take some random movements, the deviation degrees are controlled in a small range which improves the noisy robustness.The KSS-ICP presents a global shape analysis framework based on Kendall shape space.Our method fully considers the advantages of its framework while addressing its issues through adjustments to the metric based on SSF.Therefore, we achieve better results in the standard test dataset, especially in symmetrical structures of 3D points.The time cost is reported in Table 4. Benefited from the SSF, our method can simplify point cloud with less points than KSS-ICP.It reduces the computation for alignment.
6.4.Application.We also test our method in the practical 3D digitization application of Chinese ancient architecture.The target building is the Small Wild Goose Pagoda, which is a world-famous cultural relic located in Xi'an city in China.This pagoda was built in the Tang Dynasty (618-907 AD) and is one of the most well-known landmarks of the city.As a typical representative of the dense eaves pagoda of the Tang Dynasty, this pagoda has rich historical and cultural value.Thus, the 3D digitization data are very important for related academic researches.
In the past, using 3D laser scanning for such a massive architecture was not only costly but also inefficient.The most challenging aspect is that the height of the building is more than 40 m with large amount of brick carving details on the facades.This leads to a result that we cannot acquire the complete 3D data with a single technique.Fortunately, with the maturity of unmanned aerial vehicle (UAV) hardware equipment and the continuous progress of aerial  8 Journal of Sensors oblique photography measurement technology, 3D digitization of tower-style ancient architecture in relatively low cost became possible.However, it also presents higher challenges for registration algorithms.
In this practical application, the aerial tilt photography and the ground laser scanning are simultaneously used.On the ground, we utilized a terrestrial laser scanner FARO Focus Premium to collect high-precision 3D point clouds with an accuracy of 0.2 cm.In the aerial photography stage, two UAVs are used to cruise different paths to collect aerial images around the pagoda.We totally collected 7,400 figures and reconstructed 3D point cloud in the software ContextCapture.In the data processing part, we need to conduct the 3D point cloud registration algorithm not only between different views but also on different data from various sources.
The complex data sources brought challenges on the registration algorithm, which is that there are much more outliers in the real-world data; and different sources generate point clouds in various densities and features.It is worth mentioning that our proposed method is very suitable when dealing with these practical problems.Figure 7 shows the comparison results of our method and the KSS-ICP algorithm.The input two-point clouds are generated by different techniques.The laser-scanned point cloud is colored in blue, while the reconstructed point cloud is in orange.Both the two pieces of data are postprocessed with downsampling to reduce the data volume.
In the middle of Figure 7, we can see the KSS-ICP algorithm cannot realize accurate alignment for input data.The reason is the input two data are generated in different scales, and there is partial defect in scanned data comparing with the reconstructed one.Because of considering SSFs, our method reduces the influences of scaling and translation, and then realizes ideal registration results.Figure 8 further illustrates our advantages in applications.
In Figure 8, the blue point cloud is the entire data that we generate from reconstructed and scanned points.In order to improve the density of the 2D images, we captured pictures by a digital camera from the ground.The reconstruction program was also executed on these images, and the reconstructed 3D points are colored in orange in Figure 8.The features of the input data are much more different than the last inputs in the point cloud density, scales, and integrity.Applied experimental results show that our method achieves better results than KSS-ICP, which proves the effectiveness of our method on the real data.

Conclusions
In this paper, we propose an SSF-based method for the 3D point cloud registration.It combines global correspondence and local shape feature analysis during the registration.The SSF is extracted from the tangent space that improves the noisy robustness.The SSF-based manifold metric provides more accurate measurement for alignment.With a parallel structure, the computational efficiency of our method is improved significantly.Experiments show that our method achieves better balance between accuracy, robustness, and efficiency.the 3D scanning of Small Wild Goose Pagoda; and Haiying Tao worked on the project administration of the study.

FIGURE 4 :
FIGURE 4: Comparisons of different methods on models of ModelNet40.Blue: template point cloud and yellow: source point cloud and registration results by different methods.

FIGURE 5 :
FIGURE 5: Comparisons of different methods on models of RGB-D scenes.

FIGURE 7 :
FIGURE 7: Registration results from real point clouds (Small Wild Goose Pagoda).The blue point cloud is the scanned model from real object, and the orange ones are reconstructed points from images.
The quantitative metrics are represented by average value of the mean squared error (MSE) and the root mean squared error (MAE) based on Euclidean distances and normal vector-based angles.Based on the test dataset and related metrics, the experimental results can be reported.
dataset to generate two noisy datasets with different Gaussian distributions just like in Lv et al.'s [1] study.

TABLE 1 :
Metrics of different registration methods on models of ModelNet40.MSE(n) and MAE(n) represent the normal vector angle-based MSE and MAE.Bold values signify the best results.

TABLE 2 :
Metrics of different registration methods on models of RGB-D scenes.Bold values signify the best results.
FIGURE 6: Instances of noisy point clouds with different Gaussian distributions.

TABLE 3 :
Metrics of different registration methods on noisy models of RGB-D scenes. is the parameter of Gaussian distribution for noisy point generation.Bold values signify the best results. σ

TABLE 4 :
Time cost report for different registration methods.Bold values signify the best results.