An effective automatic 3D reconstruction method using a portable four-camera photographic measurement system (PFCPMS) is proposed. By taking advantage of the complementary stereo information from four cameras, a fast and highly accurate feature point matching algorithm is developed for 3D reconstruction. Specifically, we first utilize a projection method to obtain a large number of dense feature points. And then a reduction and clustering treatment is applied to simplify the Delaunay triangulation process and reconstruct a 3D model for each scene. In addition, a 3D model stitching approach is proposed to further improve the performance of the limited field-of-view for image-based method. The experimental results tested on the 172 cave in Mogao Grottoes indicate that the proposed method is effective to reconstruct a 3D scene with a low-cost four-camera photographic measurement system.
The 3D models have been widely used in various applications such as virtual cultural heritage conservation [
Real object models can be reconstructed automatically using active and passive methods. Object range scanning by laser [
A 3D model reconstruction system using images acquired from multiple stereo pairs by PFCPMS [
The data collection process is divided into two parts. The first is the hardware system. The second is the system’s matching method and how to ensure the correctness of the matches.
The PFCPMS is shown in Figure
The main hardware system: (a) the PFCPMS and (b) the structure.
The optical axes of four cameras are parallel to each other, and their parameters should be consistent as far as possible in order to ensure the precision. The four image planes and their optical centers are on an identical plane. Let the four image planes be IMG1, IMG2, IMG3, and IMG4, respectively, and the four optical centers be
The PFCPMS can perform high accuracy stereo matching quickly. As shown in Figure
The feature matching method for the PFCPMS: (a) the epipolar characteristic and (b) the flow chart for feature matching.
The last two steps of the matching flow are very important and necessary, though the short baseline can narrow the search area and reduce the mismatches markedly; the errors will appear when the surface has repetitive texture, especially the dense and small one. Usually, more than one matched
The PFCPMS improved the stereo matching effectively, but a problem should be taken into consideration in order to get enough reliable point cloud. It is the poor texture, as shown in Figure
Intensive feature point generation method: (a) poor texture, (b) square projection, (c) the checkerboard, and (d) the projector.
In order to get enough reliable point cloud, a projection method is applied, as shown in Figure
After obtained 3D point cloud, Delaunay triangulation [
The original point cloud obtained from Section If If If
Point cloud data processing: (a) original data, (b) uniform sampling, and (c) nonuniform sampling.
The result is shown in Figure
After point cloud reduction, the Delaunay triangulation process is applied to build a large number of small grids, as shown in Figure
The point cloud classification optimization: (a) original triangular mesh, (b) triangular mesh after clustering, (c) original 3D model, and (d) 3D model after clustering.
Define the point cloud as
Select one point
The
If
When a core is found, then find
The points in
After clustering, the errors have been solved. The newly generated triangular mesh and 3D model are shown in Figures
The photographic measurement system can reconstruct 3D model effectively, but a key problem remains because of limited field-of-view for normal camera. This can be reflected from Figure
3D model stitching effect: (a) the initial model and (b) after stitching.
Get the Harris corners and stereo matching in order to calculate the point’s 3D coordinates. The method is similar to Section
Feature point matching between different 3D models. First, the common correlation method is applied. Then, calculate the distance
Calculate the rotation parameters
Usually, more than three matched points can be detected, so the least square method can be applied to improve the computational accuracy. The stitching result is shown in Figure
To validate the effectiveness of our proposed 3D model reconstruction system, the experiment is conducted on the 172 cave in Mogao Grottoes, which is located 25 kilometers southwest of Dunhuang city, Gansu Province in China. In the meantime, another group has reconstructed the 159 cave by traditional 3D laser scanning technique. The new and traditional methods have been tested on different caves in the meantime because of the restricted condition of data sampling. Although the scenes are not exactly the same for these two methods, the characteristic, difficulty, and workload are similar. As a result, the contrast experiment between them is significant. A typical comparison result can be seen in Figure
The 3D reconstruction results by our method and laser: (a) the triangular mesh by PFCPMS, (b) the 3D model reconstructed by PFCPMS, (c) the 3D model scanning by laser, and (d) the laser 3D model after texture registration.
To quantitatively illustrate the effectiveness of this method, two real and touchable objects are reconstructed by a smaller PFCPMS. The 3D models can be seen in Figure
The real and touchable 3D models: (a) ornamental vase and (b) plaster head.
Since all points of the space coordinates are a relative value, the value of a single coordinate is not significant. So the reconstruction accuracy is evaluated by the space distance between two points. For each of the two models in Figure
Quantitative analysis of reconstruction results (unit: meters).
Ornamental vase | Plaster head | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Measured value | 0.1342 | 0.1282 | 0.3438 | 0.1388 | 0.2538 | 0.1610 | 0.1233 | 0.0835 | 0.1303 | 0.1095 |
True value | 0.1281 | 0.1352 | 0.3340 | 0.1290 | 0.2480 | 0.1700 | 0.1180 | 0.0765 | 0.1230 | 0.1185 |
Error | 0.0061 | −0.007 | 0.0098 | 0.0098 | 0.0058 | −0.009 | 0.0053 | 0.0070 | 0.0073 | −0.009 |
All of the errors from Table
In Section
The integrated 3D model: (a) left view, (b) right view, (c) top view, and (d) bottom view.
During the 3D reconstruction process, the most time consuming procedure is Delaunay triangulation. The original point cloud has 1257305 points; the computation speed is too slow that is impossible to calculate. After point reduction and clustering, only 41649 points are preserved, and the 3D reconstruct process spends about half an hour.
An effective automatic 3D reconstruction method using PFCPMS has been proposed. The method has been tested on the 172 cave in Mogao Grottoes and compared with the 3D model scanning by laser. It indicated that the proposed method is effective to reconstruct a 3D scene with a low-cost four-camera photographic measurement system.
The limitation of the system is that no good quantitative evaluation standard has been developed. A rough idea is evaluated performance by LIDAR scans. But the cost is so high that is against our design philosophy. Future research is to investigate an appropriate method.
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors are grateful to the anonymous reviewers for their comments, which have helped them to greatly improve this paper. This project is supported by the 973 Program (2012CB725301), the Nature Science Foundation of Hubei Province of China (2015CFC770), and the Science and Technology Foundation of the Department of Education of Hubei Province (Q20152701), and the National Nature ScienceFoundation of China (61471161).