^{1}

^{2}

^{1}

^{1}

^{1}

^{2}

Object shape reconstruction from images has been an active topic in computer vision. Shape-from-shading (SFS) is an important approach for inferring 3D surface from a single shading image. In this paper, we present a unified SFS approach for surfaces of various reflectance properties using fast eikonal solvers. The whole approach consists of three main components: a unified SFS model, a unified eikonal-type partial differential image irradiance (PDII) equation, and fast eikonal solvers for the PDII equation. The first component is designed to address different reflectance properties including diffuse, specular, and hybrid reflections in the imaging process of the camera. The second component is meant to derive the PDII equation under an orthographic camera projection and a single distant point light source whose direction is the same as the camera. Finally, the last component is targeted at solving the resultant PDII equation by using fast eikonal solvers. It comprises two Godunov-based schemes with fast sweeping method that can handle the eikonal-type PDII equation. Experiments on several synthetic and real images demonstrate that each type of the surfaces can be effectively reconstructed with more accurate results and less CPU running time.

In the field of computer vision, object shape reconstruction from images has been an active topic. There are several techniques, such as stereo vision, structured light, fringe projection profilometry, and shape-from-

Since Horn’s original work, a great number of different SFS approaches have come out (for surveys, refer to Zhang et al. [

While most of the SFS approaches assume the Lambertian reflection, there are a few researchers who are interested in non-Lambertian SFS since the Lambertian reflectance model has been proved to be inaccurate, especially for rough diffuse surfaces [

In this paper, motivated by the work of Camilli and Tozza [

A similar formulation for the SFS problem of the Oren-Nayar model has been reported in our previous work [

In this section, a very brief description for the Lambertian, Oren–Nayar and Blinn–Phong reflectance model is given in order to setup a unified imaging model.

Generally, the Lambertian reflectance is a classical assumption in most of the SFS approaches [

Reflection geometry of a local surface point.

In order to get rid of the inaccuracy resulting from the assumption of the Lambertian reflectance model for diffuse reflection, Oren and Nayar [

By assuming that the surface is composed of V-shaped cavities which are symmetric and have two planar facets and that each facet obeys a simple Lambertian reflection, for a Gaussian distribution of the facet normals, they got a simplified expression for the reflected radiance:

For smooth surfaces, we have

It is worth mentioning that Phong [

Note that it is not convenient to compute the specular reflected radiance in terms of

As mentioned before, the Lambertian reflectance model has been proved to be inaccurate, especially for rough diffuse surfaces. Thus, we can combine diffuse and specular components of a surface through a linear combination of Oren–Nayar model and the specular part of Blinn–Phong model; that is, we substitute surface reflected radiance (

Obviously, surface reflected radiance (

The following relationship between the surface reflected radiance

Substituting equation (

In this section, we will formulate the image irradiance equation under the situation where the camera performs an orthographic projection and the direction of the single distant point light source coincides with the camera.

With the basis that the optical axis of the camera is the

So the unit normal

Considering that the direction of the distant point light source

Defining that the direction vectors of

Substituting equation (

Obviously, the image irradiance equation (

Especially, for

Calculating equation (

Hence, SFS problem (

For

If the surface roughness

Hence, function (

After several numbers of iterations, an accurate zero of function (14) is obtained. Similar to the structure of the Oren–Nayar model, we can get an eikonal-type PDE for the unified model:

In this section, we will use the fast eikonal solvers which are composed of the first-order Godunov-based scheme [

We use

Thus, the viscosity solution of eikonal-type PDE (

In order to obtain a higher-order accuracy viscosity solution, the high-order Godunov-based scheme [

In order to speed up the convergence numerical schemes, we take the philosophy of fast sweeping method [

We summarize the fast eikonal solvers for the resultant eikonal-type PDE (

Step 1 (Initialization): according to the boundary condition

Step 2 (Alternating Sweepings): we compute

Step 3 (Convergence): if

Several experiments on synthetic and real images with different reflectance properties have been carried out in order to assess the effectiveness of the presented unified SFS approach. We compare our presented approach with the Ahmed and Farag’s approach [

We use two synthetic surfaces including a ball and a vase, which have been benchmark test surfaces and are determined by equations (

The ground truths of the ball and the vase surfaces. (a) The ball. (b) The vase.

In order to assess the effectiveness of the presented unified SFS approach for the surfaces of various reflectance properties, four different parameter sets of

Parameter values employed to generate the shading images.

Parameter | ||||
---|---|---|---|---|

Set ( | 0 | 0.8 | 0.2 | 5 |

Set ( | 0 | 0.5 | 0.5 | 10 |

Set ( | 0.3 | 1 | 0 | — |

Set ( | 0.3 | 0.5 | 0.5 | 10 |

The experimental results for the synthetic ball images are illustrated in Figure

Experimental results for the synthetic ball images. (a)–(d) The shading images generated by the four parameter sets shown in Table

Experimental results for the synthetic vase images. (a)–(d) The shading images generated by the four parameter sets shown in Table

As can be roughly seen from Figures

The effectiveness of our presented unified approach is further described by comparisons between the fast eikonal solvers and the Lax-Friedrichs sweeping scheme with the mean absolute (MA) error, the root mean square (RMS) error, and the CPU running time. Tables

Quantitative comparisons of schemes for the synthetic ball images.

Images | First-order Godunov-based scheme | High-order Godunov-based scheme | Lax-Friedrichs sweeping scheme | ||||||
---|---|---|---|---|---|---|---|---|---|

MA | RMS | Time (s) | MA | RMS | Time (s) | MA | RMS | Time (s) | |

Figure | 0.7199 | 0.8924 | 0.04 | 0.0370 | 0.0883 | 0.64 | 3.2049 | 3.3360 | 2.49 |

Figure | 0.7228 | 0.9176 | 0.04 | 0.0595 | 0.1318 | 0.65 | 3.9782 | 4.1671 | 3.69 |

Figure | 0.7167 | 0.8902 | 0.04 | 0.0357 | 0.0725 | 0.64 | 2.6534 | 2.7687 | 2.67 |

Figure | 0.7776 | 1.0667 | 0.04 | 0.0940 | 0.1959 | 0.65 | 4.1396 | 4.3401 | 5.07 |

Quantitative comparisons of schemes for the synthetic vase images.

Images | First-order Godunov-based scheme | High-order Godunov-based scheme | Lax-Friedrichs sweeping scheme | ||||||
---|---|---|---|---|---|---|---|---|---|

MA | RMS | Time (s) | MA | RMS | Time (s) | MA | RMS | Time (s) | |

Figure | 0.5770 | 0.7129 | 0.04 | 0.0740 | 0.1371 | 0.81 | 2.2899 | 2.5879 | 2.31 |

Figure | 0.5791 | 0.7284 | 0.04 | 0.0812 | 0.1429 | 0.82 | 3.5141 | 3.8832 | 3.53 |

Figure | 0.5739 | 0.7095 | 0.04 | 0.0731 | 0.1366 | 0.81 | 1.3744 | 1.6557 | 3.61 |

Figure | 0.6309 | 0.7429 | 0.04 | 0.0953 | 0.1550 | 0.82 | 4.0818 | 4.4402 | 4.82 |

In order to demonstrate the performance of our presented approach for real surface, we test it on two real images and also compare the reconstructed results with the Lax-Friedrichs sweeping scheme. The first image is a vase applied in [

Experimental results for the real vase image. (a) The real image. (b) The mask of (a). (c) Reconstructed surface of (a) using first-order Godunov-based scheme. (d) Reconstructed surface of (a) using high-order Godunov-based scheme. (e) Reconstructed surface of (a) using Lax–Friedrichs sweeping scheme.

Experimental results for the real bottle image. (a) The real image. (b) The mask of (a). (c) Reconstructed surface of (a) using first-order Godunov-based scheme. (d) Reconstructed surface of (a) using high-order Godunov-based scheme. (e) Reconstructed surface of (a) using Lax–Friedrichs sweeping scheme.

We only evaluate the effectiveness intuitively and qualitatively. From the reconstructed results shown in Figures

In this paper, we have reported a unified SFS approach for surfaces of various reflectance properties including diffuse, specular, and hybrid reflections using fast eikonal solvers. A unified reflectance model that is a linear combination of the Oren–Nayar model and the specular part of the Blinn–Phong model is presented. We have derived the unified image irradiance equation under this unified model with an orthographic camera projection and a single distant point light source whose direction is the same as the camera. We have also converted the PDII equation into an eikonal-type PDE through solving a high-order equation by using the Newton-Raphson method. Fast eikonal solvers which are comprised of the first- and high-order Godunov-based schemes accelerated by the fast sweeping method are employed to solve the viscosity solution of the resultant eikonal-type PDE. Finally, the experiments are conducted on both synthetic and real images and the results verify that our presented approach can provide satisfactory 3D surface reconstruction with a higher accuracy in less CPU running time.

Frankly speaking, the presented unified SFS approach can only handle the special case which assumes an orthographic camera projection and a single distant point light source whose direction is parallel to the optical axis of the camera lens. In future work, we will adopt the idea of using the Newton-Raphson method to solve the high-order PDII equations derived from the SFS problem with a more complex reflectance model and will relax the two assumptions by employing a nearby point light source and a perspective camera projection. The attenuation term of the light illumination will be also considered to eliminate the convex-concave ambiguity which can make the SFS problem ill-posed.

The data used to support the findings of this study are included within the article.

The authors declare that they have no conflicts of interest.

The work in this paper is supported by the Open Fund of Shaanxi Province key Laboratory of Photoelectricity Measurement and Instrument Technology, Xi’an Technological University (no. 2016SZSJ-60-1) and Xi’an Key Laboratory of Intelligent Detection and Perception (no. 201805061ZD12CG45).