We segment buildings and trees from aerial photographs by using superpixels, and we estimate the tree’s parameters by using a cost function proposed in this paper. A method based on image complexity is proposed to refine superpixels boundaries. In order to classify buildings from ground and classify trees from grass, the salient feature vectors that include colors, Features from Accelerated Segment Test (FAST) corners, and Gabor edges are extracted from refined superpixels. The vectors are used to train the classifier based on Naive Bayes classifier. The trained classifier is used to classify refined superpixels as object or nonobject. The properties of a tree, including its locations and radius, are estimated by minimizing the cost function. The shadow is used to calculate the tree height using sun angle and the time when the image was taken. Our segmentation algorithm is compared with other two state-of-the-art segmentation algorithms, and the tree parameters obtained in this paper are compared to the ground truth data. Experiments show that the proposed method can segment trees and buildings appropriately, yielding higher precision and better recall rates, and the tree parameters are in good agreement with the ground truth data.
With the fast pace of industrialization and urbanization, 3D models are more and more necessary for urban planning, flight simulator, and military training. It is important to identify buildings and trees in high-resolution aerial photographs for displacement maps in real-time, which is a key procedure for building 3D models, because buildings and trees not only are significant features for city modeling, but also often occlude other elements in 3D urban models. Therefore, the first step of displacement maps is to detect buildings and trees from aerial photographs. Automatic detection of trees and buildings is a challenging work because, first of all, the input data sets of aerial photographs are huge; second, the features of buildings and trees are various, and it is hard to find salient ones for training purposes; third, it is difficult to classify building from ground and classify tree from grass in desert climate regions like Arizona State in USA.
Recently, many methods have been proposed for aerial images objects detection. In this section, we will provide a description of prior art of building and tree detection from three aspects: (1) buildings and trees segmentation; (2) superpixel refinement; (3) salient region feature.
There are many methods for tree segmentation from aerial urban images. Iovan et al. [
Levinshtein et al. [
Therefore, we can find that a drawback of most of these methods is their high computational complexity and hence high computational time consumption. Second, a sophisticated method for marker image calculation which respects both the remaining natural edges in the image and the regularity of marker placement in still regions of the image still has to be developed.
Proposing a method to automatically estimate the superpixels number for an aerial photo based on measurement of image complexity and detect boundaries of the undersegmentation region. The new method makes our approach robust against missing or erroneous metadata about image resolution. Proposing a salient feature vector for training the classifier, which is simple and efficient and easy to be implemented. Proposing an approach for estimating the location and radius of a tree crown and the tree height without additional information, which is simple and efficient and easy to be implemented.
The study area is located at Phoenix, Arizona, United States, and its location is (
The overview area of Phoenix, USA.
The test images are from urban or suburban regions of Arizona, USA. These aerial images were collected at August, 2008, using the ADS40 airborne digital sensor. The height taken in the images is 600 meters. This sensor incorporates a line-array of charge-coupled devices (CCDs) and is capable of acquiring visible to near-infrared stereo data at ground resolutions of 0.5 m/pixel. The detail of this sensor can be found in [
Main EXIF information of experimental images.
Tag | Value |
---|---|
Manufacturer | Leica |
Model | ADS40 |
DateandTime | 2008 |
ExposureTime |
|
ComponentsConfiguration | 4.01 |
MaxApertureValue | 2.0 |
FocalLength | 20.1 mm |
ColorSpace | sRGB |
PixelXDimension | 6250 |
PixelYDimension | 6250 |
The original image with its zoom-in area.
Three examples of the original labeled images for classifier training.
In this paper, we propose a building and tree segmentation method using superpixels from a single large high-resolution urban image. The procedure of this algorithm is shown in Figure
The flow chart of our algorithm.
In this paper, an improved method is proposed to presegment the large scale image with a high accuracy. This method is based on Turbopixels whose number of superpixels was given by users. However, different from the Turbopixels, our method used two formulas to overcome the undersegmentation caused by Turbopixels.
In order to use superpixels [
The aim of superpixels is to reduce the problem by replacing pixels with regularly spaced, similarly sized image patches whose boundaries lie on edges between objects in the image, so a pixel-exact object segmentation can be accomplished by classifying superpixel patches rather than individual pixels. However, the boundaries of the superpixels will not match with the edges of the objects as well if the number of superpixels generated is too small, and the computation will be expensive when the number is too large.
In this paper, we use the image complexity to calculate the number of superpixels. Image complexity is defined as a measure of the inherent difficulty of finding a true target in a given image. The image complexity metric is determined by the gray-level feature or the edge-level feature [
Testing different types of images which were collected by our lab, we suggest that the internal of
In this paper, we use a method based on Canny detector to overcome the undersegmentation. Canny [
In this paper, we propose a function
Simply defining thr to be the three-dimensional feature in color space will cause inconsistencies in clustering behavior for different superpixel sizes. This produces compact superpixels that adhere well to image boundaries. For smaller superpixels, the converse is true.
A superpixel-based classifier is proposed for segmenting buildings and trees from a presegmentation image. In order to train the classifier, we assign
In our method,
Corner refers to a small point of interest with variation in two dimensions. There are many corner detectors which can be classified into two categories. (1) Corner detector based on edges: an edge in an image corresponds to the boundary between two areas and this boundary changes direction at corners. Many early algorithms detected corners based on the intensity changes. (2) Corner detector based on chained edge: lots of methods have been developed which involved detecting and chaining edges with a view to analyzing the properties of the edge. Rosten et al. proposed a high-speed corner detection method based on FAST (Feature from Accelerated Segment Test) and machine learning [
In our method, we use the Gabor filter to extract structure features of buildings and trees. Structure is an important feature to segment buildings from street and ground, and it is also useful to segment trees from grass. Furthermore, frequency and orientation representations of Gabor filters are similar to those of the Human Visual System (HVS), and they have been found to be particularly appropriate for texture representation and discrimination. Among various approaches for extracting texture features, Gabor filter has emerged as one of the most popular ones [
From the zoom-in region of Figure
Through concatenating the above features based on SVM, a salient feature vector is generated for each superpixel. Then we use this feature vector to train a classifier for building and tree segmentation. Finally, Naive Bayes classifier is chosen to be used for tree and building classification in our algorithm because it is efficient and easy to be implemented [
After segmentation, a tree location
The tree model matching is applied to
The relationship between tree and its shadow.
In order to calculate the tree’s parameter, the gradient descent method is employed to achieve iterations. The convergence speed is robust. This procedure is shown in the following steps.
Calculate the gradient value
Calculate the norm value of
Update the
Calculate the new gradient value
Calculate the norm value of
Calculate the ratio of
If
This procedure is finished until getting the minimum value of tree parameters
Our method was implemented using Microsoft Visual Studio 2010 and Matlab 2010a, and this method was coded by integrating C# and Matlab. The proposed algorithm was tested on many large high-resolution urban images taken from Phoenix, Arizona, USA.
In our experiment, the method is implemented by two steps: (1) the first step is presegmentation, and the result is compared with the result which is got by Turbopixel and Entropy Rate Superpixel (ERS); (2) the second is to segment buildings and trees, and the segmentation result is compared to the result got by using Turbopixel and ERS [
Figure
The comparative presegmentation result.
The result based on our method
The result based on Turbopixels
The result based on ERS
Figure
Figure
The comparative segmentation result.
Furthermore, two standard metrics which were undersegmentation error [
The comparison results of UE and BR. (a) Undersegmentation error curves; (b) undersegmentation error bar; (c) boundary recall curves; (d) boundary recall error bar.
According to Figure
To justify the use of Naive Bayes classifiers which is trained by the proposed algorithm in our approach, we tried the popular intersection kernel and Chi-square kernel on our feature vector including color, FAST corner, and Gabor feature for comparison. Figure
The visual comparison of building and tree classification result. (a) Classification result by our proposed method; (b) the classification based on intersection kernel; (c) the classification result based on Chi-square kernel.
In our method, we calculated the tree parameters one by one. Each tree with shadow was extracted from every segmentation image, such as Figure
The segmentation result which is shown by GUI.
The comparing result. (a) The centers of trees in horizontal direction; (b) the centers of trees in vertical direction; (c) the radiuses; (d) the height.
In this paper, we proposed a novel method for building and tree segmentation from large scale urban aerial images, and we also proposed a new approach for estimating tree parameters. In Section
In Section
In Section
Tree model is widely used to detect trees. Morsdorf et al. [
Finally, the experiments show that the locations and radiuses of the trees estimated by our approach are approximate to the manual-picking methods. However, there are some heights of trees higher or lower than the ground truth ones shown at Figure
In this paper, we proposed a building and tree detection algorithm by using improved superpixels from large high-resolution urban images, and we also proposed a method to calculate the tree parameters depending upon a cost function and shadows. A function was proposed to automatically calculate the number of superpixels, and a function is used to refine the boundaries of superpixels. We provided a new tree model with a cost function that can be minimized using gradient decent in order to identify the optimal properties of individual tree. We evaluated our method by using many aerial images and compared our method with other two state-of-the-art methods. Experiments showed that our method is fast and robust, while still being simple and efficient, and they also indicate that the shadow is a good feature to estimate the tree height. The results of our method can be implemented for generating 3D urban models. The main purpose of this paper is to design fast, accurate segmentation and classification, characteristics not focused on the learning process. However, we will examine the different supervised learning algorithms in the future research work, improving the effectiveness of the proposed algorithm. Future work will focus on comparing our method with other supervised algorithms including methods of integrating these data sources into our solution.
Parameters used in the rule sets are sample values specifically for the study area we chose and may vary for other locations of interest, but the similar principles and procedures can be applied to other areas. Using additional ancillary data, such as 1-meter resolution LiDAR (Light Detection and Ranging), may further help generate more accurate land-cover maps, which is among our planned future work.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This project is sponsored by the The National High Technology Research and Development Program of China (no. 2014AA123103 and no. 2015AA124103) and National Natural Science Foundation of China (no. 61401040).