Journal of Sensors
Volume 2018 (2018), Article ID 6869807, 12 pages
https://doi.org/10.1155/2018/6869807

Research Article

Detection of Firmiana danxiaensis Canopies by a Customized Imaging System Mounted on an UAV Platform

Chun Liu,¹ Mengchi Ai,¹ Zhuo Chen,¹ Yuan Zhou,² and Hangbin Wu¹

¹College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China
²Tongfang Surveying Engineering and Technology Co. Ltd., Shanghai 200092, China

Correspondence should be addressed to Mengchi Ai; 1610954@tongji.edu.cn

Received 1 February 2018; Accepted 27 March 2018; Published 27 May 2018

Academic Editor: Zongyao Sha

Copyright © 2018 Chun Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The objective of this study was to test the effectiveness of mapping the canopies of Firmiana danxiaensis (FD), a rare and endangered plant species in China, from remotely sensed images acquired by a customized imaging system mounted on an unmanned aerial vehicle (UAV). The work was conducted in an experiment site (approximately 10 km²) at the foot of Danxia Mountain in Guangdong Province, China. The study was conducted as an experimental task for a to-be-launched large-scale FD surveying on Danxia Mountain (about 200 km² in area) by remote sensing on UAV platforms. First, field-based spectra were collected through hand-held hyperspectral spectroradiometer and then analyzed to help design a classification schema which was capable of differentiating the targeted plant species in the study site. Second, remote-sensed images for the experiment site were acquired and calibrated through a variety of preprocessing steps. Orthoimages and a digital surface model (DSM) were generated as input data from the calibrated UAV images. The spectra and geometry features were used to segment the preprocessed UAV imagery into homogeneous patches. Lastly, a hierarchical classification, combined with a support vector machine (SVM), was proposed to identify FD canopies from the segmented patches. The effectiveness of the classification was evaluated by on-site GPS recordings. The result illustrated that the proposed hierarchical classification schema with a SVM classifier on the remote sensing imagery collected by the imaging system on UAV provided a promising method for mapping of the spatial distribution of the FD canopies, which serves as a replacement for field surveys in the attempt to realize a wide-scale plant survey by the local governments.

1. Introduction

Forests play a vital role on global carbon cycle. Understanding constitution and spatial distribution patterns of plant species in forestry is a major concern of botanists and environmentalists [1]. Great effort has been input to distinguish the plant species of forests and analyze their distribution. As an effective tool to acquire land coverage over vast regions, remote sensing technology has been applied on detection of forests. Several studies have conducted mapping of forest distribution and extraction of the species composition by applying remote sensing mounted on satellites and unmanned aerial vehicles (UAVs) [2]. The obvious advantage of satellite remote sensing is its capability in monitoring vegetation condition and forest resources over a large-scale area in a fast and cost-effective manner [3]. For instance, satellite remote sensing data has been approved effective in mapping the outline of a forest area in order to monitor the deforestation [4]. Remote sensing products supported by satellite platforms range from high spatial resolution to low resolution. Typical examples of high-resolution imagery from satellite platforms include Spot and WorldView-2 images which were used to identify urban tree species to assess the ecological services [5]. More coarse remote sensing imageries, such as the medium-resolution products from Landsat Thematic Mapper (TM) and high-resolution imagery Enhanced Thematic Mapper Plus (ETM+), were probably only suitable for mapping distribution of grouped plant species with an advantage of covering a larger area [6]. Recently, applications of remote sensing imagery acquired by an UAV have increased dramatically in various fields. Compared to satellite remote sensing, UAV remote sensing enhanced the ability to acquire high-spatial resolution images [7]. The application of UAV remote sensing included vegetation health monitoring, detailed composition analysis of plant species, and biomass estimation [8]. Further, remotely sensed images by UAVs provided important data resources for building high-resolution airborne orthophoto maps and point clouds which have been extensively applied in ecological studies [9].

In the area of forest monitoring, dynamic mapping of plant composition, properties of plant canopies, and forest health is an important task and remote sensing technology has been widely used as a data acquisition tool for such a purpose. For example, a metric camera mounted on an UAV was used to classify riparian forest species and to analyze the composition of forests [10]. Hill et al. examined whether such an UAV would be capable of creating accurate maps of the extent of patches of a single invasive plant, called IRIS (Iris pseudacorus L.) and they found that manual interpretation of the UAV-acquired imagery produced the most accurate maps, suggesting that more studies on automatic extraction of plant species are still needed to improve the classification performance from UAV images [11]. Other studies used the high-spatial resolution images acquired on UAV platforms to investigate detailed plant communities [12]. Recent advances in UAV-based remote sensing technology provide a way for obtaining highly densified point clouds from remotely sensed targets, which are useful for building a 3-D canopy structure for trees in forestry [13]. When 3-D data derived from images on UAV platforms and a digital elevation model were combined, it is possible to measure the height of plants and calculate the magnitude of changes in shape for the mapped plant species with great flexibility [14]. Zhang et al. designed a remote sensing system mounted on a light-weight drone for long-term forest monitoring and, based on the acquired data, built the canopy height model (CHM), which was useful for analyzing the species richness and abundance distributions [15]. In another application, an imaging system with near-infrared and RGB channels mounted on an UAV platform was used to investigate diseases and insect pests of vegetation [16]. Numerous studies have tested the effectiveness of mapping plant species of forests from remote sensing images. To better extract plant species and evaluate their health, Näsi et al. used RGB orthoimages, along with hyperspectral imagery collected on UAV platforms, to identify plants and map bark beetle damage on an individual tree level [17]. However, in some cases where vegetation species presented low dissimilarity in spectra and texture in remote sensing images, it was difficult to distinguish among them. For instance, Firmiana danxiaensis (FD), a Chinese unique and endangered Indus species, is one example that is difficult to be detected due to its growth condition and characteristics [18].

While recent advancement in remote sensing platforms provided possible data sources from which plant species could be potentially detected, an alternative approach for improving the result in vegetation mapping looks for advanced image classifiers. Many classification methods such as random forest (RF), artificial neural network (ANN), decision trees (DT), and support vector machine (SVM) have been proposed and applied for extracting vegetation cover from remotely sensing images. For example, Michez et al. used RF to detected riparian invasive plant species with unmanned aerial systems imagery [19]. Kwon et al. used ANNs to classify the forest vertical structure from digital images and Lidar data [20]. Frick et al. used a DT model to classify and analyze the vegetation composition in rewetted peat land from satellite imagery [21]. Han et al. detected the cropping area in the middle of Heihe River Basin based on SVM using multitemporal vegetation index data [22]. Compared to other classifiers, SVM-based classifiers are built on statistic learning theory which classify objects through the maximal margin hyperplane established from very limited input features called support vector [23]. A few studies have shown that in most cases, SVM was less sensitive to the training samples and the performance was better than other classifiers with the same number of training samples [24]. Therefore, SVM is often preferred for classification purposes when only very limited truth samples are available. The objective of this study is to verify the feasibility of mapping FD canopies from remotely sensed images acquired by a customized UAV imaging system and to test the effectiveness of a proposed image classifier, a hierarchical classification schema powered by SVM, to extract FD from the remotely sensed imagery.

The objective of this study is to verify the feasibility of mapping FD canopies from remotely sensed images acquired by a customized UAV imaging system and to test the effectiveness of a proposed image classifier, a hierarchical classification schema powered by SVM, to extract FD from the remotely sensed imagery.

2. Study Site

As the main habitat of FD, Danxia Mountain (located at 113°36E, 24°51N), Shaoguan City, Guangdong Province, China, was being studied (Figure 1). The total area of Danxia Mountain was around 200 km². The climate was characterized as subtropical monsoon, with an annual mean temperature of 19.7°C and annual precipitation of 1715 mm. The land surface was mainly covered by bare stones, conglomerates, and red soil. Dominant families of the flora included Osteomeles subrotunda, Trichophorum subcapitatum, Engelhardtia chrysolepis, Lithocarpa uvariifolia, Acer oblongum, Helieia kwangtungensis, resurrection plant, and Firmiana danxiaensis (FD). FD is a rare and endangered plant in China, and thus, protecting the plant species has become a significant task for both botanists and local governments.

(a)

(b)

(c)

Figure 1: Study region. (a) Location of Danxia Mountain in the country, (b) outline of the study site, and (c) the experiment site.

FD mainly grows in shallow soil in Danxia Mountain (Figure 2). In fact, FD is the only tree species dominant in Mt. Danxia and was listed as a threatened species in China Species Red List [25]. Some characteristics about the species are listed in Table 1. The morphological form of FD canopies varies in shape, which makes them difficult to be extracted from other plant species purely through remote sensing images. FD reaches mature growing status during summertime which is also the season that an FD plant develops into its typical shape. Local governments as well as botanists are eager to map the spatial distribution of FD canopies and may design policies to protect the plant. But accurate positioning of the plant species is so far hard to obtain. The ground survey serves as a traditional method to map FD distribution but was limited due to the steep terrain, which makes some data impossible to be collected [26]. Therefore, alternative methods having more efficiency were desired. Satellite remote sensing could be too coarse to detect FD due to its small crowns, and plant texture information in imagery could not be captured, resulting in unsatisfactory extraction results of FD detection. Instead, data acquisition with UAVs could solve the issue because of the high-spatial resolution of UAV-collected images. However, considering the large area of the mountain (200 km²), a typical experiment site with a small area (3 km × 4 km) was decided as a testing project before conducting wide-scale FD mapping by an UAV platform in this study.

Figure 2: Examples of FD distribution. The black squares denote a few examples of FD locations, and the typical canopy structure of the plant is illustrated in the inset figure.

Table 1: Physical descriptions of FD.

Characteristics
Description

Category
Arbor

Height
3 m–8 m

Canopy area
12.5–30 m²

Blade shape (outline)
Mostly ellipse but with variations

Blade color
Turquoise

3. Materials and Methods

3.1. Customized Multispectral Imaging System

3.1.1. Ground Spectra Collection and Selection of Spectral Bands

As a preliminary step, we distinguished the spectra among the plant species by acquiring on-site spectral signals of each plant species and the bare ground (denoted as Ground, hereafter) where there is no vegetation/plant covered. The on-site spectral signals were collected on July 11, 2016, using a field spectrometer (Avafield-2). This spectrometer recorded the spectral signals at wavelengths ranging from 350 nm to 1000 nm. The field of view was 25°. At each measuring unit, the signal of the reference panel was collected first for calibration. The reference panel was a sulfate plate with approximately 98% reflectance with the diameter size of 20 cm. The probe was put vertically down with the distance to the samples about 1 m. In order to avoid accidental error, each observation was repeated for five times. The averaged reflectance curves for each ground features were shown in Figure 3. Preliminary analysis indicated that Ground could be easily excluded by the signals in the near-infrared region. Four plant species, which was termed as high-spectral dissimilarity classes (HSD-classes hereafter), including Osteomeles surotunda, Engelhardtia chrysolepis, Trichophorum subcapitatum, and Helieia kwangtungensis, seem to have significant dissimilarity in spectra from the rest of the other four species. Therefore, it is possible to separate the HSD-classes and Ground from the other species using purely spectra information. The left four species, which was termed as low spectral dissimilarity class (LSD-classes hereafter), are difficult to be separated based on the spectra only. Thus, other geometric or texture information must be considered in the design of the classification schema.

Figure 3: Mean spectral reflectance from field-collected spectra for eight vegetation species and the bare ground.

3.1.2. Customized Multispectral Imaging System

Based on the analysis of the spectral signatures collected on-site as well as general knowledge of the spectral responses from green vegetation, a customized multispectral imaging system was designed for our study objective. The designed imaging system captured eight spectral bands, including three wide channels (by RGB camera) and five multispectral channels, as shown in Figure 4. Standard deviation (std) of reflectance for the selected wavelengths were calculated, resulting in 0.0623, 0.0512, 0.0838, 0.1881, and 0.1965 for 550 nm, 600 nm, 700 nm, 750 nm, and 850 nm, respectively. The imaging system was simplified by looking for suitable commercial cameras on the market. In the end, the camera Sony A 6000 was used directly for capturing the RGB channels. For the five multispectral bands, Sony A 6000 worked great after they were combined with specific band-pass filters, which realized the capturing signal channel at particular wavelengths. The main characteristics of the designed imaging system were shown in Table 2. The transmittance of the filter was 95%. Bandwidth FWHM (full width at half maximum) was 10 nm. The imaging system acquired six bands simultaneously with one exposure, and the time interval was 4 s. The spatial resolution was 12 cm when the flight height was set at 1000 m.

Figure 4: The customized imaging system with multispectral and RGB channels.

Table 2: Characteristics of the designed imaging system.

Parameters
Value

Bandwidth FWHM
10 nm

Exposure interval
4 s

Filter transmittance
95%

Frame image size
4000 × 6000

GND resolution (1000 m alt.)
12 cm

3.2. Remote Sensing Data Acquisition and Preprocessing

3.2.1. Imaging Data Acquisition

Image acquisition was carried out on July 14, 2016, a sunny day, using the customized imaging system mounted on an UAV platform. A ground control station software was used (Mission Planner, http://ardupilot.org/planner) to plan and control the flight. A route planning was designed based on the land terrain in order to balance the efficiency of data acquisition and the requirement of the overlapped area for building a fully qualified image dataset. In the end, images were acquired in a continuous mode for the whole experiment site (an area of about 10 km²), resulting in 50% side overlap and 70% forward overlap at the altitude of 1000 m. The total number of exposures was 721 times. Each exposure produced eight images (including three RGB channels and 5 multispectral channels) which were saved in tiff format.

3.2.2. Image Data Preprocessing

Geometric distortion influenced mapping accuracy and must be diminished. Calibration on the digital numbers to radiance was necessary to extract the spectral reflectance from the plants and the ground. Further, calibrated images were used to generate a mosaicked orthoimage and a DSM map. Overall, the image preprocessing included three steps, optical calibration, radiometric calibration and matching, and image mosaicking.

Optical calibration processing included geometric calibration and removal of vignetting. Geometric calibration determined the lens distortion parameters and principal points of the imaging system. The coefficients of lens distortion included radial and tangential distortions. Radial distortion was the curving effect due to the subtle radial shift. The distortion brought the pincushion effect in value position. Tangential distortion affected a planar shift in the perspective of the image due to the nonalignment of the lens [27]. The calibration toolbox in Matlab was used for geometric calibration [28]. The calibration of vignetting used a look-up table which recorded the vignetting parameters obtained in the laboratory on a per-pixel basis.

Radiometric processing was used to transform DNs (digital numbers) to radiance (Rad) for each image band. Laboratory calibration and empirical line method were adopted to determine the quantum efficiency function and to calculate the necessary parameters [29]. Quantum efficiency defined the fraction of photon flux that contributes to the photocurrent in a photodetector or a pixel. The relationship between DNs and Rad can be expressed in

where is the gray value at a pixel in an image and is the radiance value of the pixel. is the quantum efficiency of an imaging system. In our study, laboratory calibration was used to derive . The experiment determined the parameters on a per-pixel basis by establishing a function between the luminance of standard light source with known radiance and DN of the imaging system. was then used to transform DNs to radiance values.

RGB and multispectral images and point clouds were generated from the calibrated images using Pix4D software [30]. Those calibrated images had enough overlapped regions. Orthoimages were computed based on the RGB and multispectral channels separately and clipped to make a stacked image which covered the study area. The vegetation indices, as listed in Table 3, were derived from the stacked image. The DSM was generated from the point clouds by inverse distance weighing [31] and smoothed by a 5-by-5-pixel averaging filter.

Table 3: Three vegetation indices used in the study.

Vegetation index
Equation

Normalized difference vegetation index (NDVI)
/

Structurally insensitive pigment index (SIPI)
/

Red edge normalized difference vegetation index (RENDVI)
/

Photochemical reflectance index (PRI)
/

3.3. Canopy Segmentation for UAV Imagery

Image segmentation is a fundamental task of image processing that partitions the image by grouping pixels into homogeneous regions or patches. Object-based image segmentation proved to be useful for complicated images [32]. To do so, it is necessary to design a combined homogeneity index (referred to as ) based on which the variance of the index presented in each patch can be evaluated. In this study, the geometry, texture, and spectral features of different plant species appearing in the acquired imagery were fully considered. Given a patch that consists of a set of image pixels, is an index that comprehensively reflects the homogeneity of the patch. A bottom up region-merging algorithm was adopted to realize image segmentation. Region merging starts from a single pixel and looks for its surrounding pixels that must satisfy index if they are merged. The criterion for object merging is to minimize index. In each step, adjacent objects are selected based on the smallest growth of the heterogeneity. A predefined parameter, HA, is given based on a trial-and-error test. If the heterogeneity is higher than HA, the merging process will stop. is defined as [33]

where and are the weights of the spectral index and shape index and the sum of and is 1, is a spectral index which captures the spectra characteristics, and is a shape index reflecting the shape characteristics of a merged object. Spectral feature was defined by weighing each channel number of candidate objects and standard deviation. Shape feature was determined by compactness feature, smooth feature, and the ratio of them. The given object denoted as obj1 and another object denoted as obj2 to be merged to obj1, , and were calculated by (3) and (4):

where is the number of layers (bands) that participated in segmentation and is the weight assigned to band to represent its importance (=1), is the total number of pixels in the merged object, is standard deviation of spectra for pixels contained in an object (x is obj1, obj2, or the merged object), and and are the number of pixels in obj1 and obj2, respectively, before they are merged. indicates the dissimilarity of obj1 and obj2 from the spectral perspective.

In (4), and were the index of the shape compactness and smoothness defined by (5) and (6), respectively.

In (5) and (6), and represent the perimeter and the bounding box of object x (x = obj1, obj2, or the merged object) and the other parameters are the same as in (3).

In this study, we used eCognition Developer (Trimble, 2015) to segment the processed image for the study area. The eight bands (RGB bands and multispectral bands) were involved and equally weighted (wi = 1/8, i = 1, 2, …, 8). A weight of 0.6 was given to the compactness weight (wcomp) and 0.4 was given to the smoothness (wsm), considering that compactness before merging should be given more priority. An equal weight was given to the shape parameter (wsh) and the spectral parameter (wsp). One difficulty then was to decide the threshold (HA) based on which the merging process could be stopped if the calculated H index was greater than the threshold. High HA would result in larger size of patches which could include more than one plant species in the patches. Conversely, too small HA might result in a single FD crown separated into several patches. Therefore, the determination of the threshold was realized by a trial-and-error strategy by setting a range of HA and the one that segmented the area into patches making most FD located in a single patch, judged from field trothing data, was taken as the result. In the end, HA was given to 35, which seemed appropriate in terms of FD grouping which appeared in the segmented patches.

3.4. Design of Image Classification Schema

In order to classify the canopy of FD, a hierarchical classification method, for discriminating nine classes (including Ground), was decided and established based on the analysis of the spectral characteristics of the targeted plant species as well as the typical morphological form of FD canopies. The hierarchical classification method simplified the classification process into three levels. Each level allowed using specific algorithm and rules to classify the desired classes. The flowchart of classification strategy was shown in Figure 5. At the first level, segmented objects (patches) were classified into Ground and non-Ground (vegetation) objects, which was implemented after computing vegetation indices (VIs), indicators of the green density, from remote sensing images. VIs have also been used to separate vegetation species in many other studies [34]. Out of the VIs, normalized difference vegetation index (NDVI) is probably the most popular one which is calculated by comparing the magnitude of spectral reflectance at wavelengths red and near infrared, given by [35]

where is the reflectance at wavelength 850 nm and is the reflectance at wavelength 705 nm or of the red channel in a RGB image.

Figure 5: The flow chart of the classification strategy. The hierarchical classification strategy included three levels. At the last level (level 3), FD canopies were identified from the segmented objects.

In addition to NDVI, there are other vegetation indices (VIs) which were proposed for the similar purpose (Table 3). Those VIs were formulated by taking the characteristics of spectral reflectance of green vegetation from different aspects. For example, the spectral reflectance of vegetation usually presents a green peak located at wavelength 550 nm in the spectrum and strong spectral absorption in the blue and red regions of the spectrum. Furthermore, the red edge effect from vegetation is typically observed for vegetation reflectance in that a rapid increase of reflectance occurs starting at the red region (around 700 nm in the spectrum) to the near-infrared region (usually at 750 nm or 850 nm). Those information provided the basis for designing the spectral channels of our customized multispectral imaging system on UAVs.

A simple filter based on the processed VIs was applied to exclude ground patches. At level 2, plant species were classified as four HSD-classes and an LSD-class. The HSD-classes included four plant species, namely, Osteomeles surotunda, Engelhardtia chrysolepis, Trichophorum subcapitatum, and Helieiakwangtungensis. The left plant species, including FD, were assigned as a combined LSD-class which could not be separated by purely spectral information due to low dissimilarity in spectra. At this level, SVM was used to classify the 5 classes (four HSD-classes and a combined LSD-class) using the spectral features (the eight bands and vegetation indices) only. The mean spectra value and vegetation indices (NDVI, SIPI, RENDVI, and PRI, in Table 3) of all the pixels located within each patch were calculated and used in the SVM classifier. At level 3, the combined LSD-class was further divided into four plant species, including FD, based on the geometric feature, terrain feature, and texture features by another SVM classifier. Here, SVM, a nonparameter-supervised classifier, was chosen as the classifier for both levels 2 and 3 because it has advantage achieving good classification results without sensitivity to sample size [36].

Terrain properties, geometric properties, and texture characteristics were used at level 3 to identify patches covered by FD canopies. Terrain properties, including slope, aspect, and DSM, were input to the classifier because the FD was observed to appear mainly at medium terrain height and southern slope. Because we did not have DEM data with spatial resolution comparable to the processed images, a smoothed DSM was used to process slope and aspect data.

Texture and geometric information denoted the shape of the targeted patch. In this study, Gray-level Co-occurrence Matrix (GLCM) was used to represent texture [37]. Four GLCM indices were included in classification (Table 4). For geometric properties, three indices were formulated and used in the classifier (Table 4).

Table 4: The equation and description of index.

Type
Index
Equation and description of index

Texture
GLCM_homogeneity

GLCM_contrast

GLCM_dissimilarity

GLCM_Entropy

Geometry
Shape index

Density

Pi,j is the probability of a gray level pair, i and j, in the GLCM defined by a position operator, and I and J are the maximum values given in each dimension of GLCM, respectively. e and A were the perimeter and area, respectively, of the targeted patch, n was the number of pixels, and and were the width and length of the bounding box of the patch.

Standard radial Gaussian was used as the kernel function for the SVM classifier. The main parameters to be decided in the classifier included (the penalty parameter which stands for an error term) and (controlling training efficiency). As there was no prior knowledge which and were acceptable, they were determined based on a recursive loop with different combinations. During the process, a good pair of and could be identified [38].

Field trip was arranged to collect ground truth data for model training and result validation. The date of ground data collection was conducted on July 21, 2016, one week after the UAV flight. A GPS device (Trimble R2 GNSS) was used to record the coordinate of FD trunk location, and the general shape for each recorded FD canopy was drafted. RTK (Real-Time Kinematic) was set as the operating mode, producing an absolute positioning accuracy within 1-2 cm. Two thirds of the recordings were used for model training purposes, and the rest were used for result validation. eCognition Developer was used to implement the hierarchical classification strategy. The segmented patches, a DSM layer, slope layer, and aspect layer participated in the classification process. Spectra, geometric, and texture indices were calculated during the classification process by customized functions.

4. Result

4.1. UAV-Collected Imagery

The orthoimage processed from the spectra collected by UAV sensors showed clear differentiation, in terms of spectral, texture, and geometry, in the distribution of different plant species and the bare ground (Figure 6). The DSM, ranging from 214 m to 302 m, showed the spatial variations of surface height. Such height reflects the changes over the space in both DEM and the plant species. When combined with other topography information such as the slope and aspect, better understanding of the preferred locations of each plant species could be analyzed. The acquired images from UAV allowed producing various vegetation indices which could be helpful data input for classifying the plant species and the ground. For example, red edge normalized difference vegetation index (RENDVI) processed from the multispectral band at 775 nm and red band of RGB showed clearly the differentiated patterns in the distribution of the bare ground and plant species. One advantage of applying UAV platform is that the high-spatial resolution of those images provided enough detail information for building advanced image classifiers, which otherwise is impossible for other satellite platforms.

(a)

(b)

(c)

Figure 6: Example of data products computed from the spectra collected by the image system on UAVs. (a) Mosaicked orthoimage, (b) the smoothed DSM, and (c) NDVI index. One of the vegetation indices is used to distinguish vegetation area and soil area and to input to the SVM classifier at level 2 in the classification schema.

4.2. Classification Result of FD Canopy

SVM, using radial Gaussian used as the kernel function, was used to classify HSD-classes and LSD-classes. At level 2, and were determined as (5, 10) for the SVM classifier. At level 3, and were determined as (2, 10) for the classifier. The different parameter settings allowed optimized classification result at different classification steps. The final classification result, denoted as FD species, non-FD species, and Ground, for the study site was shown in Figure 7(a).

(a)

(b)

Figure 7: Classification result for the segmented patches of the image collected on an UAV platform. (a) Overall classification map for the study site. (b) The corresponding DSM contour line showing the cliff area (steep terrain) in the black rectangle where FD densely populated.

The result indicated that the FD species only covered very small part of the area, which was in agreement with the reality. Though FD could spread over the whole study area, the spatial distribution of FD tended to cluster and was dominantly located close to the bare ground and the cliff (Figure 7(b)). This finding may serve as important clue for local botanists to evaluate the FD distribution.

4.3. Accuracy Test

The GPS recordings from the field trip were used to validate the classification result. Those points that were recorded as FD were overlaid on top of the classified segments (Figure 8). The locations of FD species generally matched well with the segments assigned to FD which, after merged with adjacent ones, had an area ranging from 10 m² to 32 m² (Figure 8(b)), comparable to the statistics of the canopy size from field samples (Table 1). Classification results by the proposed hierarchical classifier showed that the overall classification accuracy at levels 2 and 3 reached 87.4% and 75.3%, respectively (Table 5). For the SVM classifier at level 2, there were 17.6% cases from HSD-classes that were mistakenly assigned to LSD classes, though most LSD-classes were labeled correctly (accuracy = 92.4%). There were 331 cases classified as LSD-class, including 53 cases which were actually HSD-class. Out of the 52 FD samples in the testing set, 6 cases were wrongly mapped as HSD-class (Table 5(a)) and the remaining 46, along with randomly selected 51 non-FD samples, were entered into the next SVM classifier at level 3 for testing. The result indicated that the SVM classifier at level 3 performed slightly worse than that at level 2, mainly due to the fact that almost one-third of non-FD samples were wrongly classified as FD. The hierarchical classification powered by SVM classifier for FD species reached an overall accuracy of 76.9%, a reasonable result considering the complex terrain and multiple plant species.

(a)

(b)

Figure 8: Validation of the classification result based on field trip. Yellow points are GPS recordings showing the FD locations. (a) Overall classification map overlaid by FD locations from GPS recordings and (b) detail of FD labeled segments/patches (in red) and the ground truth/GPS points (yellow dots).

Table 5: Accuracy assessment of the FD classification for the segmented patches.

(a) Confusion matrix at level 2

Reference class
Map class
Producer’s accuracy

HSD-class
LSD-class

HSD-classes
248
53
82.4%

LSD-classes
23 (6)
278
92.4%

User’s accuracy
91.5%
84.0%
87.4%

Number of FD samples located in the cell. Kappa = 0.748.

(b) Confusion matrix at level 3

Reference class
Map class
Producer’s accuracy

FD
Non-FD

FD
40
6
87.0%

Non-FD
18
33
64.7%

Accuracy
67.0%
84.7%
75.3%

Total number of FD: 52; number of FD correctly labeled: 40; accuracy for FD extraction: 76.9; Kappa = 0.535.

5. Discussion

Remote sensing image acquisition by sensors mounted on UAV platforms has been widely used in the classification of forests [39]. There are several advantages by applying UAV-based remote sensing. Firstly, UAV platforms can fly at low-altitude and thus images having high-spatial resolution can be acquired. Low-altitude remote sensing decrease the distance between the target objects and the sensors and thus the quantity of acquired images is improved. Because of the high-spatial resolution, extracting FD from UAV images also has less impact from mixed pixel issue. Furthermore, the collected images are less affected by the atmosphere and luminous impact. Secondly, the flight route and flight height can be customized based on the actual need and thus UAV platforms provide more flexibility for data acquisition. Sensors mounted on UAVs can be used to map most places on the land surface, and setting an appropriate flight height can balance the coverage and spatial resolution of the acquired data; thus, the efficiency of data collection can be improved because the flight range can be determined as required. Thirdly, UAV platforms can support any cameras designed to collect images for particular spectral wavelengths. As the study conducted here, RGB channels and 5 multispectral channels were designed by using particular band-pass filters on the commonly commercial cameras. By collecting and analyzing the diversity of spectra, imaging systems can be customized to collect spectra based on the mapping purposes. However, unlike satellite remote sensing platforms which can cover a large area in one glance, UAVs are usually suitable for mapping small area only due to the limited viewing angles and relatively low flight height. Further, the demand for georegistering and mosaicing multiple images collected at adjacent flight routes also introduces extra input for remote sensing by UAVs. Possible solutions may integrate UAV images with other imagery products from satellite remote sensing.

In our study, we proposed a hierarchical classification approach after the analysis of the spectral features of the plant species. Such strategy allowed us to separate different classes in a progressive manner and proved to be suitable for extracting FD from the acquired imagery. For example, the bare ground could be easily excluded based on the computed VIs simply by using threshold filters, namely, segmented patches with VIs lower than predefined thresholds were labeled as Ground. SVM has been proved effective in classifying remote sensing imagery when a limited number of samples are available [40]. Several studies have been using the hierarchical classification method and SVM algorithm to classify specific plants [41]. For example, SVM was used as a binary classifier to detect seminatural habitats [42]. The combination of a hierarchical classification schema and SVM algorithm has three main advantages. Firstly, the process for building such classification structure is readily available in several existing software packages such as eCognition and Matlab [43]. After necessary datasets, including the orthoimages, DSM, and segmented patches in this study, are ready, the classifier could be quickly setup with those input datasets. The main work of building the classifier involved establishment of the rules to label the input patches based on related data (e.g., DSM and the orthoimages in this work). The second advantage is that the classification schema can simplify the classification process. The hierarchical classification approach separates the candidate classes into several levels, and thus, the classification can be divided into a few subobjectives. When combined by appropriate algorithms at each level, improving the overall classification accuracy can be realized. Lastly, the hierarchical classification combined with SVM is efficient for extraction of a single class (e.g., FD in this study) from multiple class labels. The basic SVM classifier only supports binary classification issues, namely, 0-1 cases. SVM also supports classifying multiple classes called K-class classifier which split the process into k steps [44]. In the current study, SVM was applied to classify LSD-class and non-LSD-class at level 2 and to classify FD and non-FD at level 3, both of which were binary classification, though there were multiple plant species involved. In fact, there was no need to classify the species within HSD-classes or to classify each species within the LSD-classes, considering the very specific objective of the study.

There are two steps which may introduce uncertainties in the ultimate classification result based on the proposed hierarchical classification schema; they are image segmentation and classification of segmented patches. First, detection of FD canopy is highly dependent on the result of image segmentation which was performed based on a bottom up region-merging algorithm in this study. A homogeneity index, , was used to control the process of object merging. It is clear that the result of image classification depends on the criteria adopted for the index. Factors involving the spectra and geometry of produced patches were considered in computing the index. The weight assignment for each of the factors is critically important. In addition, the threshold value (HA) in the study determined the process of segmentation: either continuing the merging process or stopping it. The factors and their weighing schema as well as the threshold all determine the size of the segmented patches. Though we used a trial-and-error strategy to determine HA, we observed that there was still a small part of the patches (which were FD canopy tested by field GPS recordings) which were either too small or too big, resulting in mislabeling in the next hierarchical classification step by the SVM classifier. The next step, hierarchical classification for the segmented patches, also introduced certain uncertainties. There were three levels in the proposed hierarchical classification schema. We observed that there was very few mislabeled cases at the first level. However, the SVM classifiers at levels 2 and 3 did produce mislabeled patches, including cases that either non-FD patch was labeled as FD or FD patch was labeled otherwise. While SVM is regarded as an advanced classifier that has been successfully applied in many image classification studies [45], the data quality used to train the model and the data quality in the testing samples can surely have much impact on the result. Spectra as well as spectra-derived vegetation indices were taken as input in the SVM classifier at the second level. It was pointed out that mixed spectra in remote sensing imagery would inevitably lead to side effect on the classification result [46]. Still, though the study took terrain properties, geometric properties, and texture characteristics in the SVM classifier at the third level, certain confusion between FD and other plant species might also exist, making the wrong assignment of labels to some segmented patches. Terrain properties were probably the most important factors that should be taken to differentiate LSD-classes, as FD are distributed mainly close to cliff area where the slope index played a vital role in the classifier. Other factors related to geometry and texture could be examined and selected in more detail for the SVM classifier. Nevertheless, our work demonstrated the effectiveness of applying the designed object-based image segmentation and hierarchical classification schema in the extraction of FD canopy from remotely sensed images on the UAV platform.

6. Conclusion

Recent applications of UAVs make the equipment rapidly evolve into easy-to-use platforms for sensors deployable to acquire fine-scale vegetation data over large areas in a timely and cost-effective way. Customized imaging systems mounted on UAV platforms possess the capability and flexibility of collecting low-altitude remote sensing data for classifying land cover with plant species. In this work, we designed a customized imaging system mounted on UAV and acquired RGB and multispectral images covering part of Danxia Mountain where most of Firmiana danxiaensis (FD), a rare and endangered plant in China, are distributed. The identification of FD distribution has been desired by local practitioners as well as researchers in academia for a long time. However, large-scale FD mapping, based on traditional field survey, was called a halt due to the high cost and even safety issues associated with the work. With advances in remote sensing technology, especially UAV applications, this study tried to test the possibility of mapping FD through the customized imaging system mounted on an UAV platform. To this end, a limited number of field spectra collection were conducted first to help analyze the characteristic of FD and other plant species, which enabled the design of a reasonable hierarchical classification schema for the acquired UAV images. Various image products, including DSM and vegetation indices, were derived from the imaging system and involved in the classification process. Support vector machine was integrated to achieve good extraction of FD canopies from the UAV imagery. Spectra, terrain, geometric, and texture properties were fed into the proposed classifier so that the plant species could be separated. The accuracy of detection was evaluated by the ground truth GPS recordings. The study proved that UAV could be a promising platform to conduct FD surveying and mapping, which provides fundamental data to protect the rare plant species. Future work may focus on fusing UAV-based images with other products from satellite remote sensing so that the efficiency for mapping FD could be further improved.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Acknowledgments

This research is funded by the National Natural Science Foundation of China (no. 41771481, no. 41671451, and no. 41701524), the Fundamental Research Funds for the Central Universities and China Postdoctoral Science Foundation (Grant no. 2017M621530), and Shanghai Municipal Commission of Economy and Informatization (no. JMRH-2018-1016). The authors thank Tongfan Surveying Engineering and Technology Co. Ltd. and Aerospace ShuWei High Tech. Co. Ltd. for their help in data measurements.

References

X. Chen and B. L. Li, “Spatial distribution of forest biome energetics in China,” Forestry, vol. 78, no. 5, pp. 461–469, 2005.

C. Toth and G. Jóźków, “Remote sensing platforms and sensors: a survey,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 115, pp. 22–36, 2016.

M. Fagan, R. DeFries, S. Sesnie et al., “Mapping species composition of forests and tree plantations in northeastern Costa Rica with an integration of hyperspectral and multitemporal Landsat imagery,” Remote Sensing, vol. 7, no. 5, pp. 5660–5696, 2015.

S. Zaman and M. Katoh, “Assessment of forest covers change in tropical moist deciduous forest in Thakurgaon, Bangladesh using ALOS data (multipurpose forest management),” Journal of Forest Planning, vol. 16, pp. 285–292, 2017.

Y. T. Mustafa and H. N. Habeeb, “Object based technique for delineating and mapping 15 tree species using VHR WorldView-2 imagery,” in Proceedings Volume 9239, Remote Sensing for Agriculture, Ecosystems, and Hydrology XVI, 92390G, Amsterdam, Netherlands, October 2014.

M. Ozdogan, “A practical and automated approach to large area forest disturbance mapping with remote sensing,” PLoS One, vol. 9, no. 4, article e78438, 2014.

I. Colomina and P. Molina, “Unmanned aerial systems for photogrammetry and remote sensing: a review,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 92, pp. 79–97, 2014.

B. Lu, Y. He, and H. Liu, “Investigating species composition in a temperate grassland using unmanned aerial vehicle-acquired imagery,” in 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA), pp. 107–111, Guangzhou, China, July 2016.

O. Nevalainen, E. Honkavaara, S. Tuominen et al., “Individual tree detection and classification with UAV-based photogrammetric point clouds and hyperspectral imaging,” Remote Sensing, vol. 9, no. 3, p. 185, 2017.

R. Dunford, K. Michel, M. Gagnage, H. Piégay, and M. L. Trémelo, “Potential and constraints of unmanned aerial vehicle technology for the characterization of Mediterranean riparian forest,” International Journal of Remote Sensing, vol. 30, no. 19, pp. 4915–4935, 2009.

D. J. Hill, C. Tarasoff, G. E. Whitworth, J. Baron, J. L. Bradshaw, and J. S. Church, “Utility of unmanned aerial vehicles for mapping invasive plant species: a case study on yellow flag iris (Iris pseudacorus L.),” International Journal of Remote Sensing, vol. 38, no. 8–10, pp. 1–23, 2017.

J. W. Karl, J. K. Gillan, N. N. Barger, J. E. Herrick, and M. C. Duniway, “Interpretation of high-resolution imagery for detecting vegetation cover composition change after fuels reduction treatments in woodlands,” Ecological Indicators, vol. 45, pp. 570–578, 2014.

K. Lim, P. Treitz, M. Wulder, B. St-Onge, and M. Flood, “LiDAR remote sensing of forest structure,” Progress in Physical Geography: Earth and Environment, vol. 27, no. 1, pp. 88–106, 2003.

M. Dalponte, L. Bruzzone, and D. Gianelle, “Fusion of hyperspectral and LIDAR remote sensing data for classification of complex forest areas,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 5, pp. 1416–1427, 2008.

J. Zhang, J. Hu, J. Lian, Z. Fan, X. Ouyang, and W. Ye, “Seeing the forest from drones: testing the potential of lightweight drones as a tool for long-term forest monitoring,” Biological Conservation, vol. 198, pp. 60–69, 2016.

J. Albetis, S. Duthoit, F. Guttler et al., “Detection of Flavescence dorée grapevine disease using unmanned aerial vehicle (UAV) multispectral imagery,” Remote Sensing, vol. 9, no. 4, p. 308, 2017.

R. Näsi, E. Honkavaara, P. Lyytikäinen-Saarenmaa et al., “Using UAV-based photogrammetry and hyperspectral imaging for mapping bark beetle damage at tree-level,” Remote Sensing, vol. 7, no. 11, pp. 15467–15493, 2015.

S. F. Chen, M. W. Li, H. J. Jing et al., “De novo transcriptome assembly in Firmiana danxiaensis, a tree species endemic to the Danxia landform,” PLoS One, vol. 10, no. 10, article e0139373, 2015.

A. Michez, H. Piégay, L. Jonathan, H. Claessens, and P. Lejeune, “Mapping of riparian invasive species with supervised classification of unmanned aerial system (UAS) imagery,” International Journal of Applied Earth Observation & Geoinformation, vol. 44, pp. 88–94, 2016.

S.-K. Kwon, H.-S. Jung, W.-K. Baek, and D. Kim, “Classification of forest vertical structure in South Korea from aerial orthophoto and Lidar data using an artificial neural network,” Applied Sciences, vol. 7, no. 10, p. 1046, 2017.

A. Frick, P. Steffenhagen, S. Zerbe, T. Timmermann, and K. Schulz, “Monitoring of the vegetation composition in rewetted peatland with iterative decision tree classification of satellite imagery,” Photogrammetrie - Fernerkundung - Geoinformation, vol. 2011, no. 3, pp. 109–122, 2011.

H. Han, M. Ma, and X. Wang, “Classifying cropping area of middle Heihe River Basin in China using multitemporal normalized difference vegetation index data,” Journal of Applied Remote Sensing, vol. 8, no. 1, article 083654, 2014.

G. M. Foody and A. Mathur, “Toward intelligent training of supervised image classifications: directing training data acquisition for SVM classification,” Remote Sensing of Environment, vol. 93, no. 1-2, pp. 107–117, 2004.

G. Mountrakis, J. Im, and C. Ogole, “Support vector machines in remote sensing: a review,” ISPRS Journal of Photogrammetry & Remote Sensing, vol. 66, no. 3, pp. 247–259, 2011.

S. Wang and Y. Xie, China Species Red List, vol. 1, Higher Education Press, Beijing, China, 2004.

J. Ouyang, H. Peng, X. Luo, Z. Chen, A. Zhang, and Y. Ma, “Environmental features of the micro-landforms of the spatial distribution of the national rare species of Firmiana danxiaensis on the Danxiashan Mountain,” Scientia Geographica Sinica, vol. 37, no. 10, pp. 1585–1592, 2017.

J. Kelcey and A. Lucieer, “Sensor correction of a 6-band multispectral imaging sensor for UAV remote sensing,” Remote Sensing, vol. 4, no. 5, pp. 1462–1493, 2012.

J. Y. Bouguet, “Camera Calibration Toolbox for Matlab,” High Speed Vision System & Object Tracking, 2013, http://www.vision.caltech.edu/bouguetj/calib_doc/index.html.

D. Olsen, C. Dou, X. Zhang, L. Hu, H. Kim, and E. Hildum, “Radiometric Calibration for AgCam,” Remote Sensing, vol. 2, no. 2, pp. 464–477, 2010.

F. J. Mesas-Carrascosa, J. Torres-Sánchez, I. Clavero-Rumbao et al., “Assessing optimal flight parameters for generating accurate multispectral orthomosaicks by UAV to support site-specific crop management,” Remote Sensing, vol. 7, no. 10, pp. 12793–12814, 2015.

R. Buffat, T. Sarjakoski, M. Santos, and L. Sarjakoski, “Feature-aware surface interpolation of rooftops using low-density Lidar data for photovoltaic applications,” in Geospatial Data in a Changing World, Lecture Notes in Geoinformation and Cartography, Springer, Cham, 2016.

V. Dey, Y. Zhang, and M. Zhong, “A review on image segmentation techniques with remote sensing perspective,” Pattern Recognition, vol. 38, no. 9, pp. 1277–1294, 2010.

M. Baatz and A. Schäpe, “Multiresolution segmentation: an optimization approach for high quality multi-scale image segmentation,” in Angewandte Geographische Informations-Verarbeitung XII, J. Strobl, T. Blaschke, and G. Griesbner, Eds., pp. 12–23, Wichmann Verlag, Karlsruhe, Germany, 2000.

R. M. Igamberdiev, R. Bill, H. Schubert, and B. Lennartz, “Analysis of cross-seasonal spectral response from kettle holes: application of remote sensing techniques for chlorophyll estimation,” Remote Sensing, vol. 4, no. 11, pp. 3481–3500, 2012.

R. S. Defries and J. R. G. Townshend, “NDVI-derived land cover classifications at a global scale,” International Journal of Remote Sensing, vol. 15, no. 17, pp. 3567–3586, 1994.

Z. Sha and Y. Bai, “Mapping grassland vegetation cover based on support vector machine and association rules,” in 2013 Ninth International Conference on Natural Computation (ICNC), pp. 44–49, Shenyang, China, July 2014.

Q. Yu, P. Gong, N. Clinton, G. Biging, M. Kelly, and D. Schirokauer, “Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery,” Photogrammetric Engineering & Remote Sensing, vol. 72, no. 7, pp. 799–811, 2006.

D. Giveki, M. A. Soltanshahi, and G. A. Montazer, “A new image feature descriptor for content based image retrieval using scale invariant feature transform and local derivative pattern,” Optik - International Journal for Light and Electron Optics, vol. 131, pp. 242–254, 2017.

Y. Yuan and X. Hu, “Random forest and objected-based classification for forest pest extraction from Uav aerial imagery,” in ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B1, pp. 1093–1098, Prague, Czech Republic, July 2016.

L. Bruzzone, M. Chi, and M. Marconcini, “A novel transductive SVM for semisupervised classification of remote-sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 11, pp. 3363–3373, 2006.

B. Lu and Y. He, “Species classification using unmanned aerial vehicle (UAV)-acquired high spatial resolution imagery in a heterogeneous grassland,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 128, pp. 73–85, 2017.

C. Schuster, T. Schmidt, C. Conrad, B. Kleinschmit, and M. Förster, “Grassland habitat mapping by intra-annual time series analysis – comparison of RapidEye and TerraSAR-X satellite data,” International Journal of Applied Earth Observation and Geoinformation, vol. 34, no. 34, pp. 25–34, 2015.

M. J. Huang, Development of Knowledge-based Classification System: Urban-feature Extraction Using Aerial Imagery with Airborne LiDAR Data, VDM Verlag Dr. Müller, 2009, http://www.vdm-verlag.de.

Z. Li, S. Tang, and S. Yan, “Multi-class SVM classifier based on pairwise coupling,” in Pattern Recognition with Support Vector Machines, vol 2388, S. W. Lee and A. Verri, Eds., Lecture Notes in Computer Science, pp. 321–333, Springer, Berlin, Heidelberg.

P. Du, J. Xia, W. Zhang, K. Tan, Y. Liu, and S. Liu, “Multiple classifier system for remote sensing image classification: a review,” Sensors, vol. 12, no. 4, pp. 4764–4792, 2012.

C. Hecker, M. van der Meijde, H. van der Werff, and F. D. van der Meer, “Assessing the influence of reference spectra on synthetic SAM classification results,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 12, pp. 4162–4172, 2008.