Our goal in this work is to demonstrate that detectors behave differently for different images and targets and to propose a novel approach to proper detector selection. To choose the algorithm, we analyze image statistics, the target signature, and the target's physical size, but we do not need any type of ground truth. We demonstrate our ability to evaluate detectors and find the best settings for their free parameters by comparing our results using the following stochastic algorithms for target detection: the constrained energy minimization (CEM), generalized likelihood ratio test (GLRT), and adaptive coherence estimator (ACE) algorithms. We test our concepts by using the dataset and scoring methodology of the Rochester Institute of Technology (RIT) Target Detection Blind Test project. The results show that our concept correctly ranks algorithms for the particular images and targets including in the RIT dataset.
Ideally, one would like to choose a hyperspectral detection algorithm for use in a particular scenario with the assurance that it would be “optimal,” that is, that the type of algorithm to be used and its free parameters would be optimized for the particular task for which it is being considered. Of course, in such cases, the complexity of realworld scenarios and the difficulties of predicting the exact target signature
An inherent weakness of the RBTA method is its assumption that subpixel targets will each be contained within a single pixel. In light of our recent work [
Sections
Sections
Manolakis et al. [
To estimate detector performance, Rotman and BarTal proposed a multistep process that begins with an analysis of the unmodified reflectance image that is available in the website without any embedded targets. (We assume that ideally no targets are present in the datacube being analyzed; if one were present, it would slightly distort the histogram of the image. We trust that such a distortion will not disturb the overall analysis of the image statistics). The algorithm being tested is evaluated for each pixel, and the results are summarized in what we call a falsealarm histogram. Next we embed targets into every pixel and evaluate each of the algorithms. This is done independently for each pixel (rather than simultaneously) so that surrounding pixels are not changed prior to algorithm evaluation. The results are arranged in a target detection histogram. Each histogram (falsealarm and target detection) is then normalized; a variable threshold is set and the area of the falsealarm and target histograms to the right of the threshold are measured. For any particular threshold, a pair of
We note that the target implantation mechanism as given here has ignored several possibly significant effects which would affect the values found of PD. In particular, the target spectrum is a nearly noiseless lab spectrum that does not have the same artifacts, noise, and degradation as the real imagery. Additionally, this approach assumes the data has been perfectly atmospherically compensated by RIT’s algorithm, which is not necessarily true. In our opinion, this seems to limit the use of our method rather than to invalidate it. Since the atmospheric conditions at the time of the measurement were not known, we cannot implant atmospherically corrected signatures or validate the reflectance dataset that is available in RIT website. Instead, we are testing the response of the algorithms to an implanted nonatmospherically corrected target which has been substituted in the reflectance dataset as described above; in each examined pixel, the fraction of the laboratory signature replaced the fraction of the background signal. While inaccurate atmospheric correction may result in an unknown decrease in the target detection, we note that the final comparisons are for variations in algorithm selection for a given target signature. The method should not be used to calculate absolute values for the probability of detection of a particular target which indeed has been altered by atmospheric and other effects. Rather, we are attempting to determine which algorithms will have a superior probability of detecting a target of this type in the scenario. Future work should include a quantitative determination to what degree atmospheric effects change the ranking of different algorithms.
This methodology can be used for the following reasons: as a rule, the ROC curve, which are generated tend to have probabilities of detection which range from 0 to 1; the value of probabilities of false alarm, on the other hand, vary from 0 to some chosen threshold
Now, the exact distribution of the background pixels is crucial for the analysis of our detection algorithms; it will indeed be the exceptional pixels in the tail of the distribution which will determine the ROC curve. However, since the probability of detection is being determined by the entire P_{D} scale from 0 to 1, all pixels contribute. In other words, the target detection scheme in this paper is extremely sensitive to a few false alarms; it is much less sensitive to a few pixels with missed “synthetic” target signatures. As such, subtle effects affecting the exact form of the target signature
To summarize, ROC curve evaluation entails the following steps as demonstrated in Figure
RBTA flow chart.
In many cases, it is convenient to scale the matched filter such that it has a value of 1 when the target signature fills the pixel being examined. This scaling can be achieved by normalizing the matched filter to its value when operating on the designated target spectrum:
Manolakis and his group [
The ACE algorithm, a variation of the GLRT algorithm, is expressed as
In the context of target detection, the sign of
The corresponding ACE algorithm for target detection, also a variation of the GLRT algorithm, is expressed as
In Figure
Improving target detection involved replacing the global mean with the local mean. Using the local mean is definitely double edged: on one hand, we would expect that the closer the points used to evaluate the background are to the suspected target, the more likely it is that the estimate will be accurate. On the other hand, the noise in the estimate will decrease given more points entering into the estimation, assuming that the background is stationary and the noise is linearly added to the background and independent thereof. Our empirical experience confirmed by several studies (
We note that we are not dealing here with a “local” covariance matrix which would change when evaluating each pixel in the image. Rather, we use the same covariance matrix throughout the image; it will simply be based on the difference of the sample pixels and their “local” background.
Since we are dealing with a subpixel target, which in the physical domain can affect only pixels in a limited spatial area surrounding the center of the target, we used the eight nearest neighbors approach to estimate the value of the test pixels. The CEM algorithm does not use the mean and will therefore be unaffected by the above changes. The GLRT can be improved as follows:
Segmentation [
We tested our algorithms on the online reflectance data sets and the hyperspectral data collected over Cooke City. The Cooke City imagery was acquired on 4 July 2006 using the HyMap VNIR/SWIR sensor with 126 spectral bands. Two hyperspectral scenes are provided with the dataset, one intended to be used for development and testing (the “Self Test” scene, where the positions of some targets are known) and the other intended to be used for detection performance evaluation (the “Blind Test” scene, where the position of targets is unknown). The data was corrected for atmospheric effects and available in the website but the exact atmospheric condition and the atmospheric correction algorithm are not available in the website and we assume that the reflectance dataset is good but not perfect. In Figure
Falsecolor RGB of the Cooke City imagery.
The target signatures, used both in the algorithm for detection and in the implantation of the synthetic targets in the RBTA method were laboratory measured and in reflectance units. The GSD is approximately 3 m. In Figure
Spectral signatures of the targets that are present in the Blind test image
The list of all targets is presented in Table
Targets description.
Target ID  Target description  size (m^{2}) No. 1  size (m^{2}) No. 2  Self test ground truth  Blind test ground truth 

F1  Red cotton fabric panel  3 × 3  N/A  Yes  No 
F2  Yellow nylon fabric panel  3 × 3  N/A  Yes  No 
F3  Blue cotton fabric panel  2 × 2  1 × 1  Yes  No 
F4  Red nylon fabric panel  2 × 2  1 × 1  Yes  No 
F5  Maroon nylon fabric panel  2 × 2  1 × 1  No  Web score 
F6  Gray nylon fabric panel  2 × 2  1 × 1  No  Web score 
F7  Green cotton fabric panel  2 × 2  1 × 1  No  Web score 
V1  Chevy Blazer, green  4 × 2  N/A  Yes  Web score 
V2  Toyota T100, white with black plastic liner  3 × 1.7  N/A  Yes  Web score 
V3  Subaru GL Wagon, Red  4.5 × 1.6  N/A  Yes  Web score 
The general form for local target detection as described in Section
For the case in which the PUT (pixel under testing)
Let us define the scalar C as−
Therefore, when
Assuming that the data is normally distributed,
When imaging, the target can often fall across several pixels even if its total size is only a single pixel; we will call this effect pixel phasing even though it is a natural consequence of imaging system quantization. The pixel phasing effect can be demonstrated by a target one pixel in size, the imaging of which leads to pixel phasing registration defined by
Pixel phasing schema.
Simple case
Pixel phasing case
We obtain the following:
We now evaluate the terms
The GLRT result now becomes
For the case in which
In this model, the complete lack of ACE degradation as a function of pixel phasing may explain why ACE is a more robust detector than GLRT in many test cases, as noted in the literature [
The difficult task of synthesizing a synthetic image to help predict which algorithm to select is simplified and detector selection is facilitated if we synthesize only the target signature of our real image. Suppose we want to determine the proper detector for a specific target. We have already selected our method (e.g., CEM, GLRT, or ACE), and now we want to select the size of the local window. One approach is to assume that the best size for the local window is that under which the PUT value can be predicted with minimum error visàvis the real PUT (in which we normalize each band by the mean values of the pixels in that band).
The approach outlined above depends only on the background image, not on the target signature, and it entails two assumptions: first, estimating signature values will improve our detector results independent of the different target signatures and second, the target has no effect on its neighbors. Address these assumptions in the following sections.
The implementation of RBTA, which depends on our ability to implant realistic signals into backgrounds and measure detector response, should be done carefully. We cannot expect the real signature to be identical to a library signature, but we can hope for a high level of similarity. The low percentage of the target signature that actually enters any particular pixel is demonstrated in Figure
As a rule, to test and challenge our algorithms by examining the area under the ROC curve, we need to test targets which neither “saturate” the ROC curve (with a probability of detection close to one with no false alarms detected) nor result in a “diagonal” ROC curve (in which the probability of detection equals the probability of false alarms. As the allowable false alarm rate decreases, the strength of our synthetic implanted target would need to increase; if we know what the acceptable false alarm rate is, we can select the target percent that will demonstrate the dynamic range around this rate and get results for our detectors (Figure
RBTA results for different size of local windows.
CEM results (2D and 3D) for target with pixel size.
For the values found experimentally for
In terms of real data, we must expect each target to affect more than one pixel even if its total physical size is at the subpixel level. A discussion of this point follows below and leads to improvement of the RBT algorithm.
As will be discussed in Sections
If we take into account only the spatial sampling effect, we can estimate the percent of pixel area partially occupied by the target. Notice that even targets of subpixel size often spread over neighboring pixels (Figure
Demonstration for target size of 4/3 over 2/3 with origin at (0.3,0.2) and different rotation angle.
Put formally, the percent of pixel area covered as a function of target size, target location, and target orientation is
Variable demonstration.
The expected value
Pixel covered as function of target size for
Calculating (
Pixel coverage as a function of target size.
In the graphs depicting pixel coverage as a function of physical target size, the
The PSF effect, present in any optical system, is not always known. Let us assume that the PSF is a typical, rotationally symmetric Gaussian filter of size
Figure
Synthetic spread emulation.
A comparison of Figures
IRBTA results for different type of detectors.
For our next stage of proof of concept, we need to compare real detection performances to these simulated results. We obtain this by using the selftest dataset and by submitting our algorithms to the RIT target detection blind test and comparing the ranking of our algorithms for each of the target signatures available in the set; we can then see if the IRBTA predictions of the preferred algorithms are true.
Detection algorithm performances are given according to the RIT target detection methodology applied to the aforementioned Cooke City hyperspectral dataset. The score is based on a comparison of the values given to the background pixels in the image to the value given to the target. The target value is defined as the maximum value given the pixels in the target area; the metric then counts how many pixels there are in the overall image greater than or equal to the target value. Since the threshold needed to detect the target would have to be less than or equal to the target value, all points above this value are false alarms. The score given for any algorithm/target combination would thus be the number of pixels above the target value. Perfect detection would equal the value 1, since the only pixel equal or above the target value would be the target itself; no false alarms are present.
In Tables
Results of global methods.
Global  

Target ID  CEM  GLRT  ACE  
Selftest  F1  15  13 


F2 




F3  1 m^{2} 


 
2 m^{2}  30  28 


F4  1 m^{2}  208  207 


2 m^{2}  23  24 


 
Blind test  F5  1 m^{2}  163  156 

2 m^{2}  19  19 


F6  1 m^{2}  75  76 


2 m^{2}  13  13 


F7  1 m^{2} 


 
2 m^{2}  315  318 


 
Selftest  V1  324  321 


V2 


 
V3 




 
Blind test  V1  422  428 


V2 


 
V3 



Results of local GLRT for differentsized windows.
GLRT local  

Target ID  3 × 3  5 × 5  F5 × 5  7 × 7  F7 × 7  
Selftest  F1 

5  6  6  8  
F2 






F3  1 m^{2} 

208  230  293  426  
2 m^{2} 

13  13  18  22  
F4  1 m^{2} 

101  121  120  152  
2 m^{2} 

8  10  12  18  
 
Blind test  F5  1 m2 

48  66  78  125 
2 m^{2} 




3  
F6  1 m^{2} 

24  34  33  54  
2 m^{2} 

5  5  5  6  
F7  1 m^{2} 

152  202  244  388  
2 m^{2} 

120  152  160  230  
 
Selftest  V1  101 

87  77  107  
V2 




 
V3 




 
 
Blind test  V1 

46  58  77  156  
V2 

392  435 

 
V3 





Results of local ACE for differentsized windows.
ACE local  

Target ID  3 × 3  5 × 5  F5 × 5  7 × 7  F7 × 7  
Selftest  F1 






F2 






F3  1 m^{2} 

17  19  33  62  
2 m^{2} 






F4  1 m^{2} 

68  75  72  75  
2 m^{2} 






 
Blind test  F5  1 m^{2} 


12  13  13 
2 m^{2} 






F6  1 m^{2}  6  5  5  5  5  
2 m^{2} 






F7  1 m^{2} 




9  
2 m^{2}  2  2  2  2 


 
Selftest  V1 




7  
V2 




 
V3 




 
 
Blind test  V1 

5  8  15  21  
V2 

106  119  198  359  
V3 





Within the results for the global methods (Table
Similar to the global methods analysis, here (Tables
The RIT website provides a comparison between the results of different algorithms which have been submitted throughout the world. Table
Benchmark results.
Target ID  Local ACE  Web rank  

3 × 3  
Blind test  F5  1 m^{2} 

12/148 
2 m^{2} 

1/148  
F6  1 m^{2}  6  11/90  
2 m^{2} 

1/90  
F7  1 m^{2} 

3/82  
2 m^{2} 

5/82  
 
Blind test  V1 

1/50  
V2 

3/82  
V3 

52/86 
It is possible to compare the actual results obtained by the target detection algorithms on the RIT system to those obtained from the IRBTA simulation. In Tables
RIT results for the actual detection of V1 in RIT test image are shown in the first two lines. The percentage of implanted target was 0.75%. The GLRT and ACE algorithms were calculated as presented in the text. The size of the background was calculated for 3 × 3, 5 × 5 and 7 × 7 frames, excluding the center pixel. The “Only” 7 × 7 and 5 × 5 algorithms only used the outer ring of the window. The third and fourth lines represent the same results normalized by dividing by the values obtained by the 3 × 3 filter.
Website results  

Window  Only  7 × 7  Only  5 × 5  3 × 3  Global 
7 × 7  5 × 5  
V1_GLRT  156  77  58  46  37  428 
V1_ACE  21  15  8  5  1  179 
 
Normalize relative to 3 × 3  
 
V1_GLRT 






V1_ACE 






The
Ath by IRBTAThreshold = 10^{3}  

Window  Only  7 × 7  Only  5 × 5  3 × 3  Global 
7 × 7  5 × 5  
V1_GLRT  0.0007  0.0011  0.0010  0.0014  0.0032  0.0003 
V1_ACE  0.0436  0.0908  0.0946  0.1657  0.4981  0.0420 
 
Normalize relative to 3 × 3  
 
V1_GLRT 






V1_ACE 






In this paper, we showed that there is no “best hyperspectral detection algorithm” for all images and targets. We noted the significant effect spatial distribution has on detector performances, and we showed that the RBTA can be used to select the proper detectors from among several detectors but without any need for ground truth. However, point targets can influence their neighboring pixels, due either to the PSF or to the target spreading across more than one pixel. To account for this potential source of inaccuracy, therefore, we introduced the improved RBTA (IRBTA), whose exact method of use depended on the target size. In addition, we showed that when detectors calculated the mean for estimating the pixel signature value, we did not need ground truth to find the best estimate. We tested our concept through the selection of the best detectors from among stochastic algorithms for target detection, that is, the constrained energy minimization (CEM), generalized likelihood ratio test (GLRT), and adaptive coherence estimator (ACE) algorithms, using the dataset and scoring methodology of the Rochester Institute of Technology (RIT) Target Detection Blind Test project. The results showed that our concepts predicted the best algorithms for the particular images and targets provided by the website.
The authors would like to acknowledge the contribution of the Rochester Institute of Technology (RIT) Digital Imaging and Remote Sensing (DIRS) Laboratory to this work by providing the dataset and an open forum to discuss hyperspectral target detection. The authors also acknowledge the Paul Ivanier Center for Robotics and Industrial Production, BeerSheva, Israel, for partial support of this work. (The work in this paper was performed and submitted before [