A multifrequency radar system for detecting humans and classifying their activities at short and long ranges is described. The short-range radar system operates within the S-Band frequency range for through-wall applications at distances of up to 3 m. It utilizes two separate waveforms which are selected via switching: a wide-band noise waveform or a continuous single tone. The long-range radar system operating in the W-Band millimeter-wave frequency range performs at distances of up to about 100 m in free space and up to about 30 m through light foliage. It employs a composite multimodal signal consisting of two waveforms, a wide-band noise waveform and an embedded single tone, which are summed and transmitted simultaneously. Matched filtering of the received and transmitted noise signals is performed to detect targets with high-range resolution, whereas the received single tone signal is used for the Doppler analysis. Doppler measurements are used to distinguish between different human movements and gestures using the characteristic micro-Doppler signals. Our measurements establish the ability of this system to detect and range humans and distinguish between different human movements at different ranges.
The ability to detect human targets and identify their movements through building walls and behind light foliage is increasingly important in military and security applications. Expeditionary warfighters and law enforcement personnel are commonly faced with unknown enemy threats from behind different types of walls as well as those concealed behind shrubs and trees. Technology that can be used to unobtrusively detect and monitor the presence of human subjects from stand-off distances and through walls and foliage can be a powerful tool to meet such challenges. Although optical systems achieve excellent angular resolution, optical signals are unable to penetrate solid barriers and foliage cover and therefore are totally ineffective in detecting humans in defilade. However, signals in the microwave frequency range can penetrate barriers to an acceptable degree and are therefore the sensors of choice in detection of targets through optically opaque walls. In this case, the choice of the frequency of operation depends on the application, specifically on the barrier type, target position behind the wall, stand-off requirement, and resolution requirements, all of which are somewhat interrelated. Furthermore, since signals in the millimeter-wave frequency range are able to penetrate light foliage cover to an acceptable degree and can be focused to isolate a single human being, they are emerging as the sensors of choice in detection of targets hidden in foliage. The choice of the frequency of operation depends on the application, specifically on the atmospheric attenuation, stand-off requirements, and resolution requirements, all of which are somewhat interrelated.
Low-frequency microwave signals, less than 5 GHz in frequency, can penetrate building walls made of concrete, brick, or cinder blocks, with reasonably low loss. A noteworthy point is that humans behind walls are located at much shorter range from the radar sensor (typically 6–10 feet); thus portable antennas with relatively wider beamwidths can easily isolate a single human. Millimeter-wave systems typically operate in one of the atmospheric “windows,” which offer low propagation loss. These windows exist around 35, 95, 140, and 220 GHz frequencies. The W-Band of the microwave part of the electromagnetic spectrum ranges from 75 to 110 GHz, thus covering the 95 GHz window. The short wavelengths at these frequencies permit the use of small portable antennas to achieve the required angular resolution in order to isolate a single human.
The antenna beamwidth
The down-range resolution
This paper discusses the architecture of the multifrequency radar system and presents data showing that human detection and human activity characterization are possible through different types of barriers. Section
While adequate cross range resolution can be achieved using small size antennas for short-range wall penetration, a suitable modulation scheme must be used to obtain the wide transmit bandwidth of 500 MHz to achieve the desired down-range resolution. Random noise modulation is an ideal candidate for military applications since it possesses several desirable properties, such as covertness, low probability of detection (LPD), low probability of intercept (LPI), immunity from jamming, and resistance to interference, owing to its totally featureless characteristics [
Let
A simplified block diagram of a noise radar is shown in Figure
Simplified block diagram of a noise radar.
When a moving target is illuminated with a single tone frequency of
The micro-Doppler signals are also present in human activity, such as breathing and swinging arms, since each activity involves different types of motions of the chest, torso, and limbs. Figure
The micro-Doppler signatures of concealed human activities at 9 meters stand-off distance in front of a wooden shed: (a) breathing, (b) lifting a large object from the ground, and (c) moving arms up and down rapidly.
A brief summary description of the S-Band through-wall radar system is provided below. A more complete description can be obtained from [
In order to both detect humans and characterize their micro-Doppler signatures, a composite waveform is used, consisting of a wideband noise waveform for ranging and a single tone continuous wave signal for micro-Doppler detection. These waveforms are generated at lower frequencies, called baseband, and then upconverted to the desired frequency range of operation. The noise waveform of 500 MHz bandwidth is generated over the frequency range 100 Hz to 500 MHz, while the single tone is located at 300 MHz. An RF switch is used to select either waveform; therefore, the system operates in either the ranging mode (using the noise waveform) or in the Doppler mode (using the single tone). Each waveform is split and one-half is upconverted to the desired frequency range of operation, while the other half of the signal is routed to the receiver for performing the ranging or the micro-Doppler processing with the received and downconverted signal.
A simplified block diagram of the system is shown in Figure
Simplified block diagram of the S-Band radar system.
A two-stage downconversion receiver processor is used in the system. The time-delayed received signal is collected by an identical receive antenna, amplified, and filtered to remove out of band interference and noise. Then, it is downconverted using the same single tone S-Band signal at frequency
The component layout of the S-Band radar is shown in Figure
Component layout of S-Band radar system.
Fully packaged S-Band radar system (minus antennas).
RF antennas are usually linearly polarized. However, most reinforced building walls contain a lattice of reinforcing bars, or rebars which may be either vertically oriented, or horizontally oriented, or both. Such a structure will affect the propagation of EM waves through it, especially if the rebars are oriented in the direction of the wave polarization. A method to overcome this limitation is to employ a circularly polarized wave, wherein the instantaneous polarization of the wave moves around a circle, thereby allowing most of the wave to pass through with very little loss due to the choice of the wrong polarization. In our system, therefore, we employed helical antennas which are able to transmit and receive circularly polarized signals [
It is known that targets reflect the oppositely handed polarization when illuminated by a circularly polarized wave. The helices used for the transmit and the receive antennas are oppositely wound so that the transmitted wave is right-hand circularly polarized whereas the receive antenna is left-hand circularly polarized. The helical antennas were operated in the axial mode; that is, the antenna dimensions were comparable to the wavelength, wherein a directional endfire pattern, along the axis of the helix, is achieved. The helical antennas designed for this application had the following dimensions: (a) outside rim diameter of the salad bowl shaped ground plane = 18.8 cm (7.4 in), (b) bottom diameter of the ground plane = 8.89 cm (3.5 in), and (c) overall axial length = 35.3 cm (13.9 in). The designed antenna is shown in Figure
View of the helical antenna showing the construction details.
A wall support frame was constructed to house different masonry materials in a dry-stack fashion. The frame was designed to support a wall (e.g., brick or cinder block) that was 2.44 m (8 ft) tall × 2.44 m (8 ft) wide. In addition, the frame had an adjustable width for wall thicknesses of 10.2, 20.3, or 30.5 cm (4, 8, or 12 in, resp.). The structure stood a foot above the ground (adding additional height to the wall) on castor wheels enabling the wall to be mobile. Figure
20.32 cm (8 in) thick cinder block wall in the wall support frame.
To collect radar data, the antennas were mounted on a wooden stand that positioned the antennas approximately 1.37 m (54 in) above the ground and about 1.83 m (6 ft) from the front of the wall. Care was taken to align the antennas properly since poor alignment could negatively influence the results. Coaxial cables of adequate length were connected to the antennas from the radar system (allowing some separation). Targets, such as metallic trihedral corner reflectors and humans were located behind the wall at various distances.
A brief summary description of the W-Band foliage penetration radar system is provided below. A more complete description is provided in reference [
In order to both detect humans and characterize their micro-Doppler signatures, a composite waveform is required, consisting of a wideband noise waveform for ranging and a single tone continuous wave signal for the micro-Doppler detection. These waveforms are generated at lower frequencies, called baseband, and then upconverted to the desired frequency range of operation. The noise waveform of 500 MHz bandwidth is generated over the frequency range of 1.1–1.6 GHz in the L-Band frequency range, while the embedded single tone is located at 1.1 GHz. Both signals are summed together, upconverted to the desired frequency range at W-Band, and transmitted as a composite multimodal waveform. Thus, the system operates simultaneously in both the ranging mode (exploiting the noise waveform component) and the Doppler mode (exploiting the single tone component). Just prior to waveform summation in the transmit chain, one-half of each signal is split and routed to the receiver for performing the cross correlation operation with the received and downconverted signal. Our system was designed in two main sections, a low-frequency L-Band section and a high-frequency mm wave section. Both the L-Band and mm wave sections can then be further subdivided into transmit and receive chains.
A simplified block diagram of the system is shown in Figure
Block diagram of the W-Band radar system.
Upconverted spectrum at W-Band.
Once the signal reflects off of an object, the W-Band receive chain captures the backscattered signal through the receive antenna. The received signal is amplified using a low noise amplifier and downconverted back to the 1.1–1.6 GHz frequency range. The signal is then sent to the L-Band receive chain for further processing. The L-Band receive chain takes this signal and prepares it for the final downconverting stage. This process consists of amplifying and filtering the signal before downconverting to baseband. Down converting both the Doppler and noise waveforms to baseband simultaneously is possible because the single tone is placed at the beginning edge of the band. After downconversion to baseband, the Doppler and noise waveforms are separated by splitting the signal and filtering appropriately. Since the Doppler signal is located in the range of DC to a few kHz, a low-pass filter (LPF) is used to band-limit the signal and to avoid aliasing unwanted signals components when the signal is digitized. The noise waveform contains a DC offset created by the single tone mixing with itself so a DC blocker in addition to a LPF is used to prepare the noise waveform for digitizing. A copy of the transmitted signal is needed as a reference to the matched filter. A copy of the noise waveform is sampled from the L-Band transmit chain and downconverted using an identical mixer and local oscillator as is used for the received signal. The signal is then filtered and attenuated before being digitized. Once the Doppler and noise waveforms are available at baseband, they are digitized using two separate digitizers. The Doppler signal is digitized using a low sample rate digitizer since these frequencies are quite low. Both the received and reference noise waveforms are digitized using high-speed digitizer with the sample rate set to 1 Gs/sec to satisfy the Nyquist sampling criterion [
The component layout of the W-Band radar is shown in Figure
Component layout of W-Band radar system.
(a) Pyramidal horn antenna; (b) circular dielectric horn lens antenna with a sighting scope attached.
Two different antennas were used, depending upon the target range considerations. For close range measurements, pyramidal horn antennas of aperture size 1.0625 inches × 0.875 inches, shown in Figure
In addition to unobstructed long-range measurements, data were also collected to investigate the radar system’s ability to detect targets through light foliage. To do this, we aimed the radar at a Border Forsythia (
Geometry of foliage penetration measurement setup.
Photographs of the foliage penetration measurements. (a) shows the radar aimed at the bush, while (b) shows a corner reflector target placed behind the bush.
To collect radar data, the antennas were mounted on a wooden stand that positioned the antennas approximately 1.37 m (54 in) above the ground. Care was taken to align the antennas properly using sighting scopes since poor alignment could negatively influence the results. Targets, such as metallic trihedral corner reflectors and humans, were located at various distances.
A major problem in through-wall radar is the existence of large peaks in the reflected response due to direct antenna coupling as well as the reflection from the wall itself. These signals can obscure the target reflections, as can be seen in Figure
Detection of a small trihedral target placed 1.22 m (4 ft) behind a 10.2 cm (4 in) thick brick wall. Both no target and target present cases are shown, as well as the implementation of the background subtraction algorithm which suppresses non-target responses and enhances target response.
Direct antenna coupling also obscured the low target reflections from longer ranges for the W-Band radar, as can be seen from the correlation plot in Figure
Detection of a human at a range of 213 m (700 ft). (a) No background subtraction; (b) background subtraction implemented; (c) background subtraction and distance correction implemented. Note the suppression of non-target responses and enhancement of target response at longer range.
For the S-Band though-wall radar, a human was located at a distance of 1.22 m (4 ft) behind a 10.2 cm (4 in) thick brick wall with the antennas located at a distance of 1.83 m (6 ft) in front of the wall. Thus, the distance between the antennas and the human was about 3.05 m (10 ft). We note from Figure
Detection of a human target placed 4 feet behind a 10.4 inch thick brick wall. Both no target and target present cases are shown, as well as the implementation of the background subtraction algorithm which suppresses non-target responses and enhances target response.
In addition, background subtraction can also be used for detecting moving targets. Subtraction of successive frames of the cross correlation signals between each received element signal and the transmitted signal has been shown to be able to isolate moving targets in heavy clutter [
Tracking of a moving human using successive scene subtraction.
For the W-Band radar foliage penetration experiments, we used two targets, a corner reflector and a human. The radar was located at a stand-off distance of 30 m (98.4 ft) from the bush and each target was placed 2 m (6.6 ft) behind the bush. Correlation data were averaged over 100 looks to reduce the effects of noise. Figures
Detection of a corner reflector behind a bush.
In addition, data were also collected for a human target. Correlation data taken for the human behind the bush with no leaves is shown in Figure
Detection of a human behind a bush.
Since different human activities result in different micro-Doppler signatures, a technique was developed and implemented for automatic classification of specific human activities, more fully described in [
Classification of signals, such as the micro-Doppler signatures, requires a unique feature vector for each signal. EMD readily provides a feature vector by the calculation of the energy of each IMF component or the inner product of the signal with itself. When the EMD process is conducted on the micro-Doppler signals, the collection of IMF energies provides us with a vector that is unique to the movement that caused the Doppler frequency shift.
Support vector machines (SVMs) have proven to be an effective alternative to traditional classification techniques, such as the Bayesian classifiers and artificial neural networks (ANNs) [
The classification is performed using an SVM with a Gaussian kernel. The constrained optimization problem is formulated as
SVMs were originally developed to solve the binary classification problem; therefore, modifications must be made in order to extend the binary problem to a multiclass problem. Multiple methods have been proposed to tackle this problem. Because of its intuitiveness and its ability to be easily adapted for additional classes, the one-against-all (1-a-a) method was chosen for the experiments [
Experimentally observed time-frequency plots of unobstructed (without wall) human activities are shown in Figure
Time-frequency plots of different human activities as measured by the S-Band through-wall radar.
Comparison of the micro-Doppler signal of humans swinging their arms without a wall barrier and with a wall barrier using the S-Band through-wall radar.
Three cases were examined for activity classification for the S-Band through-wall radar: (1) direct transmission without a wall barrier, (2) transmission through a 10.2 cm (4 in) thick brick wall, and (3) transmission through a 20.3 cm (8 in) thick cinder block wall. For the direct transmission case, the person was located 3.35 m (11 ft) away from the radar antennas. For the brick wall case, the person was located 1.52 m (5 ft) behind the wall with the wall 1.83 m (6 ft) away from the radar antennas. For the cinder block case, the signals were very weak, so the distances were shortened. The person was located 91 cm (3 ft) behind the wall and the wall was located 61 cm (2 ft) away from the radar antennas. The five different motions listed above were considered for classification. The training set consisted of data from five of the six test subjects and was used both for training the classifier and for the cross-validation. The cross-validation set also consists of data from five of the six test subjects, but these data were used only for cross-validation and not used for training the classifier. The test set consists of data from one of the six test subjects and these data were used neither for training the classifier nor for cross-validation. Classification results are shown in Table
Human activity classification results for S-Band through-wall radar.
Human test subject number | Average classification accuracy (%) | |||||
---|---|---|---|---|---|---|
No wall | 4 in thick brick wall | 8 in thick cinder block wall | ||||
(11 ft range to target) |
( |
( | ||||
Mean | Standard deviation | Mean | Standard deviation | Mean | Standard deviation | |
1 | 76.0 | 3.3 | 61.4 | 5.1 | 57.2 | 5.0 |
2 | 52.8 | 6.9 | 48.6 | 9.8 | 66.0 | 4.5 |
3 | 61.0 | 3.8 | 44.2 | 4.3 | 68.4 | 5.3 |
4 | 70.4 | 3.6 | 48.0 | 4.5 | 71.4 | 4.8 |
5 | 66.8 | 3.6 | 46.6 | 5.0 | 59.6 | 5.8 |
6 | 56.2 | 3.3 | 49.8 | 2.7 | 61.8 | 6.0 |
Five different motions were investigated for the W-Band foliage penetration radar. These include (a) no activity (for reference), (b) breathing, (c) swinging arms, (d) picking up object from ground, and (e) transitioning from crouching to standing. Experimentally observed time series and time-frequency plots of unobstructed (without foliage cover) human activities at a range of 30 m (98.4 ft) are shown in Figure
Time series and time-frequency plots of different unobstructed human activities as measured by the W-Band radar at a range of 30 m (98.4 ft). (a) No activity, (b) breathing, (c) swinging arms, (d) picking up object from ground, and (e) transitioning from crouching to standing.
Time-frequency plots of different unobstructed human activities as measured by the W-Band radar at a range of 90 m (295 ft). (a) No activity, (b) breathing, (c) swinging arms, (d) picking up object from ground, and (e) transitioning from crouching to standing.
Time series and time-frequency plots from the fully-foliated bush with leaves at 30 m (98.4 ft). (a) Unfiltered, and (b) filtered.
Time series and time-frequency plots from arm swinging behind fully foliated bush at 30 m (98.4 ft). (a) Unfiltered and (b) filtered.
All five different motions listed above were considered for classification. The training set consisted of data from five of the six test subjects and was used for both training the classifier and for the cross-validation. The cross-validation set also consists of data from five of the six test subjects, but these data were used only for cross-validation and not used for training the classifier. The test set consists of data from one of the six test subjects and these data were used neither for training the classifier nor for cross-validation. Classification results are shown in Table
Human activity classification results for W-Band radar under free space conditions.
Human test subject number | Average classification accuracy (%) | |||||
---|---|---|---|---|---|---|
100 ft range to target | 200 ft range to target | 300 ft range to target | ||||
Mean | Standard deviation | Mean | Standard deviation | Mean | Standard deviation | |
1 | 88.6 | 1.3 | 86.0 | 6.0 | 94.8 | 3.7 |
2 | 87.0 | 1.9 | 80.4 | 1.3 | 95.2 | 3.2 |
3 | 85.4 | 1.9 | 86.6 | 1.6 | 94.0 | 3.4 |
4 | 80.6 | 3.1 | 84.4 | 4.6 | 95.2 | 1.9 |
5 | 80.2 | 5.8 | 71.2 | 3.9 | 93.4 | 4.1 |
6 | 53.4 | 9.3 | 69.2 | 7.1 | 93.4 | 5.0 |
To the best of our knowledge, this is the first reporting of the ability to classify different types of human activity behind opaque walls and foliage cover. While the results obtained thus far are quite encouraging, more research and system development are needed to improve the classification accuracies in the presence of barriers and to include additional movements and gestures. We are currently working on expanding the range of human activities as well as the variety of humans for additional data collection.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported by the U.S. Army ARDEC Joint Service Small Arms Program (JSSAP) under Contract no. W15QKN-09-C-0116. The authors appreciate helpful comments provided by E. Beckel, W. Luk, and G. Gaeta of ARDEC.