This work presents a novel indoor video surveillance system, capable of detecting the falls of humans. The proposed system can detect and evaluate human posture as well. To evaluate human movements, the background model is developed using the codebook method, and the possible position of moving objects is extracted using the background and shadow eliminations method. Extracting a foreground image produces more noise and damage in this image. Additionally, the noise is eliminated using morphological and size filters and this damaged image is repaired. When the image object of a human is extracted, whether or not the posture has changed is evaluated using the aspect ratio and height of a human body. Meanwhile, the proposed system detects a change of the posture and extracts the histogram of the object projection to represent the appearance. The histogram becomes the input vector of K-Nearest Neighbor (K-NN) algorithm and is to evaluate the posture of the object. Capable of accurately detecting different postures of a human, the proposed system increases the fall detection accuracy. Importantly, the proposed method detects the posture using the frame ratio and the displacement of height in an image. Experimental results demonstrate that the proposed system can further improve the system performance and the fall down identification accuracy.
In Taiwan, falls represent the second leading accidental cause of death among elderly people. The rate of falling down among the elderly ranges from roughly 15% to 40% annually, and the incidence of falling increases as they grow older. Most falls lead to hospitalization of the elderly, residing in nursing homes, and barriers to daily activities. Elderly people most commonly fall in the bathroom, toilet, living room, and bedroom. Therefore, the ability to detect the falling of elderly people quickly would decrease the rate of injuries and reduce medical treatment costs. The damage degree of falls among the elderly is often decided by the time of discovery, transport, and emergency medical services. Developing electronic technologies facilitates the integration of sensors, computer vision, and the increasingly popularity of the wireless network. Such integrated applications can help the elderly to avoid potentially dangerous situations. This automatic system also reduces neglect among individuals and achieves zero-distance medical treatment.
The rest of this paper is organized as follows. Section
Recognizing human behaviors using computer vision techniques is actively researched in various fields. A simple method detects a fall by analyzing the aspect ratio of the bounding box of a moving object [
Another design [
Figure
Proposed system architecture.
While a vision surveillance system heavily emphasizes how to detect moving objects, the segmentation accuracy of moving objects can improve the performance of further analysis (e.g., object extraction or posture analysis). Figure
Foreground segmentation flow.
When a foreground image is identified, some noises appear in the image or some holes appear in moving objects. Additionally, the morphology model is used to eliminate some small noise interference and fill holes of moving objects. Moreover, large noises are eliminated by using the information of each foreground object boundaries and foreground object area to distinguish between larger noises and regions of interest. Figure
Image processing flow.
The proposed system finally recognizes the posture of each foreground object. Features of each foreground object are first extracted for further analysis and classification. The classifier uses the K-Nearest Neighbor (K-NN) algorithm and combines it with posture change detection. When a change event of the posture is detected, the proposed system recognizes which posture is changed by using the K-NN based classifier. Finally, the foreground object between the real fall and only a supine position is discriminated by using the falling speed. Figure
Behavior analysis flow.
Extracting useful feature information in a foreground image is of priority concern in recognition processes. The proposed system extracts three main features, as described in the following.
Calculate the horizontal and vertical projection histogram of foreground objects:
Apply
Normalize based on the following formula:
The proposed posture classification system has two major components: K-NN classifier and falling speed detection. The system first detects whether or not the posture has changed by using the features of the aspect ratio and human body height. When a posture change is detected, exactly which posture has changed is recognized using the K-NN based classifier. Otherwise, the system selects the recent output in the classifier as the current output. Figure
Posture change detection flow.
Five postures defined in our system.
Standing
Sitting
Bending
Lying toward
Lying
During the classification phase, the distance between the current frame and the stored template
While implemented by OpenCV library and Visual C++ 2008, the proposed system runs on an Intel Core i7 3.4 GHz laptop PC with 8 GB memory. To evaluate the system performance and accuracy, the experimental environment is an indoor place with a single and fixed camera. The distance between the individual and the camera is approximately 4-5 meters. The experiment is conducted by observing the video and noting the detection results to determine whether or not the current image is classified accurately.
To reduce the computational cost and stabilize the classifier output, this work also develops a posture change detection method to overcome these problems. Figure
Comparison between output of classifier with and without posture change detection.
Table
Comparison of different methods in terms of the execution time.
Scenario | Method | ||
---|---|---|---|
Nasution and Emmanuel [ |
Silapasuphakornwong et al. [ |
Ours | |
Video 1 (ms) | 85.25 | 15.00 | 54.26 |
Video 2 (ms) | 88.00 | 15.40 | 60.26 |
Video 3 (ms) | 99.34 | 15.27 | 73.23 |
Video 4 (ms) | 111.23 | 15.66 | 82.77 |
Average execution time (ms) | 95.96 | 15.33 | 67.63 |
In our video clips, eight subjects of different heights and weights were asked to participate in the project. The image resolution is 640 × 480 pixels. Four of the video clips were taken to train the templates of the classifier, and the remaining ones were taken to evaluate the system performance. In the posture change detection method, the threshold for the aspect ratio of the human body was set to 0.1, and the threshold for the height ratio of the human body was set to 0.15. In the height test, the threshold was set to 0.45. Table
Recognition rates for various postures in the proposed system.
Posture | Recognition | |||
---|---|---|---|---|
|
|
|
|
|
Standing | 1459 | 1432 | 27 | 98.15 |
Sitting | 419 | 391 | 28 | 93.31 |
Bending | 634 | 603 | 31 | 95.11 |
Lying | 297 | 297 | 0 | 100.00 |
Lying toward | 236 | 216 | 20 | 91.53 |
Fall down | 514 | 476 | 38 | 92.60 |
Table
Comparison of different methods in terms of recognition rate.
Posture | Method |
||
---|---|---|---|
Nasution and Emmanuel [ |
Silapasuphakornwong et al. [ |
Ours (%) | |
Standing | 76.56 | 95.68 | 98.15 |
Sitting | 99.28 | None | 93.31 |
Bending | 63.25 | 9.30 | 95.11 |
Lying | 66.00 | 100.00 | 100.00 |
Lying toward | 100.00 | None | 91.53 |
Fall down | None | None | 92.60 |
This work also evaluates the performance of different fall detection methods by using a video recorded with three individuals, who participated in the project with different heights and weights. Our video clips contain 100 fall events and 100 false fall events. Two widely used criteria in fall detection systems are adopted here for comparisons [
Parameter definitions in (
System |
Fall incident | |
---|---|---|
Occurs | Does not occur | |
Positive | TP | FP |
Negative | FN | TN |
Parameter
Experimental results of our system with four possible recognitions.
System |
Fall Incident | |
---|---|---|
Occurs | Does not occur | |
Positive | 94 | 3 |
Negative | 6 | 97 |
Comparison of fall detection methods.
Criterion | Method |
||
---|---|---|---|
Nasution and |
Rougier et al. [ |
Ours | |
Sensitivity (%) | 83 | 40 | 94 |
Specificity (%) | 88 | 63 | 97 |
This work designs an indoor video surveillance system with fall detection capability. The proposed system can also detect and evaluate a human posture. An attempt is also made to improve the overall system performance by developing three methods to reduce the execution time for recognition and increase the recognition rate of human postures. The first method utilizes the mean ratio of the height in standing and supine postures to distinguish between these two postures. By using the posture change detection, the second method reduces the computational cost and stabilizes the classifier output. By using the height of a human body, the third method distinguishes between the bending and supine postures. Experimental results indicate that the proposed design can further reduce the execution time and increase the recognition rate of human postures. Additionally, the proposed system can achieve a recognition rate higher than 90% for each posture. Moreover, the proposed system can also detect the fall down with the accurate rate of 92.60%. Performance comparisons reveal that the proposed system performs better than previous designs. Efforts are underway in our laboratory to incorporate capabilities of multiobject tracing and face recognition in the proposed system.
This work is supported in part by Taiwan’s Ministry of Education under Project no. 101B-09-027.