Using Healthcare Resources Wisely: A Predictive Support System Regarding the Severity of Patient Falls

Background An injurious fall is one of the main indicators of care quality in healthcare facilities. Despite several fall screen tools being widely used to evaluate a patient's fall risk, they are frequently unable to reveal the severity level of patient falls. The purpose of this study is to build a practical system useful to predict the severity level of in-hospital falls. This practice is done in order to better allocate limited healthcare resources and to improve overall patient safety. Methods Four hundred and forty-six patients who experienced fall events at a large Taiwanese hospital were referenced. Eight predictors were used to ascertain the severity of patient falls solely based on the above study population. Multinomial logistic regression, Naïve Bayes, random forest, support vector machine, eXtreme gradient boosting, deep learning, and ensemble learning were adopted to establish predictive models. Accuracy, F1 score, precision, and recall were utilized to assess the models' performance. Results Compared to other learners, random forest exhibited satisfying predictive performance in terms of all metrics (accuracy: 0.844, F1 score: 0.850, precision: 0.839, and recall: 0.875 for the test dataset), and it was adopted as the base learner for a severity-level predictive system which is web-based. Furthermore, age, ability of independent activity, patient sources, use of assistive devices, and fall history within the past 12 months were deemed the top five important risk factors for evaluating fall severity. Conclusions The application of machine learning techniques for predicting the severity level of patient falls may result in some benefits to monitor fall severity and to better allocate limited healthcare resources.


Introduction
Falls are the second leading cause of death by unintentional or accidental injuries worldwide [1]. An estimated 37.3 million falls critical enough to necessitate medical attention occur each year, and about 646,000 fatal falls occur annually [1]. An injurious fall is considered to be one of the main indicators of care quality in monitoring patient safety in healthcare facilities [2]. Ensuring patient safety in hospitals has remained a major public health concern [2][3][4]. Individuals who had fall events within healthcare environments regarding patient safety are of extreme importance to monitor and necessary to prevent accidental and injurious falls [5]. In addition, when a fall occurs, patients have different levels of injury, prolonging their treatment periods and increasing healthcare costs, perhaps even causing death [3,6,7]. us far, several tools such as the Morse Fall Scale [8], the St.
omas Risk Assessment Tool in Falling Elderly Inpatients [9], and the Hendrich II Fall Risk Model [10] have been applied to evaluate fall risk. ese tools have surely provided valuable assistance to existing fall measures by identifying all susceptible patients. Healthcare facilities then foster and engage in coping strategies to diminish the occurrence of these plausible fall events among their patient population. is approach may be feasible with sufficient healthcare resources. During particular occasions such as the COVID-19 pandemic, all healthcare resources are so limited that it is not possible to allocate abundant resources in detecting and handling all fall events as once before. erefore, these fall risk screening tools will have been more valuable and comprehensive if they are able to predict the severity of fall events. By knowing the severity level, healthcare facilities can better allocate limited resources and use resources more wisely for the most needy patients [11].
Prior studies have done specific machine learning methods for selecting factors and classification modes according to incident reports and predicted the probability of adverse event occurrence in healthcare facilities [12][13][14][15]. However, there is a dearth of studies evaluating the severity level of patient falls, not to mention the inclusion of machine learning techniques. In order to better improve patient safety practices, a user-friendly prediction system for screening patients of plausible severe fall events is critically required. erefore, the purpose of this study (see Figure 1) is to develop a useable predictive support system based on machine learning techniques, which can facilitate the assessment of the prior severity level of patient falls in healthcare facilities.

Related Works.
To date, previous studies have identified various fall risk factors inherent to hospital environments. For example, Liu et al. [16] found several factors such as age, unassisted movement, impaired mobility, and fall history within the past 12 months as being imperative to the prediction of patient falls. ey were however unable to identify whether having any companions present at the time of fall acts as a risk or mitigating factor. Cho et al. [17] contrasted three fall risk assessment scales (i.e., Morse Fall Scale, e John Hopkins Hospital fall risk assessment tool, and Hendrich II fall risk model) and indicated important risk factors such as fall history, ambulatory aid, gait, mental status, medication, requirement of assistance or supervision, symptomatic depression, altered elimination, and dizziness. In their study of fall risk factors of inpatients, Najafpour et al. [4] showed important risk factors such as longer length of stay, using chemotherapy drugs, anticonvulsants, sedatives, angiotensin-converting enzyme inhibitors, benzodiazepines, balance condition, visual acuity, and manual transfer aid. ey were however unable to prove fall history being a significant risk factor in patient care. Ozturk et al. [18] found fall-related factors for emergency patients including being physically active before the fall event, being between 65 and 79 years of age, having chronic diseases, and being on benzodiazepine and other specific medications. Aryee et al. [19] confirmed that recent surgery was protective, joint replacement, psychotropic agents, male gender, and history of fall were critical fall risk factors for surgical patients. Another study reported that rates of fall injury diagnoses in the emergency department were generally higher among older women [20]. Chen et al. [21] demonstrated independent risk factors for falls among aged inpatients including insomnia at the time of admission, new episodes of leg weakness, postural hypotension, newly prescribed hypnotics during admissions, more than one fall event history within the previous year, and caregiver's accompaniment. ese studies have paved the foundation on this important topic. In accordance with prior evidence, we can see these identified fall risk factors being quite diversified in nature and no consensus on this issue has yet to be reached. Furthermore, the studies mentioned adopted a traditional statistical model requiring strict assumptions for analyzing the data [22]. us far, several studies have used machine learning to approach issues related to patient safety (see Table 1). For example, Ong et al. [23] adopted Naïve Bayes and support vector machine to develop text classifiers for automatically detecting extreme-risk adverse events taken from clinical incident reports. eir results showed that support vector machine performed best with F-measure � 0.86, precision � 0.88, recall � 0.83, and area under the receiver operating characteristic curve (AUC) � 0.92 in identifying incident types. Cheng and Zhao [25] proposed a heterogeneous, network-assisted inference framework to support for forecasting drugdrug interactions. ey applied five algorithms including decision tree, Naïve Bayes, k-nearest neighbor, support vector machine, and logistic regression to establish the proposed framework. Support vector machine demonstrated the best performance with an AUC of 0.67. In order to screen hazardous cases related to electronic health records, Marella et al. [12] tested several algorithms comprising Naïve Bayes kernel, Naïve Bayes, k-nearest neighbor, and rule reduction. ey found that Naïve Bayes kernel performed best in terms of AUC (0.84) and accuracy (0.75). ese prior efforts surely add knowledge and improve our understanding of patient safety. A literature review, however, demonstrates research gaps regarding applied machine learning techniques and patient safety issues. First, these prior studies have not yet investigated the severity level of patient falls. Second, most studies focus on assessing the types of patient safety incidents [12,13,15,16,23,26,29]. e characteristics of various patient safety incidents remain diversified, and it is therefore difficult to reach a satisfying solution. Instead, studies may wish to further identify a specific incident such as patient fall event in order to gain a better understanding on the nature of such an incident.
ird, evidence shows that support vector machine is frequently adopted as algorithm for establishing predictive models; however, this allows room for testing how other algorithms might perform with regard to patient safety problems.

Study Population and
Setting. Data were obtained from the Taiwan Patient-safety Reporting System (TPRS) [30] of a large hospital in southern Taiwan from 2019 to 2020. TPRS is an incident report system that can collect patient adverse events including medication, falls, operations, blood transfusions, health care, and public accidents. Our study focused on fall adverse events because the collected data are more comprehensive than the other types of adverse events. Eligibility criteria were that a patient must (1) be aged over 20 and (2) belongs to a fall event. e Institutional Review Board of E-Da hospital permitted the study protocol and waived informed consent (EMRP-109-159).

2.2.
Measures. Patients' baseline information comprising gender, age, patient sources, fall history within the past 12 months, ability to perform an independent activity (assessed   Journal of Healthcare Engineering by means of the Barthel index), companionship (such as family members, relatives, friends, or caregivers being present at the event), and use of assistive devices (defined as a device that assists patients in accompanying day-to-day functions [21]) were collected and used as features for building the predictive model. e relative severity of patient falls is the target of the predictive model. Table 2 shows the operational definitions of variables included in this study.

Experimental Setup.
In order to build a machine learning model to forecast the severity of fall adverse events, we adopted R 4.1.2 [31] for data analysis and Python 3.7 [32] with scikit-learn 1.1.0 [33] for building predictive models. We chose seven learners including multinomial logistic regression, Naïve Bayes, random forest, support vector machine, eXtreme gradient boosting, deep learning, and ensemble learning to construct the predictive model. ese learners were chosen for comparative purpose, and these learners are frequently used in building healthcare predictive models with good performance [34]; Chen et al. [35][36][37].
Regarding ensemble learning, we adopted the stacking approach because two other ensemble approaches, including bagging and boosting, were implemented by random forest and eXtreme gradient boosting. We used logistic regression as the meta-model. Models based on multinomial logistic regression, Naïve Bayes, random forest, support vector machine, eXtreme gradient boosting, and deep learning with better performance were then chosen as the base models since the rationale for using the stack approach is to learn how best to combine the predictions from multiple wellperformed machine learning algorithms [38]. We adopted a random search approach to automatically determine the optimal combinations of parameters for the selected machine learning algorithms in order to reach a better prediction performance (see Table 3). We used the 2019 data as a training dataset and the 2020 data as the test dataset, aiming to improve the model's accuracy and to diminish any possible overfitting issues [22]. Since the collected data were imbalanced in the target variable, a synthetic minority over-sampling technique, for the training dataset, was leveraged by under-sampling the adequate class and over-sampling the inadequate class in order to improve the model performance [39]. Further, we applied a 10-fold cross-validation method to assess the six learners with the training dataset in order to better estimate the model performance. Mean and standard deviation of performance metrics are calculated.

Performance Measure.
Since the target in this study is three class, more suitable metrics including accuracy, F1score, precision, and recall are adopted to assess the performance of predictive models [40].
ese four metrics are based on a confusion matrix which includes four predicted results: true positive, false positive, false negative, and true negative; as such, the four metrics can then be derived [22]. When interpreting the model performance, the closer to the value of 1 is considered the better for all metrics concerned [22].

Data Profiles.
A total of 446 patients who experienced falls were included, 209 patients in 2019 and 237 in 2020. e distributions of patient's demographics are similar in 2019 and 2020. Among these cases, males were more represented than females (see Table 4). Most patients were aged 51-80 years, while most subjects belonged to inpatient care. In addition, 73.21% and 74.26% of patients had fallen in the past 12 months in 2019 and 2020, respectively; over 90% of patients were labeled at high risk of falling. Approximately 61.24% and 59.49% of patients belonged to the group who were partially dependent on their family members or friends in the prior two years to the event, and 75.6% and 72.15% of them did not have usual companions present. Further, in both years, 46.89% and 45.57% of the patients used assistive devices. Regarding the severity of fall events, 47.85% and 54.01% of patients had severe adverse effects, 15∼16% of patients had mild adverse effects, and more than 30% of patients had no adverse effects whatsoever. Table 5 shows the performance results of seven learners including multinomial logistic regression (MLR), Naïve Bayes (NB), random forest (RF), support vector machine (SVM), eXtreme gradient boosting (XGBoost), deep learning (DL), and Stacking for the training and test dataset, respectively. Accuracy, F1-score, precision, and recall were used to assess the performance of those learners. Based on the results of 10-fold cross-validation, RF, SVM, XGBoost, and DL were included in the Stacking as the base models while MLR and NB were not included due to all four metrics being lower than 0.5 for both learners (see Table 5).
A comprehensive assessment of the performance for the selected learners based on four metrics shows that RF performs better than MLR, NB, SVM, XGBoost, DL, and Stacking. e area under the receiver operating characteristic curve and confusion matrix of the RF learner for the test dataset are illustrated in Figures 2 and 3, respectively. Further, there is no clear evidence of overfitting based on the performance metrics of training and test datasets in addition to MLR and NB [22].

Comparison with Benchmark Models.
To better support our findings, we compared the results of the performance of our built predictive model based on random forest learner with the results of the Morse Fall Scale [8].
e target variable in this study is three class indicating the severity of fall events (no, mild, and severe adverse effect) which is inconsistent with the three risk levels (no, low, and high risk) of the Morse Fall Scale. Moreover, severe adverse events and high Morse fall risk scores were less frequently observed in patients who experienced falls. We therefore segment the severity of fall events into two classes, namely no adverse effect and possible adverse effect (including mild and severe adverse effect), and segment the Morse risk levels into two levels, namely no risk and possible risk (including low and high risk). An accuracy score of about 0.57, better than randomly guessing, was obtained by comparing the predicted results of the test dataset with the re-coded Morse fall risk levels. is result is expected because the primary purpose of our study is to predict the severity of fall events rather than the fall risk. Further, high risk of falling does not mean severe adverse effect when patients do happen to fall.

Feature Importance and Model Explainability.
Apart from comparing the performance of learners, we also graded the feature importance based on Shapley additive explanations (SHAP) values [41]. Higher SHAP value indicates more strength of features' contribution to the prediction results [41]. As shown in Figure 4, the top five important features for predicting severe adverse effect included age, ability of independent activity, use of assistive devices, fall history within the past 12 months, and patient source.
We further analyzed the relationship between features and predicted adverse effect based on the beeswarm plot. As shown in Figure 5, we can acquire some insights such as higher age, higher dependence, use of assistive devices, and patients without fall history within the past 12 months all lead to higher probability of severe adverse effect.

Severity-of-Fall Prediction
System. Based on the performance of seven learners, we chose RF as the primary learner for developing a severity-of-fall prediction system, which can be considered as a supporting system adding to patient safety. To ensure this support system can be deployed  Journal of Healthcare Engineering in differing computer platforms, we adopted a representational state transfer (RESTful) application programming interface based on a Flask package [42] to establish application efficacy. Based on the RESTful approach, users can use the prediction support system from browsers without temporal and spatial limitations. As depicted in Figure 6, healthcare professionals can first input required information, and they can then predict the probability of there being severe, mild, and no adverse effects present when caring for at-risk patients, as shown in Figure 7.

Discussion
Fall severity can exert direct catastrophic percussion on patients and their families and indirect direct catastrophic percussion upon healthcare facilities and involved staff; thus, prevention of fall events should be dealt with due diligence. erefore, prior knowledge of what constitutes the fall risk factors and what is the severity of falls are paramount issues for healthcare providers and facilities. In this study, we utilized machine learning techniques to predict the level of  severity caused by patient falls. Knowing the risk factors of severity of patient falling events and also being able to predict the plausible severity of such fall events, healthcare facilities can improve patient safety. Likewise, they can better utilize limited resources to undertake proper interventions on the most needful patients. Among the seven algorithms adopted in our study, RF showed the best predictive performance over the remaining algorithms examined. Based on the RF algorithm, we built a web-based prediction support system which can unravel geographic and temporal     Journal of Healthcare Engineering constraints. Moreover, we also identify the top five fall severity risk factors, based accordingly on the SHAP value, including age, ability of independent activity, use of assistive devices, patient source, and fall history within the past 12 months aspect. Several findings of this study are interesting and deserve further attention. First, prior patient safety studies that employed machine learning techniques largely relied on SVM [13,23,25,27,29]. e overall performance of SVM in our study is lower than that of RF. A plausible reason might be that prior studies included all forms of incident types, while our study focused only on the singularity of fall events. Further, previous studies also incorporated text mining techniques for incident reports while our study only included reportage of structured data.
Second, regarding the five important features identified in our study. Our study found that age is an important predictor of fall risk severity based on the SHAP value. is finding corroborates with prior evidence [2,18,24] which clearly demonstrated that fall rate frequently increased with patient age. A higher fall rate may indicate a greater chance of severe injury by fall occurrence. Further, patients with impaired mobility generally tend to be at high risk of falling when moving independently, potentially resulting in the occurrence of serious fall injuries. A prior survey reports that elderly patients without the ability of independent activity have 14 times more likelihood to report having 2 or more falls in the prior 12 months when compared with the elders without limitations on their mobility [43]. Several other investigations of fall risk factors have also shown mobility being of some importance [2,17] and lend credence to our findings. e use of assistive devices was identified as an important predictor for fall severity which is consistent with prior evidence [21]. Chen et al. [21] demonstrated that those patients who fell were more regular to use assistive devices than non-fallers. is pointed out that patients with walking aids were more likely to move independently and thus lowered the probability of severe fall risks.
Fall history has been found to be an important risk factor for predicting future falls [2,17,21]. Our study also confirmed that a fall history event within the past 12 months is an influential feature for predicting fall severity, which is consistent with the literature [2,17,21]. Special precautions should be provided for patients with a prior fall history.
is study also found that patient sources (i.e., inpatients, outpatients, or emergency department patients) play a key role in predicting the severity of falls. Prior evidence showed that those patients who fall were discharged after their index visit to emergency department. ey had higher probability to revisit emergency department for a fall-related complaint when compared with non-fallers [44]. Other evidence [2][3][4] also reported several potential fall risk factors for inpatients. Based on our findings, it can thus be suggested that factors including age, ability to engage in meaningful independent activity, use of assistive devices, an incidence of fall history within the past 12 months, and patient source are important for the predictive determination of fall severity. All of these factors should be considered when healthcare facilities and providers are planning patient fall-prevention strategies.
ird, the fall severity prediction support system is webbased, easy-to-use, and user-friendly.
is system can be assessed without time and space limitations and any device that can browse the Internet brings about accessibility. Further, this prediction system can be used as a clinical decision support tool for management of in-hospital fall patients served by frontline healthcare professionals. Once the probability of severity of fall events is identified, this information can be integrated into respective hospital information systems, and it can thus be used to inform the healthcare professionals how much effort they have to put in this post-occurrence of the accident. is would include, but is not limited to, the frequency of nurses' vital sign assessments and the requirement of further brain computer tomography, physical exams, etc.
Our data were extracted from a patient safety reporting system, based on the specification of Joint Commission of Taiwan [30], from one hospital in Taiwan. e main idea of reporting adverse incidents is anonymity, voluntariness, confidentiality, unaccountability, and mutual learning [30]. Such a practice is supportive towards encouraging healthcare professionals to proactively report any incidents in their sphere of activity. However, several study limitations must be mentioned.
is practice of reporting may inhibit healthcare professionals from disclosing every incident since it is not mandatory, which may have potential impacts on the generalizability of our predictive model. Future research may collect incident data across hospitals or even countries to compare and contrast the findings of this study.

Conclusions
Patient falls have received much attention among healthcare facilities worldwide due to its plausible severe adverse effects to patients in aging populations. A fall event is one of the important, perhaps most preventable, incidents involving patient safety, and it should be handled cautiously. Based on this understanding, our study developed a fall severity prediction support system based on machine learning techniques. By comparing the performance of seven machine learning algorithms (MLR, NB, RF, SVM, XGBoost, DL, and Stacking), we chose an algorithm with the best predictive performance in terms of four metrics, including accuracy, F1-score, precision, and recall. Among the seven algorithms, RF outperformed the remaining algorithms in all four metrics. Moreover, we have also identified important features for predicting severity levels of falling, including age, ability of independent activity, use of assistive devices, a fall history within the past 12 months, and patient source. We then built a web-based severity prediction system by using RF algorithm and Flask package. By leveraging this predictive system, healthcare facilities can improve patient safety practices and better allocate limited resources to critical fall occurrences. Moreover, frontline healthcare professionals can use this prediction system while facing fall incidence and perform risk stratification that provides an adequate response to vulnerable patients.
is will be Journal of Healthcare Engineering 9 beneficial to patients, healthcare professionals, family, and bedside caregivers alike.

Data Availability
Data are not available due to ethical constraints.

Conflicts of Interest
e authors declare that they have no conflicts of interest.