What Does It Cost to Prevent On-Duty Firefighter Cardiac Events? A Content Valid Method for Calculating Costs

Cardiac arrest is a leading cause of mortality among firefighters. We sought to develop a valid method for determining the costs of a workplace prevention program for firefighters. In 2012, we developed a draft framework using human resource accounting and in-depth interviews with experts in the firefighting and insurance industries. The interviews produced a draft cost model with 6 components and 26 subcomponents. In 2013, we randomly sampled 100 fire chiefs out of >7,400 affiliated with the International Association of Fire Chiefs. We used the Content Validity Index (CVI) to identify the content valid components of the draft cost model. This was accomplished by having fire chiefs rate the relevancy of cost components using a 4-point Likert scale (highly relevant to not relevant). We received complete survey data from 65 fire chiefs (65% response rate). We retained 5 components and 21 subcomponents based on CVI scores ≥0.70. The five main components include, (1) investment costs, (2) orientation and training costs, (3) medical and pharmaceutical costs, (4) education and continuing education costs, and (5) maintenance costs. Data from a diverse sample of fire chiefs has produced a content valid method for calculating the cost of a prevention program among firefighters.


Introduction
Firefighters in the United States suffer cardiac-related line of duty death (LODD) at exceedingly high rates with events 10-100 times more likely to occur during, or in the hours immediately following, fire suppression [1]. The underlying mechanism for cardiovascular events among firefighters is unknown, but it is believed to be multifactorial including both occupational and lifestyle factors. Occupational factors such as shift work, lifestyle factors, and the exposures associated with fire suppression (e.g., smoke, chemicals) may predispose the firefighter to earlier onset of heart disease [2]. Additionally, rates of hypertension, overweight, and obesity within the fire service exceed those seen in the general population increasing the baseline risk for a cardiovascular event [3,4].
Fire suppression activities result in significant cardiovascular strain and provide potential triggers for ischemic events (e.g., myocardial infarction, stroke). There is a rapid rise in heart rate following the activation of a fire department which may persist for as long as 20 minutes [5,6]. Even in cases where heavy work is not being performed, the repetitive upper body exercise associated with tool use raises heart rate disproportionately to oxygen consumption [7]. Finally, the heat stress associated with fire suppression has been shown to enhance coagulation potential and activate platelets reducing platelet closure time [8][9][10].
Given the multiple risk factors and potential triggers for cardiovascular events associated with fire suppression, there is a tangible need for preventative measures among firefighters. Potential interventions include weight loss and fitness programming initiatives and pharmaceutical interventions. Two studies have been funded through the FEMA Fire Prevention & Safety mechanism to examine the role of daily or acute aspirin on platelet activation in firefighters. However, the cost and benefit for these types of programs have not been examined in great detail within the fire service. Lack of a valid method for quantifying costs with any type of intervention, fitness or pharmacological, may prevent program adoption amongst those most likely to benefit from a prevention program. In this study, we sought to create a valid method for quantifying the costs of programs that aim to reduce firefighter mortality related to on-duty cardiac events.

Methods
We used a mixed-method approach approved by the University of Pittsburgh Institutional Review Board that included qualitative in-depth interviews, a cross-sectional survey, and validity calculation. First, in 2012, we performed indepth interviews with four experts. The experts included a convenience sample of fire department chiefs, mid-level department administrators, and a representative of insurance companies. We included a representative from the insurance industry in order to identify the principal components of cost that an insurance company might cover for a fire department. We recruited these experts from a convenience sample of experts listed in an internally maintained database of fire industry contacts who have participated in previous fire safety research. Our in-depth interviews were guided by theories of costs estimation accepted in Human Resource Accounting (HRA), which includes direct and indirect costs [11]. Direct costs refer to materials, goods, and other tangibles commonly listed on balance sheets and budgets as reoccurring or one-time purchases with capital from the organization. Indirect costs refer to the time that organizational staff devotes to specific components of their occupation. We chose to follow a HRA framework based on evidence of "real world" utility and application of previously developed HRA-based cost models [12][13][14][15][16]. We used HRA to frame our interview questions, which were designed to identify common themes/components of costs considered important and relevant by industry experts. For example, we asked expert interviewees to consider and comment on the amount of time employees may devote to activities such as exercising and continuing education while on-duty and the costs that a fire department may incur as a result. This is an example of indirect costs [12][13][14][15][16]. We ended in-depth interviews once the interviewer declared saturation [17]. Saturation refers to the point when the interviewer determines that interviewees are no longer expanding the range of information on a cost component or offering new information beyond that already provided.
Next, in 2013, we randomly sampled 100 fire chiefs from the International Association of Fire Chiefs (IAFC) database of 7,461 fire chiefs located within the USA and internationally. We then contacted each fire chief via a mailed recruitment packet. The packet included (1) a description of the study, (2) letter of support from the National Fallen Firefighters Foundation (NFFF), and (3) a $5.00 gift card. The recruitment letter directed fire chiefs to a study web site where they rated the relevancy of each component of the draft method for calculating costs. We obtained ratings with a fourpoint Likert scale anchored from "Highly Relevant" to "Not Relevant. " Finally, we calculated the content validity of fire chief ratings using the widely accepted Content Validity Index (CVI). The CVI is proportion of raters assigning a rating of 3 = quite relevant or 4 = highly relevant [18]. Two CVI scores may be calculated, one for components (the Scale-CVI or S-CVI) and one for subcomponents (the Item-CVI or I-CVI) [18]. We calculated S-CVI and I-CVI scores based on techniques developed by Lynn, Waltz, and Bausell with S-CVI and I-CVI scores ≥ 0.70 as benchmarks for content validity in this study [18]. We then stratified subcomponents into direct and indirect costs.

Results
In-depth interviews with our four experts resulted in a draft method for cost calculation that included six components and 26 subcomponents. Components included (1) preparatory costs, (2) investment costs, (3) orientation and training costs, (4) medical care and pharmaceutical costs, (5) education and continuing education costs, and (6) maintenance costs. Figure 1 is a graphical illustration of this draft model for cost calculation.
We received 65 completed surveys from fire chiefs (65% response rate). The S-CVI score for preparatory costs (0.66) fell below the 0.70 benchmark suggesting that fire chiefs do not believe this component should be considered a valid part of cost calculation (Table 1). Content validity scores showed that two additional subcomponents fell below the 0.70 cut point for I-CVI. These include one subcomponent of direct costs for the Orientation & Training Costs component and In the 3 months prior to adopting your program, record the amount of organizational capital disbursed to the following: (a) resource documents such as articles and text on cardiovascular disease among firefighters ($$$);  (c) payment for speakers and materials that provide ongoing/focused education or continuing education on components of the program-outside of (in addition to) an existing continuing education program supported by the organization ($$$).

Discussion
Risk of morbidity and mortality from cardiovascular events in the line of duty is high among firefighters [1]. Fire chiefs, fire department insurers, and others share a common concern for firefighter health, wellness, and safety. There are numerous potential solutions for reducing risk; however, the costs associated with these solutions or programs are unknown and likely vary by fire department. We have developed a model to aide fire chiefs that can support the calculation of cost associated with the implementation and maintenance of a program to prevent cardiovascular events or reduce the cardiovascular-related morbidity and mortality among firefighters. Our model represents a starting point from which fire chiefs and others can begin. Application of this model to a candidate program or intervention is required to fully test model efficacy. Lack of data from any actual model application precludes us from evaluating select measures of reliability, construct validity, and predictive validity. The principal benefit of this model is its validity. The components and subcomponents of this model were developed based on a widely accepted framework for cost estimation and calibrated to fit the fire industry with input from 65 randomly sampled fire chiefs.
Like those before us [12][13][14][15][16], we began by developing a framework for cost calculation based on the Human Resource Accounting (HRA) approach. We have demonstrated success with this approach in the past by developing a model for determining the cost associated with turnover in Emergency Medical Services (EMS) organizations [19]. Others have applied the HRA approach as well, which we believe is supportive evidence for HRA's value in development of cost calculation modeling [12][13][14][15][16]. However, we believe that no The summation of direct cost components (Formula 1) can be used as a crude formula for calculating direct cost, unadjusted for total months. Formula 1: = + + + + . Where IC: investment costs, OTC: orientation and training costs, MPC: medical and pharmaceutical costs, ECE: educational and continuing education costs, and MC: maintenance costs. Formula 2 can be used to calculate time-adjusted direct cost biannually and annually.
Where ∑ ℎ is the specified sum of months (i.e., 6 months, 12 months) Formula 3 excludes investment direct costs [ ∑ ℎ ] unique to the first year of program adoption and implementation and is adjusted for biannual and annual calculations. With these data, the estimated total indirect costs for OTC in month 3 would be $824.20. The example above should be replicated for all staff members that engage in activities or tasks linked to any of the six components of program costs (See Table 1). In other words, quantifying indirect costs should not be limited to administrative personnel and should include line-staff workers (front-line workers) who devote time to the program. Failure to consider these indirect costs would produce an underestimation of total program costs to the department/agency. Formula 4 is an example calculation model for quantifying indirect costs linked to time devoted to orientation and training. one approach is optimal and acknowledge that additional research is needed to: (1) fully expose known and unknown factors associated with program cost using different frameworks, (2) to determine feasibility in application in organizations to which they were designed; and (3) to document the perceived utility among the administrators that will use these models.
The value of our work at this time is supported by the growing body of evidence that cardiovascular events are the most common cause of firefighter line-of-duty death and believed to be higher than other professions [1]. We have previously shown that, in one large community, non-fatal cardiac events occur in similar proportions for fire, police, and EMS making this tool widely applicable across public safety [20]. Improving firefighter fitness is a key step in reducing the number of job-related cardiovascular events [21]. However, there are exposures and stressors associated with fire suppression that cannot be mitigated by fitness alone [2]. Therefore, it is reasonable for the fire service to pursue both fitness related programs and other potential solutions such as technology (e.g., physiologic monitoring, enhanced respiratory protection) and pharmacotherapy to reduce the number of cardiovascular events in public safety providers.
Our work will also help expedite future work employing cost-benefit, cost-effectiveness, and related types of economic valuation and analysis techniques. These techniques require multiple steps, including the identification and gathering of data to value costs of diverse decision options [22]. Experts in methods for quantitative synthesis point out that the single most common thread among all techniques is cost valuation or the determination of costs [22]. Given its importance, the overarching goal of cost-effectiveness analysis-to compare the outcome of different decision options in terms of their costs-is not achievable without a comprehensive identification of costs, including direct and indirect costs [22]. In this study, we have demonstrated a focused effort to comprehensively identify the contributors to the cost of a prevention program for firefighters. Our study has addressed a gap not yet filled by the literature and completed a fundamental step to cost analyses that will accelerate future efforts to reduce firefighter morbidity and mortality.
We recognize that many recommendations exist for improving firefighter health and safety. Reports in the peerreviewed literature have examined the utility of standard health surveillance [23], advanced imaging [24][25][26], and the potential need for enhanced respiratory protection following 6 Advances in Preventive Medicine fire suppression [27,28] to mitigate the risk of cardiovascular disease among firefighters. However, few studies have examined the costs associated with these programs [24]. Additionally, there are two trials of aspirin therapy in firefighters to reduce heat stress-induced platelet activation (http://www.clinicaltrials.gov). It is likely that a clinically viable program will be identified to improve the health and safety of the public safety provider. However, given limited resources in a public safety organization, the costs associated with new programs must be balanced against the existing operations' needs of that organization.

Limitations
Our study was limited by the sample of key informants, experts, and stakeholders interviewed and the 65 fire chiefs that responded to our survey. There is reason to believe that additional elements costs may have surfaced with added interviews and responses from additional fire chiefs. However, our sample was selected at random and more than half of fire chiefs surveyed provided input on the relevancy (content validity) of the method for cost calculation.

Conclusions
The cost calculation model presented in this paper represents formative work towards calculating the costs associated with programs and interventions led by fire department administrators targeting firefighters. We believe this model is a starting point for prevention in a higher risk industry and has broad applicability for industries with low penetration of prevention programs. While formative, decision makers may use this model as a framework for determining cost of prevention programs.

Ethical Approval
This study was approved by the University of Pittsburgh Institutional Review Board (IRB).