Development and mixed-methods evaluation of a pain assessment video training program for long-term care staff

Centre on Aging and Health, University of Regina, Regina, Saskatchewan Correspondence: Dr Thomas Hadjistavropoulos, Department of Psychology, University of Regina, Regina, Saskatchewan S4S 0A2. Telephone 306-585-4457, fax 306-337-2321, e-mail thomas.hadjistavropoulos@uregina.ca As many as 50% of seniors living in the community and 80% of seniors living in long-term care (LTC) facilities experience persistent pain (1). Health professionals agree that the undertreatment of pain in older adults is a growing ethical concern (2-4). The high prevalence of dementia in LTC facilities increases challenges to pain assessment because individuals may not effectively communicate their pain to staff. Pain is less effectively treated in individuals with cognitive impairments (5-8), although systematic examination of pain expressions can facilitate its evaluation (9). Moreover, pain assessment practice guidelines have been developed to improve quality of care in LTC (10-13) but staff unfamiliarity with pain assessment procedures remains a major barrier to effective pain management (14). Researchers have found that nurses receive inadequate pain education (14-16). Pain-related knowledge among nursing students during or at completion of their training has been deemed to be insufficient (15,17,18). While pain curricula significantly improve pain knowledge among students in health-related programs (19,20), curriculum reviews of Canadian health care programs indicate that pain is rarely covered comprehensively during training (21). Continuing education (CE) presents an avenue for health care professionals to gain knowledge about effective pain practices (22). Pain-focused CE programs implemented among nurses have led to positive outcomes, including increased pain-related knowledge, improved pain assessment ability and increased documentation of patient pain (18,22,23). CE programs have also been found to increase satisfaction and reduce pain among patients (24,25). Despite these positive outcomes, CE for LTC staff is limited due to financial and resource barriers (26,27), and rural and remote staff are often isolated from locations offering programs (28). Moreover, LTC staff who attend CE have reported feeling overwhelmed by the amount of information presented and experience uncertainty about how to original article

l'élaboration d'un programme de vidéoformation sur l'évaluation de la douleur pour le personnel en soins de longue durée et son évaluation par méthodes mixtes InTRoDucTIon : Les lacunes en matière d'évaluation et de gestion de la douleur sont bien étayées en soins de longue durée.La formation insuffisante sur la douleur et les convictions erronées sur la nature de la douleur et le vieillissement pourraient y contribuer.La présente étude porte sur la nécessité d'offrir des formations continues efficaces, faisables et améliorées à l'aide d'une vidéoformation sur l'évaluation.MÉTHoDoloGIe : Au total, 148 membres du personnel en soins de longue durée ont regardé et évalué la vidéoformation.Les chercheurs ont évalué les nouvelles connaissances et les convictions à l'égard de la douleur après la vidéo et lors du suivi quatre semaines plus tard.Ils ont également examiné les convictions à l'égard de la douleur, ainsi que de la douleur et du vieillissement (au moyen d'analyses multivariées), pour déterminer si ces variables influaient sur l'évaluation de la vidéo par les participants.Ils ont aussi organisé des groupes de travail et en ont analysé la transcription au moyen d'une analyse à contenu thématique.RÉsulTATs : Les connaissances sur l'évaluation de la douleur s'étaient améliorées après la vidéo et lors du suivi quatre semaines plus tard.Les participants ont évalué de manière positive le contenu et la qualité de la vidéo.Ceux qui croyaient le plus (au départ) à la nature organique de la douleur donnaient des évaluations plus positives.Les groupes de travail (et l'analyse qualitative) ont permis de déterminer les obstacles à la mise en oeuvre des pratiques présentées dans la vidéo, soit le temps, la charge de travail et la résistance au changement.Quant aux éléments favorables à la mise en oeuvre, ils incluaient un soutien continu de la direction et l'observation des avantages de la mise en oeuvre.eXPosÉ : La présente étude appuie l'utilisation de la vidéoformation.Cependant, compte tenu des résultats du groupe de travail, il faudra peutêtre privilégier des démarches de mise en oeuvre descendante s'appuyant sur la participation continue de la direction pour obtenir des changements soutenus aux pratiques d'évaluation de la douleur.Les chercheurs proposent et présentent un modèle utile pour une mise en oeuvre soutenue qui, ils l'espèrent, facilitera les futures recherches.implement changes on returning to their facility (22,29).A video training approach would address many of these issues by providing the opportunity for staff to review the video as they see fit (eg, more than once) and to watch it at their convenience.
Beliefs about the nature of pain and pain in old age have also been identified as factors influencing pain assessment and management in LTC (25,(30)(31)(32)(33). LTC nursing directors recognize negative staff attitudes and misconceptions about pain in older adults as major barriers to proper pain management (33).If staff are unaware of their inaccurate beliefs, they may not seek out pain assessment programs, thereby jeopardizing these programs' utility and efficacy.

Purpose
The primary purpose of the present study was to develop and evaluate a training video on pain assessment in older adults.As such, this format addresses many previously identified barriers to CE (22,(26)(27)(28)(29).As a corollary, we also aimed to examine the way in which beliefs about the nature of pain influence staff views about the video.
It was expected that video ratings would be positive.In addition, it was hypothesized that nurses would rate the video content more positively than care aides (because of a better preparatory background knowledge base), that participants' knowledge of pain assessment would increase, that these increases would be maintained over four weeks, and that beliefs about pain would be predictive of the valence of video evaluations.

Participants
Care staff (n=148) working in LTC facilities affiliated with two large rural Canadian health regions were recruited to participate in the present study.Demographic characteristics of the sample are summarized in Table 1.Informed consent was obtained and ethics review was completed.

Materials and measures Pain assessment training video:
A training video on pain assessment in LTC with a special focus on individuals with dementia was developed for the present study.The video included a combination of video clips from training sessions on pain assessment in LTC conducted by an expert in the area and on-screen text paired with an audio discussion of the content.Role-played illustrations of pain assessment with an LTC patient, as well as material from interviews with health care staff, were included.The 45 min video discussed the benefits of using pain assessment in LTC, self-report measures of pain assessment appropriate for individuals with mild to moderate dementia, and clinically useful standardized observational approaches (involving use of the Pain Assessment Checklist for Seniors with Limited Ability to Communicate) suitable for individuals with severe limitations in their ability to communicate (34,35).The video concluded with guidelines for assessing pain in seniors with cognitive impairments (10) and suggestions of feasible implementation strategies.A copy of the video can be obtained from the authors.Demographic information: Information regarding participants' sex, age, degree or certification as a nurse, years of experience as a nurse/ care aide and years of experience working in LTC was collected.Participants also self-reported (using 0 to 6 Likert scales) the extent of formal training in pain assessment and in the assessment of pain in dementia.Pain Beliefs Questionnaire: Beliefs about pain were assessed using a modified Pain Beliefs Questionnaire (PBQ) (36).The original version of this questionnaire consists of two subscales, the Organic subscale (PBQ-O), which addresses beliefs about the nature of pain, and the Psychological subscale (PBQ-P), which evaluates beliefs concerning the relationship between psychological factors and pain.The measure has been modified to include an Aging subscale (PBQ-A), which evaluates beliefs about the relationship between aging and pain (37).Stronger endorsement of psychological pain beliefs and weaker endorsement of organic and aging pain beliefs are considered to reflect a more adaptive understanding of the pain experience.The PBQ-O, PBQ-P and PBQ-A subscales of the PBQ have demonstrated acceptable reliability, with Cronbach's alphas of 0.73, 0.70 and 0.73, respectively (36,37).In the current sample, Cronbach's alpha was 0.51 for the PBQ-O subscale, 0.78 for the PBQ-P subscale and 0.69 for the PBQ-A subscale.Knowledge test: To evaluate knowledge gains, a 14-item questionnaire was developed that queried video-specific knowledge about pain assessment using a multiple choice format.The knowledge test (KT) was administered prevideo, postvideo and four weeks following training.Video evaluation questionnaire: The video evaluation questionnaire (VEQ) (31 items) was developed to evaluate participants' appreciation of the video and was administered immediately after participants watched the video.The VEQ includes two scales, the Content Quality Scale (CQS; 11 items), which poses questions (0 to 4 Likert format) related to value seen in the video and willingness to implement the practices, and the Technical Quality Scale (TQS; 14 items) which addresses the technical nature of the video and quality of video production (eg, sound quality).Higher scores were indicative of more satisfaction.The CQS includes two subscales.The first examines perceived general value of the information (General Content Quality; G-CQS) and the second examines value seen in the information in terms of the LTC facility and the participants' specific job (Specific Content Quality; S-CQS).Cronbach's alphas for the G-CQS, the S-CQS and the TQS were 0.87, 0.89 and 0.89, respectively.The VEQ included two additional questions that queried participants' willingness to implement the practices described in the video and whether they felt confident in using these practices.Follow-up questionnaire: The follow-up questionnaire (FQ) was distributed four weeks following training.This 13-item, 0 to 4 Likert scale questionnaire, developed for the present study, inquired as to whether changes in pain assessment practices had occurred.Cronbach's alpha for the FQ was 0.91.

study design
Administrators from participating facilities informed qualifying staff members of the study.Interested participants registered to attend a training session.A total of 13 training sessions were organized across the two health regions.At each training session, participants were asked to complete a demographic information sheet, the KT and the PBQ.Next, the 45 min training video was presented.On its conclusion, participants were asked to complete the VEQ and to repeat the KT.Four weeks following the training, participants were mailed packages containing FQs and KTs.The questionnaires were returned to the researchers using prestamped envelopes.
Focus groups and individual interviews were arranged with a subset of participants who were nominated by the facility administrators as influential leaders within their peer group, as per qualitative

Quantitative results
Video evaluation and group differences: Mean scores and SDs (Table 2) on the CQS (G-CQS and S-CQS), and TQS of the VEQ were used to determine participants' evaluation of the video content, the presentation of the information and the technical aspects of the video.Scores reflected satisfaction with the content and quality.Independent-samples t tests were conducted to examine differences between nurse and care aide CQS total score (ie, sum of G-CQS and S-CQS) and CQS subscale scores.Significant differences were found on total CQS score (t[145]=2.75;P=0.007) and on S-CQS (t[145]=3.38;P=0.001), with nurses providing more positive evaluations.A large proportion of participants (85.3%) reported that they would feel confident using the practices taught in the video, and the majority of participants (90.0%) indicated being willing to implement the practices.overall learning: A two-way mixed factorial ANOVA was conducted to compare scores on the prevideo, postvideo and follow-up KTs, with nurses' and care aides' results compared between groups.The main effect of time was significant (F[1.83,86]=102.81;P<0.001, partial η 2 =0.54).Pair-wise analyses indicated that scores significantly increased from prevideo to postvideo (P<0.001) and from prevideo to follow-up (P<0.001),but significantly decreased between post-test and follow-up (P<0.001).There was no main effect of job type (ie, nurse or care aide) and the interaction between job level and time was not significant.

Role of beliefs in video evaluation:
To determine whether maladaptive beliefs about the nature of pain and pain in old age would be significant contributors to overall video evaluation, multiple regression was used.Three regressions were conducted, a separate one with each of G-CQS, S-CQS and FQ as the dependent variable.The predictors of age, sex, job type (nurse or care aide), years of experience in LTC, extent of past pain assessment training, extent of past pain assessment in dementia training and the three PBQ scores (one score for each subscale) were included.If the overall models were found to be significant, each variable's unique contribution to the regression was examined using a conservative regression approach in which each predictor's ability to account for unique variance was examined after all other predictors had been entered into the model.

Qualitative results
Following from the results of the self-report measures, qualitative analysis focused on understanding whether the training led to a change in practice and on identification of facilitators and barriers to change implementation; herein, participants' perceptions of the video's usefulness are also summarized.Thematic content analysis (38,39) was used to organize the text, first based on whether it referenced video evaluation and utility, or pain assessment in the facility, and subsequently into thematic categories using a bottom-up procedures.To ensure consistency of the coding and trustworthiness of themes, a second coder analyzed a random 15% of transcripts into the established themes.Disagreements were discussed and resolved by consensus.Problematic themes were reconsidered and reorganized.Member checking, implemented to ensure accurate interpretation of the data, was conducted with all focus-group participants (40).
Participants stated that the qualitative results were consistent with their experiences.The discussion regarding video evaluation and utility was consistent with the quantitative findings.Participants were pleased with the training and found the information useful and relevant.However, participants were unable to recall specific details of the practices and did not link their descriptions with behavioural examples indicative of changes in practices (see Discussion section for a recommendation concerning this issue).Participant discussions of pain assessment practices currently in use within the facilities similarly denoted lack of implementation.
The themes related to pain assessment in the facility were examined to provide a better understanding of factors that interfere with or facilitate a change in practices (Table 4).Factors identified as barriers to implementation included insufficient time, insufficient staff, and coworker negativity and resistance to change.Staff who believed that there was sufficient time still identified time as an issue, describing it as challenging to overcome the negativity of staff who perceived time as lacking.
Management support was identified as an essential factor to successful changes in practices.Participants indicated that large-scale change had to be initiated by management and that support must be ongoing throughout all phases of implementation.Participants emphasized the necessity of management awareness of the experiences of front-line staff and availability to address concerns.Without sufficient support, participants suggested that the barriers could not be overcome.
Witnessing the benefits to implementation was identified as a second essential factor to successful change.Participants described

Proposed model of successful change in pain assessment practices
Despite positive attitudes about the training, participants were not implementing the recommended practices.Quinn ( 41) defines a successful CE program as one that results in a change in practices.Thus, implementation of the practices would represent optimal success of this training program.To improve the success of future training using this program and to increase changes in practices following training, a model, subject to cross-validation by future research and offering a conceptualization of the qualitative findings, is proposed (Figure 1).This model focuses on the two main aspects identified by staff as facilitators to change: continuous management support and the importance of seeing the benefits.A perception of how changing practices would alleviate existing challenges was identified as necessary to change.Quantitative results suggest that staff members perceive these benefits immediately following training.The perception of benefits alone did not lead to any lasting change in practices.It is possible that participants initially saw ways in which pain could be better assessed and managed despite existing challenges, but on return to the facility, perceived these issues as too numerous and too overwhelming to enable change.Management support becomes essential to allow staff members to experience the true benefits of the practices through mandated implementation, continuous emphasis of the benefits and recognition of the pain assessment challenges staff members face.To counteract the barriers to implementing new practices, managers must be aware of how implementation is occurring at the front-line level and respond to staff concerns.Managers will influence the degree to which staff members are able to overcome barriers and experience the benefits of the pain assessment practices.While additional research is needed to investigate the proposed model, some of its basic tenets are consistent with other implementation approaches discussed in the literature.For example, Gnaedinger (42) underscored the importance of leadership, a team approach to overcoming obstacles and ongoing training.Moreover, Stolee et al (26) obtained preliminary research support for several aspects of our proposed model including the importance of securing management commitment and on-the-job support.

DIscussIon
The present investigation provides support for the use of this videobased program as a method of knowledge translation for LTC staff.Video evaluations suggested that, in general, participants were pleased with both the technical presentation of the video and the content and information in the video.Nurses provided more positive ratings, which may be due to differences in training.Care aides receive less training on pain assessment in their initial training program and may be less aware than nurses of the importance of conducting pain assessments.
Knowledge of pain assessment in LTC was significantly greater following the video; however, knowledge decreased between the postvideo phase and the four-week follow-up.These findings suggest that this tool can effectively educate staff in evidence-based pain assessment practices, but support the necessity of refresher or booster training, especially in the absence of immediate clinical implementation.
It was expected that beliefs would influence participants' evaluation of the video and implementation of practices.Organic beliefs resulted in more positive evaluations of the degree to which the content of the video was relevant to participants' specific job and duties in the LTC setting.It is possible that individuals who more strongly believe in the organic nature of pain found the video most beneficial because it contained information about the role of nonorganic factors in the pain experience.
The content of the video is consistent with evidence-based recommendations and may assist LTC facilities in meeting clinical and policy pain management recommendations (43).The video format addresses many of the previously identified challenges to CE programs (22,27,28) and, with proper implementation, this training tool has the potential to facilitate pain management decision making (44) and to address the ethical concern of the undertreatment of pain in older adults (2).
Our results indicate that many obstacles are present for LTC staff wishing to change pain assessment practices following training.These obstacles are formidable for front-line staff to overcome on their own.Participants in the present study stressed the need for strong, ongoing management support for a training program to result in a change in practices.This idea is further reinforced by the qualitative finding that staff were unable to recall certain details of the practices in the video and suggests that, with central management support, additional training time can be allocated to enable review and consolidation of the knowledge that was conveyed.
Nurse perceptions underscore the need for training programs to account for pre-existing challenges in LTC and provide strategies for mitigation.Bottom-up approaches to CE in which staff are educated in the hope that practices will be implemented have demonstrated moderate success (22,25).The present study suggests that these approaches may not be realistic given the resources and hierarchy of the LTC setting.Bottom-up approaches require a small subset of staff members to advocate for the implementation of practices, which may not be possible given staffing and time limitations.Instead, the findings of the present study support an approach involving consistent and continuous management involvement.
With this said, we acknowledge, as a limitation to our conclusions, that data on the perspectives of managers regarding this issue were not available.As such, we encourage future research on the validity of our recommendation concerning the importance of management support.Nonetheless, we do believe that such support, sustained over the long term, would likely result in better pain assessment and management within the facility.An added benefit would be that improved pain assessment and management has been shown to improve workplace environment for the front-line staff and quality of life for LTC residents (45).

lessons FoR PRAcTIce
Video-based pain assessment training can be an effective way to increase knowledge of pain assessment practices among LTC nurses and care aides.Copies of the video used in the present study can be obtained from the authors.Focus group results suggested that strong management involvement is essential to the implementation of changes in pain assessment practices in a LTC setting.

Figure 1 )
Figure 1) Proposed model of successful change in pain assessment practices.The dark arrow along the right of the model depicts management involvement in the staff members' experience.For front-line staff, the process begins with the completion of the training program (Box 1).Participants perceived several benefits to the training program (Box 2).Pre-existing challenges to pain assessment (Box 3) may interfere, making it challenging for staff members to adjust their practices independently.Management should oversee the implementation of practices (taking a consultative team approach) while providing ongoing support to enable actual implementation to occur (Box 4).Barriers to implementation (Box 5) must be addressed.The true outcomes (Box 6) of using the pain assessment practices will occur.Sufficient management support should lead to the positive outcomes (Box 7); however, should negative outcomes be experienced (Box 8), these must be immediately addressed with management support.With the experience of benefits and continued management support, sustained changes should occur within the facility(Box 9)

Table 4 Facilitators and barriers to changing practices and sample responses Facilitators to change
Seeing the benefits to implementation "Show us how to use it, the benefits.Like what are we going to get out of this that we aren't doing right now." (N)Management support and follow-up "It seems like all the managers are done at 3:30 [p.m.] and they're gone, and you can't carry these things out as a nurse on your own.You're responsible for so much.When you bring in change you got to oversee it, seven days a week.Not Monday to Friday."(N) of certain people here, and I think, short of an atomic bomb, nothing will change their outlook on life . . .there are some, they would rather die than change.'We've done it this way, and we're always going to do it this way, and don't tell me that I have to change.'"(N)