Reliability and Validity of the Japanese Version of the Implementation Leadership Scale for Nurse Managers and Staff Nurses: A Cross-Sectional Study

Background . Strategic leadership is key to implementing evidence-based practice (EBP). Evaluating the readiness and processes necessary to implement EBP using the Japanese version of the implementation leadership scale (ILS) may be useful to systematically promote the implementation of EBP in Japan. Tis study aimed to evaluate the reliability and validity of the Japanese version of the ILS for nurse managers and staf nurses. Methods . Data were collected in a cross-sectional study. Te original ILS was translated into Japanese and back-translated into English. Clinical nurses reviewed it and confrmed its face validity. We distributed a web questionnaire to 119 nurse managers and 2,858 staf nurses working at three university hospitals in Japan’s metropolitan areas. Construct validity was assessed for nursing managers and staf nurses, respectively, using confrmatory factor analysis. Known-group validity for nurse managers was assessed by verifying diferences in ILS scores by educational background and experience of learning EBP and working on EBP. We evaluated reliability using Cronbach’s alpha and test-retest reliability. Results . Te response rates for nurse managers and staf nurses were 56.3% and 16.9%, respectively. Data from 67 nursing managers and 484 staf nurses were analyzed, excluding duplicate responses. Confrmatory factor analyses of both samples supported the four-factor structure of ILS. Te ILS total score of nurse managers with experience learning EBP or experience working on EBP was statistically signifcantly higher than that of those with no experience, and known-group validity was supported. Across both samples, internal consistency reliability was strong (Cronbach’s alpha: 0.91–0.97) and test-retest reliability was moderate. Conclusion . Tis study illustrated the reliability and validity of Japanese versions of the ILS for both nurse managers and staf nurses. Tis study enabled international comparisons of the leadership required for EBP implementation and may support the development of intervention programs and strategies to promote EBP’s implementation in diferent countries. Trial Registration . UMIN Clinical Trials Registry (UMIN-CTR). Tis trail is registered with UMIN000045782.


Introduction
Evidence-based practice (EBP) is the integration of the best available research evidence with clinical expertise and individual patient values and circumstances [1].Successful implementation of EBP is expected to result in more efective healthcare service delivery, reduced costs, and increased clinician and patient satisfaction [2][3][4].Hence, EBP implementation is widely accepted worldwide as a priority to promote quality clinical practice and optimal patient outcomes [5,6].
Previous studies have indicated that EBP is not consistently used by nurses in clinical settings [7,8].EBP implementation at the unit level has various obstacles, such as fewer opportunities to learn about EBP implementation [9] and nursing managers' passive attitude toward EBP implementation [10].EBP is challenging for staf nurses to implement on their own.Nurse managers play an important role in all processes of EBP implementation because they can infuence the organizational climate of the workplace, the degree of cooperation among team members, and the attitudes of staf nurses toward EBP [11][12][13][14].Terefore, EBP implementation in the unit requires not only the efort of individual staf nurses but also a proactive approach by nurse managers.Nurse manager leadership is an important factor in successful EBP implementation [15,16].
Evidence suggests a relationship between EBP implementation and managerial leadership.Supervisors' transformational leadership and transactional leadership are associated with followers' positive attitudes toward the adoption of EBP [17,18].Nurse manager leadership in EBP implementation is defned as a multidimensional process that infuences staf, their environment, and the organizational infrastructure to facilitate the integration of evidence into clinical practice [19].Nurse managers are required to strategically create an EBP implementation environment and take action to promote the implementation and sustainability of EBP.Strategic leadership involves anticipating organizational change, making strategic decisions, and managing the change process [19,20].A meta-analysis conducted by Hong et al. [13] confrmed that strategic leadership is advantageous for specifc organizational change initiatives.Hence, the strategic leadership of nurse managers can be useful for EBP implementation.
Based on theory and previous research on EBP implementation, leadership, and organizational membership, Aarons et al. [11] developed the implementation leadership scale (ILS), a psychometric scale that assesses the strategic leadership demonstrated by leaders in EBP implementation.Te scale consists of 12 items and four subscales, and its brevity makes it appropriate for busy healthcare settings.Te ILS has both leader and staf versions, which is useful in that it allows evaluation from multiple perspectives.Te ILS, which was originally in English, has been translated into Chinese, Greek, and Norwegian, and its reliability and validity have been confrmed [3,21,22].Previous studies reported that the ILS was associated with the multifactor leadership questionnaire (MLQ) [11,21] and the implementation climate scale [23] as measures of convergent validity and less associated with the evidence-based practice attitude scale (EBPAS) [21,23] and organizational climate measure [3,11] as measures of discriminant validity.Moreover, the ILS has been used to develop intervention programs [24] and as an evaluative tool for educational intervention research [25].As mentioned above, the ILS has been shown to be associated with multiple psychological measures and has been used in intervention research, but its psychometric properties (i.e., which individual characteristics are associated with the ILS scores) have not been adequately tested [3,26].
Te Japanese version of the ILS will allow the exploration of relevant factors, the development and evaluation of educational intervention programs, and international comparisons.As in many other countries, the implementation of EBP in Japan is of growing interest to ensure the best possible care delivery by integrating research evidence and clinical practice.However, EBP implementation and research in Japan are ongoing and developing slowly [27].Te Japanese version of the ILS is expected to promote research on EBP implementation in Japan's healthcare settings.Tis study aimed to evaluate the translated ILS factor structure and psychometric properties of nurse managers and staf nurses in a Japanese nursing context.

Materials and Methods
2.1.Study Design.Tis was a cross-sectional study for the psychometric testing of the Japanese version of the ILS.Te original ILS has two versions: one for leaders to measure their leadership, and one for staf to measure the leadership of the head of their unit.

Te Japanese Version of ILS.
Te original ILS was translated into Japanese by a professional English translator who was not a nursing expert, and the translation was reviewed and confrmed by the authors.Face validity was also assessed by 20 clinical nurses.Subsequently, the ILS was back-translated into English by the authors, and permission was obtained from the original authors.
Te validity of the ILS in the Japanese population was assessed in terms of content validity, construct validity, and criterion-related validity (known-group validity).Reliability was assessed using internal consistency and the test-retest method.

Participants and Sample
Size.Te participants were nurses working at three university hospitals in the Greater Tokyo area of Japan.Using convenience sampling, we selected hospitals where ILS scores were expected to vary among subjects.Te inclusion criterion for participants was nurse managers and staf nurses working at the target facility, and the exclusion criterion was the head of the nursing department.
In setting the sample size, several issues were considered.First, the ILS is available in two types: one for staf and one for leaders; therefore, a minimum of 100 cases were required for each analysis [28].Second, the response rate in previous studies on Japanese nurse managers was approximately 50% [29,30].Tird, the selection of units and staf may raise issues of subject selection bias and ethical considerations.Finally, the results from only one facility may have biased responses.We estimated approximately 50 nursing managers per hospital and a 50 percent response rate.We planned to recruit participants from four hospitals to reach the needed sample size of nurse managers, but we refrained from recruiting one of them due to the COVID-19 pandemic.
For the survey of staf nurses, we were concerned about the arbitrary selection of staf and the possibility of discouraging voluntary participation.Terefore, we targeted all staf nurses in approximately 150 departments at the three facilities to which the targeted nursing managers belonged and estimated a maximum of 3,000 survey forms.

Data Collection.
Te study was conducted from October 2021 to February 2022.We requested for their cooperation in the study and obtained consent from the head of the nursing department at each facility.An online survey tool was used, and leafets with QR codes were distributed to 119 nurse managers and 2,858 staf nurses in the three facilities.Te remainder of the study was sent to nurse managers in January 2022.For test-retest reliability, participants were asked to answer the survey again at least two weeks after the completion of the initial survey.

Demographics.
Participants were asked about their gender, age, academic background, years of experience as clinical nurses, and years of experience working in the unit.

Te Implementation Leadership Scale (ILS). Tis scale measures unit-level leaders' leadership in implementing
EBPs and consists of 12 items and four subscales (proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership).Tis scale has two versions, one for leaders and one for staf.Te leader version is a selfassessment, whereas the staf version is an evaluation of the leader's leadership by others.In the original ILS, because the target of each version is evaluated diferently, the items are written with "I" in the leader's version and the name of the leader to be evaluated in the staf version.Respondents rated the items on a 5-point Likert scale ranging from 0 (not at all) to 4 (very great extent).Te scale scores were calculated as the mean of each subscale and all items.

Experiences in Learning EBP and Working on EBP
Implementation.Te participants were asked about their experiences in learning EBP and working on EBP implementation, respectively, with "No," "Yes," and "Unsure" as possible answers.

Statistical Analysis.
We tested the construct validity, known-group validity, internal consistency, and test-retest reliability of the ILS using the following statistical methods: Construct validity was tested using confrmatory factor analysis (CFA) and the following model ft indexes: comparative ft index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR).CFI and TLI values greater than 0.90, RMSEA values less than 0.10, and SRMR values less than 0.09 indicate an acceptable ft of the model [31,32].Tis study assumed four-factor structure as the original study [11].Furthermore, since staf nurses working in the same unit have the same leader, the CFA was conducted considering a multilevel nested data structure for staf nurses.
We tested for known-group validity using the nurse managers' ILS scores.Nurse leaders' graduate-level education, years of leadership experience, and completion of leadership courses are important factors that positively infuence their leadership for EBP implementation [33].Previous research has suggested that nurses with a master's degree or higher had higher knowledge and skills of EBP and more positive attitudes about EBP implementation than nurses with diploma and baccalaureate degrees [9,34,35].Based on these fndings, we hypothesized that nurse managers with postgraduate degrees would have higher ILS scores than those with other educational levels.Te nurse managers' educational backgrounds were divided into three groups: (a) high school/vocational school/junior college, (b) university, and (c) graduate school.Te Kruskal-Wallis test with a post hoc Bonferroni-Dunn test was used to examine our hypothesis.Furthermore, nurse managers who learned EBP reported enhanced leadership in EBP implementation [36].Mentoring by nurses with adequate EBP experience facilitates EBP implementation [10,37].Tese results imply adequate EBP learning and working by unit leaders, which probably strengthens their leadership for the implementation of EBP.Terefore, we hypothesized that nurse managers with experiences in learning and working on EBP would have higher ILS scores than those without such experiences.Te Mann-Whitney U test was used to verify the prediction that nurses with experiences in learning and working on EBP implementation would score higher than those without such experiences.
Internal consistency was assessed using Cronbach's alpha for each subscale.Te intraclass correlation coefcient (ICC): ICC (1) and average within-group correlation (a wg ) were calculated as indexes of agreement to determine whether staf nurses' scores were a unit-level component.Te ICC (1) was interpreted based on 0.12 proposed by James [38], although there is no clear criterion, with higher values indicating higher group-level agreeableness.Values of a wg greater than 0.60 represent acceptable agreement [39].
In the test-retest method, ICC (2,1) was calculated for each subscale of the ILS.Values of ICC 0.5-0.75 was moderate while ICC< 0.5 was poor [40].Weighted kappa coefcients were calculated for agreement for each item.Te agreement levels were suggested as follows: 0-0.2 (poor), 0.2-0.4(fair), 0.4-0.6 (moderate), 0.6-0.8(substantial), and 0.8-1.0(almost perfect) [41,42].Te analyses were conducted using IBM SPSS ver.28.0 and Mplus ver.8.0.ICC (1), and a wg were calculated using the "multilevel" package [43] in the statistical software R ver 4.2.0.Te signifcance level was set at 5%.Journal of Nursing Management 2.7.Ethical Consideration.Tis study was approved by the Institutional Ethics Review Board of the Chiba University Graduate School of Nursing.Participants accessed the explanatory document via the web page individually from the study participation leafet.After reading it, consent was obtained at the beginning of the study.Participants were given 500 JPY in electronic money as an incentive after responding to the frst survey.

Content Validity.
In the Japanese version of the ILS for staf, for the sake of generality, the name of the leader of an item was set as the "leader of the department."Te content of all items in the Japanese version of the ILS was confrmed through discussions among researchers, and consensus was reached, which was judged to ensure content validity.

Questionnaire Participants and Demographic Data.
Te fow diagram of the study participants is given in Figure 1.In total, 67 nurse manager and 484 staf data were analyzed (response rate; nurse manager: 56.3%, staf: 16.9%), and the response rate for each hospital ranged from 16.8 to 21.5%.Of these, 53 nurse managers and 265 staf nurses responded to the retest survey.Te participant demographics are shown in Table 1.More than 90% of the participants were female.97% of nurse managers were over 40 years old, and 77.5% of staf nurses were under 40 years old.

Internal Consistency and Group-Agreement Indexes.
Table 2 shows the ILS items and means, SD, reliabilities, and aggregation statistics.Cronbach's α ranged from 0.91 to 0.97 for nurse managers and from 0.94 to 0.97 for staf nurses.For staf nurses, the within-group agreement index a wg ranged from 0.47 to 0.56, all items below the criteria of 0.60.ICCs ranged from 0.16 to 0.22, which are generally acceptable values.

Construct Validity.
CFA validated the original model, consisting of 12 items and four factors.Te nurse manager data showed CFI � 0.981, TLI � 0.975, RMSEA � 0.081, and SRMR � 0.038, indicating an acceptable model ft (Figure 2).First-order factor loadings ranged from 0.81 to 0.99, and second-order factor loadings ranged from 0.77 to 0.93.Te staf nurse data showed CFI � 0.981, TLI � 0.973, RMSEA � 0.057, and SRMR � 0.027, indicating an acceptable model ft (Figure 3).First-order factor loadings ranged from 0.86 to 0.97, second-order factor loadings ranged from 0.80 to 0.94.Te Japanese version of the ILS supports the original factor structure for both leaders and staf.
Te ILS total scores of nurse managers were also compared based on their experience learning EBP and working on EBP implementation.Tose with experience learning EBP scored 1.89 (0.84) and those with no experience scored 1.03 (0.57) (U � −3.452, p < 0.001).Tose with experience working on EBP scored 2.04 (0.85) and those with no experience scored 1.00 (0.48) (U � −4.229, p < 0.001).

Validity of the Japanese Version of the ILS.
Tis study translated the ILS into Japanese and confrmed the validity and reliability of the nurse manager and staf nurse versions.CFA, assuming a four-factor model [11], showed a moderate to good ft with the Japanese version of the ILS, suggesting its construct validity.Previous validation studies of translated ILS have supported the original four-factor model [3,21,22], and this factor structure was consistent across studies in diferent contexts.
Testing the known-group validity of the nurse managers' version of the scale showed a statistically signifcant group diference between (a) high school/vocational school/junior college and (c) graduate school.Nurse managers whose last education was (b) college showed no statistically signifcant group diferences compared with the other groups, but the mean of the scale was confrmed to increase according to educational background.Previous studies have reported that factors associated with an individual's EBP implementation included EBP training, university position, higher education, professionalism, and belief in EBP [8], and our results mostly support this.Tese results confrm the known-group validity of the scale for nurse managers.However, this study did not examine the staf's known-group validity.Te relationship with the nurse manager and experience of working together   Tis study reveals diferences in ILS scores according to nurse managers' attributes.Tis knowledge can assist in evaluating the implementation of leadership and developing intervention programs for EBP implementation.

Reliability of the Japanese Version of the ILS.
Cronbach's α, indicating internal consistency, was greater than 0.9 for each subscale for both nurse managers and staf nurses.Te Kappa coefcient, which indicates temporal stability, shows an overall moderate agreement was confrmed.In addition, the ICC of the ILS showed moderate values, confrming temporal stability.With these results, the validity and reliability of the Japanese version of the ILS were confrmed for both nurse managers and staf nurses.Whereas a few items in the nurse manager version showed particularly low temporal stability.Tese items might have infuenced responses at each point in terms of the interpretation of adjectives such as "clear department standard" and "critical issues" in the wording.Terefore, several items need to be examined more closely to improve validity and reliability.Journal of Nursing Management units, as well as to make international comparisons of leadership for EBP implementation.Moreover, ILS was found to be compatible with the Ottawa Model of Implementation Leadership, a theoretical model of implementation leadership [24], thus providing a foundation for intervention research.In future studies, the ILS can be used as an evaluation instrument for educational interventions targeting nurse managers in EBP implementation and compound interventions in facilities.

Linking Evidence to Action. Te Japanese version of ILS allows us to understand the relationship between nurse managers' strategic leadership and EBP implementation in
4.4.Limitation.Te participants were nurses from three university hospitals in a metropolitan area of Japan, which may have biased the population.In addition, university hospitals may have more nurses with experience learning or working with EBP than other hospitals because of their role as educational and research institutions.Terefore, this sample may not necessarily be representative of all clinical nurses in Japan, and additional validation using other samples is needed.Furthermore, this study did not examine associations between the ILS and other scales with similar concepts; further validation of the relationships between other indicators related to leadership is needed in the Japanese context.

Conclusion
Tis study developed Japanese versions of the ILS for nurse managers and staf nurses.Our fndings suggest that it is a valid and reliable measurement of leadership in EBP implementation.Te fndings of this study may contribute to increasing the reliability of assessing EBP implementation in Japan since the ILS is considered an efective tool for measuring leadership when implementing EBP.

Figure 3 :
Figure 3: Second-order confrmatory factor analysis for the ILS of the staf nurses.

Table 1 :
Continued.It includes high school, vocational school, and junior college.

Table 2 :
Implementation leadership scale, subscale, and item statistics.Figure2: Second-order confrmatory factor analysis for the ILS of the nurse managers.