Evaluation of a novel method of teaching endobronchial ultrasound : Physician-versus respiratory therapist-proctored simulation training

1Department of Medicine, University of Calgary, Calgary, Alberta; 2Department of Medicine, University of Sherbrooke, Sherbrooke, Quebec; 3Division of Respirology, Critical Care and Sleep Medicine, University of Saskatchewan, Saskatoon, Saskatchewan; 4Department of Medicine, McMaster University, Hamilton, Ontario Correspondence: Dr David Ryan Stather, Division of Respiratory Medicine, Health Sciences Center, 3330 Hospital Drive Northwest, Calgary, Alberta T2N 4N1. Telephone 403-210-3866, fax 403-944-1577, e-mail davestather@yahoo.ca Endobronchial ultrasound (EBUS) is a minimally invasive respirology procedure that has revolutionized diagnostic bronchoscopy. EBUS enables safe, highly accurate biopsies of intrathoracic structures during bronchoscopy (1,2). The acquisition and interpretation of ultrasound images during EBUS can be challenging. Some experts suggest that the slow implementation of this technique is due to the significant learning curve associated with EBUS (3-5). Teaching EBUS is an essential component of patient care; it is crucial to provide adequate numbers of trained physicians to meet patient needs for this specialized procedure. Flexible bronchoscopy has historically been taught via the apprenticeship method: a patient-care approach whereby trainees learn by practicing on patients. When compared with procedures without trainee participation, however, flexible bronchoscopy procedures (including EBUS) with trainees learning via the apprenticeship model have been demonstrated to have increased procedure time, increased sedation requirements and increased complication rates (6-8). Simulation-based training has been demonstrated to be effective, cost effective, and to increase patient comfort and safety when used for surgical and medical procedural education (9-13). Although there is currently no accepted standard teaching method for EBUS, the use of computer and animal simulation has been shown to be effective (14). Computer EBUS simulators have been validated, with studies demonstrating the ability of the EBUS simulator to discriminate between operators of different clinical EBUS experience levels (5). The use of an EBUS simulator has been shown to improve the rate of procedural skill acquisition in respiratory medicine trainees (15), with other studies demonstrating skill transfer from the simulated environment to reality (16). original articlE

When proctored by an interventional respirologist (IR), computer EBUS simulators have been demonstrated to improve trainee procedural skills (15).This, however, takes a considerable amount of time (15,16) and may not be a sustainable teaching method given the current demand for IRs.The present study aimed to compare EBUS performance in respiratory medicine trainees following training with computer simulation proctored by specially trained EBUS respiratory therapists (RTs) versus the same simulation training proctored by an IR.

Study design
The present analysis was a prospective, nonrandomized study of two cohorts of learners.

EBuS simulation training
All trainees in both groups underwent a 30 min introductory session to the EBUS simulator to familiarize them with the equipment.Before proceeding with the simulation training, all trainees underwent a standardized education process to ensure they had knowledge of the different lymph node stations and what is considered to be a complete EBUS lymph node examination.To simplify and standardize the examinations, the same clinical scenario -a left-sided tumour that required lung cancer staging -was used for every case.Trainees were instructed that only the 11R, 4R, 7, 4L and 11L stations were to be examined in all patients.Trainees were not permitted to proceed with the simulation training until they demonstrated understanding of a correct lymph node station examination and the associated airway and vascular anatomy by correctly answering a series of standardized questions (Appendix A).The introduction session and knowledge testing (Appendix A) were performed by an IR for both study groups.
Trainees from both groups then completed two introductory simulator-based EBUS cases.They subsequently performed 15 additional cases (EBUS simulation training).All trainees performed the same EBUS training cases in the same order.Cases 9 and 13 were more difficult than the other cases because the trainees were asked to biopsy subcentimetre lymph nodes (by design, to improve their biopsy technique).intervention group 1: All EBUS simulation training was proctored by an IR. group 2: Group 2 received EBUS simulation training identical to group 1; however, all training was proctored by an RT with special training in EBUS.
All trainees in groups 1 and 2 were then asked to complete two simulator-based EBUS test cases with EBUS performance metrics recorded by the simulator to evaluate their post-training EBUS skill level using a previously validated method (5).All testing was completed by an IR (DRS or AC).

Rt training
The RTs participating in the present study had >2 years of experience assisting with clinical EBUS in the bronchoscopy suite.They subsequently completed the EBUS simulation training (19 cases on the simulator as previously described [15]) proctored by an IR.Each RT spent an additional 4 h with an IR analyzing the simulated EBUS cases to be taught in detail, and learning EBUS teaching methods.

Measurement tools
Subjects were given a presimulation questionnaire that assessed demographic information (Appendix B).The EBUS simulator software comes with a number of built-in metrics that are recorded during each simulated case including total procedure time, time to intubation, number of successful lymph node aspirations, quantity of lidocaine used, quantity of sedation used and number of intubation attempts.
Two additional clinically relevant measurement metrics included were the percentage of lymph nodes correctly identified during lymph node ultrasound examination and the percentage of successful lymph node punctures.The percentage of lymph nodes correctly identified was defined as the lymph nodes verbally identified by the learner divided by the total number of lymph nodes for each case.The percentage of successful lymph node punctures was defined as the total number of successful attempts divided by the total number of attempts.Trainees were required to perform three successful punctures per lymph node station, up to a maximum of five attempts.

learning curve evaluation
The EBUS simulator performance metrics were monitored over the course of 19 cases (two introduction cases, 15 practice cases and two final test cases) in both groups.A composite value, the 'EBUS Efficiency Performance Score', defined as the percentage of lymph nodes correctly identified on EBUS examination/total procedure time was also calculated.The EBUS Efficiency Performance Score is believed to be more representative of true EBUS performance including values of procedural time and procedural accuracy (5,(14)(15)(16).

Statistical analysis
Three performance metrics were selected a priori to be most relevant to the assessment of EBUS skill (primary outcome measures): total procedure time; percentage of lymph nodes correctly identified on lymph node examination; and percentage of successful lymph node biopsies (successful lymph node biopsies/[failed lymph node biopsies + successful lymph node biopsies]).The results were analyzed between groups using the averaged performance from the two EBUS test cases, using a standard t test.When data were not normally distributed, the Fisher's exact test was used.Analysis of the learning curves between groups was performed using a repeated measures ANOVA.Power calculations: In a previous study (5), the mean total procedure time (in minutes) within each trainee group was normally distributed with an SD of 2. Using 10 experimental trainees and four control trainees, it was possible to detect a true difference in the mean response of experimental and control trainees of ±4.2 min with probability (power) 0.9 and alpha of 0.05.Previous data assessing the percentage of lymph nodes correctly identified revealed that the expected rate among controls is 0.9.Using 10 experimental trainees and four control trainees, it was possible to detect a true failure rate of 12% in exposed trainees with probability (power) 0.9 and alpha of 0.05.An uncorrected χ 2 statistic was used to evaluate this null hypothesis.Previous data assessing the mean EBUS efficiency score revealed a normal distribution with an SD of 0.5.Using 10 experimental trainees and four control trainees, it was possible to detect a true difference in the mean response of experimental and control trainees of ±1.0 with probability (power) 0.9 and alpha of 0.05.

RESultS
Fourteen trainees were enrolled, with four IR proctored (group 1) and 10 RT proctored (group 2).All trainees completed the study.Table 1 summarizes trainee demographics.The EBUS performance metrics for the combined EBUS simulator test cases for each study group at the end of EBUS simulation training are shown in Table 2. Testing was completed within seven days of EBUS simulation training in all trainees in both groups.
Figures 1A to 1D illustrate mean on-simulator learning curve data for trainees in both study groups.The learning curves were highly similar between groups and did not show an obvious plateau after 19 simulated cases in either group.The EBUS simulation training (15 cases) took a mean of 7.11 h with a range of 5.65 h to 10.81 h per learner, involving five or six simulator sessions over a time period of ≤2 months in both groups.There were no differences in the length of time for the EBUS simulation between groups.

DiSCuSSion
The present study demonstrated that acquisition of basic EBUS technical skills can be achieved using computer EBUS simulation proctored by specially trained RTs or by an IR.Our data suggest that the level of EBUS technical skill obtained via EBUS simulation proctored by the RTs is comparable with the skill level obtained in trainees proctored by an IR.
Computer bronchoscopy simulators have been validated in the assessment of bronchoscopy skill for basic bronchoscopy and EBUS (5,18,19), demonstrating improved rates of procedural skill acquisition in trainees when compared with clinical EBUS training (15), with the skills learned on the simulator being transferable to clinical EBUS skills (16).These studies used IRs to proctor the simulator learning (5,(14)(15)(16), an exceedingly time-consuming process (a mean of 7 h for an individual trainee to achieve a basic level of EBUS skill in our study).It appears impractical, expensive and unsustainable to have an IR devote this length of time to each trainee.RTs perform diagnostic and therapeutic procedures (ie, arterial blood gases, endotracheal intubation) and, in many centres, teach these procedures to other health care professionals, including physicians in training.In our centre, we expanded the role of some specially trained RTs to include teaching physicians how to perform both basic and advanced flexible bronchoscopy techniques, including EBUS.
Using data from previous studies of EBUS simulation at our centre (5,(14)(15)(16), we were able to identify what we believed to be clinically significant differences in our primary outcome measures (total procedure time, percentage of lymph nodes successfully identified and percentage of successful biopsies) and to then power the present study accordingly.We also used a previously validated method to measure EBUS technical skill (5).These factors lend confidence to the validity of our results, whereby no significant differences were found between groups for either teaching method, suggesting that the level of EBUS technical skill obtained via EBUS simulation proctored by specially trained RTs is comparable with the skill level obtained in trainees proctored by an IR.
There is a 'proctor' built into the EBUS simulator software that provides learning pointers during the simulated cases.Based on our experience with the use of the computer proctor for teaching trainees and RTs, we believe that the EBUS simulator alone is inadequate for learners to learn this complex technique, given the sometimes confusing and occasionally incorrect information provided by the computer proctor.Perhaps future improvements in computer simulator proctor software will eventually warrant further study with a self-directed learner group.
Our study evaluated the training of learners with regard to the technical skills of EBUS, with no attempts at evaluating the cognitive aspects of EBUS (patient selection, result interpretation, etc).The trainees in the present study performed fewer than the recommended number of procedures suggested for EBUS competency (3,4); however, our goal was not to assess competency for independent practice or to suggest that simulator training alone could lead to competency, but rather to determine whether the technical skills obtained via EBUS simulation proctored by specially trained RTs are comparable with the skills obtained in trainees proctored by an IR.Given the mounting evidence that there is significant variability in the rate of trainee procedural skill acquisition (15,20), the use of a combined approach of IR teaching, RT-proctored EBUS simulation and clinical cases may enable an approach specifically tailored to each individual trainee's learning styles.
Few studies have investigated the EBUS learning curve.Early data from a multicentre study investigating the EBUS learning curve (21) demonstrated ongoing improvements in technical skill, even after 200 clinical EBUS procedures.Another study demonstrated that no plateau was observed in EBUS technical skill after 19 simulated cases (15).Both groups in our study demonstrated similar EBUS learning curves, with no plateau in total procedure time, percentage of lymph nodes correctly identified or EBUS proficiency score apparent after 19 cases.The reduction in metrics values apparent in cases 9 and 13 was apparent in all subjects and corresponded to two particularly difficult cases, designed to give the learners experience sampling small, subcentimetre lymph nodes.Further study with larger numbers of trainees and EBUS cases are required to establish a better understanding of the true EBUS learning curve.A limitation of the present study was that trainees were not randomized; therefore, factors impacting skill acquisition and performance may not have been equally distributed between the two groups, possibly resulting in selection bias.Due to the limitations of a yearly fellowship program, the groups consisted of small sample sizes.Confirmation of our findings with larger, randomized trials would be ideal.Although the EBUS simulator records objective performance metrics data, having IRs perform the testing is another study limitation due to the possibility of bias during testing.
In summary, the present study demonstrated that acquisition of basic EBUS technical skills can be achieved using computer EBUS simulation proctored by specially trained RTs or by an IR.There appeared to be no significant advantage to having an IR proctor the simulation training.Using such simulation teaching methods minimizes the number of patient care experiences required to achieve competence, reduces patient discomfort and risk during the learning process and may further assist in providing the necessary training to ensure that an adequate number of physicians develop the skills required to meet the increasing patient needs for this important specialized procedure in the future.Given the significant learning curve and the substantial length of time required to learn this challenging technique, this novel approach of using specially trained RTs to teach EBUS provides a more cost-effective, sustainable method of teaching EBUS to trainees.AuthoR ContRiButionS: Dr Stather: had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.Dr Stather contributed to study concept and design, acquisition analysis, interpretation of data; statistical analysis: drafting of the manuscript; and critical revision for important intellectual content.Dr Chee: contributed to study concept and design; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.Dr MacEachern: contributed to study concept and design; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.Dr Dumoulin: contributed to study concept; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.Dr Hergott: contributed to study concept; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.Dr Gelberg: contributed to study concept; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.S Scott: contributed to study concept; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.S Deguzman: contributed to study concept; acquisition analysis; interpretation of data; drafting of the manuscript; and critical revision for important intellectual content.Dr Tremblay: contributed to study concept and design, acquisition analysis; interpretation of data; statistical analysis: drafting of the manuscript; and critical revision for important intellectual content.

:
This work was performed at the Division of Respiratory Medicine, University of Calgary; Tamarratt Research Bronchoscopy Suite, Calgary, Alberta.The University of Calgary has received grants from Olympus Canada for support of an Interventional Pulmonary Medicine Training Program and for CME events relating to EBUS.

Figure 1 )
Figure 1) Mean learning curves from group 1 and group 2. A Total procedure time.B Percentage of lymph nodes correctly identified.C Percentage of successful biopsies.D Endobronchial ultrasound (EBUS) performance efficiency score.IR Interventional respirologist; RT Respiratory therapist