Wet laboratory versus computer simulation for learning endobronchial ultrasound: A randomized trial

L inear endobronchial ultrasound with transbronchial needle aspira- tion (EBUS-TBNA) is a revolutionary diagnostic procedure that enables visualization and highly accurate biopsies of intrathoracic structures during bronchoscopy in a safe, cost-effective outpatient approach (1-4). EBUS-TBNA can be challenging to learn, even for experienced bronchoscopists (5-12). Growing demand has resulted in a rapid increase in the number of EBUS-TBNA continuing medical education (CME) courses being offered (13). Although there are many available different methods of learning EBUS-TBNA, there is currently minimal published literature on the topic, with no widely accepted method of learning EBUS-TBNA. Methods of learning EBUS-TBNA include the apprenticeship model (practicing on patients), use of inanimate models, use of live or cadaveric animal models and the use of computer-based simulators. learning randomized BACkgRounD: Linear endobronchial ultrasound with transbronchial needle aspiration (EBUS-TBNA) is a revolutionary bronchoscopic procedure that is challenging to learn. oBJECTivES: To compare two methods used to teach EBUS-TBNA: wet laboratory (lab) versus computer EBUS-TBNA simulation. METHoDS: A prospective, randomized study of respirologists, thoracic surgeons and trainees learning EBUS-TBNA at a two-day continuing medical education course. All subjects received education via a series of lectures and live cases, followed by randomization to learn EBUS-TBNA predominantly either by wet lab simulation (n=6) or computer simulation (n=6). All subjects then completed testing of their EBUS-TBNA skills via a previously validated method using simulated cases on EBUS-TBNA simulators and questionnaires evaluating learner preferences. RESulTS: There were no significant differences between the computer EBUS-TBNA simulator group and the wet lab group in procedure time (25.3±6.1 min versus 25.2±2.5 min; P=0.984) and percentage of successful biopsies (81.3±14.9% versus 74.0±17.3%; P=0.453). The computer simu- lator group performed significantly better than the wet lab group in the percentage of lymph nodes correctly identified (70.4±16.7% versus 42.9±19.9%; P=0.002). Wet lab simulation was associated with increased learner confidence with operating the real EBUS-TBNA bronchoscope. All subjects responded that wet lab and computer EBUS-TBNA simulation offered important complementary learning opportunities. DiSCuSSion: Computer EBUS-TBNA simulation leads to improved skill at correctly identifying lymph nodes, while wet lab simulation pro- vided increased learner confidence due to increased realism. ConCluSion: Computer EBUS-TBNA simulation and wet lab simula- tion are effective methods of learning basic EBUS-TBNA skills and appeared to be complementary.

L inear endobronchial ultrasound with transbronchial needle aspiration (EBUS-TBNA) is a revolutionary diagnostic procedure that enables visualization and highly accurate biopsies of intrathoracic structures during bronchoscopy in a safe, cost-effective outpatient approach (1)(2)(3)(4). EBUS-TBNA can be challenging to learn, even for experienced bronchoscopists (5)(6)(7)(8)(9)(10)(11)(12). Growing demand has resulted in a rapid increase in the number of EBUS-TBNA continuing medical education (CME) courses being offered (13). Although there are many available different methods of learning EBUS-TBNA, there is currently minimal published literature on the topic, with no widely accepted method of learning EBUS-TBNA.
Methods of learning EBUS-TBNA include the apprenticeship model (practicing on patients), use of inanimate models, use of live or cadaveric animal models and the use of computer-based simulators.
With the exception of the apprenticeship model, all of these learning models are currently in use in EBUS-TBNA CME courses. The nature of the apprenticeship model requires that learners practice on numerous patients; therefore, it is not practical for one-or two-day CME courses.
training methods using practice on patients, with the skills learned on the simulator being directly transferable to clinical EBUS skills (12). Wet laboratory (lab) simulation has been described previously (14) and is often used in EBUS-TBNA CME courses; however, the authors are not aware of any published studies evaluating its efficacy.
Wet lab simulation and computer simulation techniques each have specific advantages and disadvantages for teaching EBUS-TBNA. There are ethical, cost and staffing issues associated with the use of wet lab models that do not exist with the computer EBUS-TBNA simulator. There are, however, several advantages of wet lab models over the computer model, including increased realism and use of the actual EBUS-TBNA bronchoscope equipment. The aim of the present study was to compare wet lab model simulation versus computer simulation for the acquisition of basic EBUS skills, and to determine which teaching method learners preferred.

Study design
The present study was a randomized trial comparing EBUS-TBNA performance and preferences following wet lab versus computer EBUS-TBNA simulation training. Subjects consisted of 12 volunteers who attended a two-day EBUS CME course (University of Calgary, Calgary, Alberta, February 10 to 11, 2011). Course attendance was limited to 20 participants to ensure adequate hands-on experience (due to a cancellation, the course had 19 participants). The course consisted of lectures and live case demonstrations in the mornings, followed by simulation-based education in the afternoons. All wet lab sessions were approved under the University of Calgary Animal Care Committee protocol M09109. The present study was approved by the Calgary Health Research Ethics Board (Ethics ID #23665). All subjects provided written informed consent.

Study subjects inclusion criteria:
Physicians having completed a respirology or thoracic surgery training program, or senior trainees in respirology or thoracic surgery training programs. Exclusion criteria: Physicians with clinical EBUS-TBNA experience >20 cases, >1 h computer EBUS-TBNA simulator experience and >1 h previous wet lab EBUS-TBNA experience.
Subjects were randomly assigned to either the EBUS-TBNA simulator training group or the wet lab simulation training group using block randomization via a computer-generated list to ensure six subjects per group.

intervention -day 1 EBuS-TBnA simulator training (n=6):
The EBUS-TBNA simulator training group received four 45 min training sessions using the computer EBUS-TBNA simulator, in groups of three subjects per simulator proctored by an interventional respirologist experienced with the EBUS-TBNA simulator. Wet lab training (n=6): The wet lab training group received three 45 min EBUS-TBNA training sessions using wet lab simulation. To ensure that the subjects were oriented to the simulator system before the performance evaluation, they also received a 45 min training session using the computer EBUS-TBNA simulator (three subjects per training model). Baseline knowledge assessment: During their first computer EBUS-TBNA simulator session, all subjects (both groups) were reminded of the different lymph node stations and what is considered to be a complete EBUS-TBNA lymph node examination. Subjects were tested before their postsimulation training testing to ensure they had a complete understanding of a correct lymph node station examination (Appendix A). Evaluation of EBuS-TBnA skill: Following their respective training sessions, all subjects were given one EBUS-TBNA case to complete, with performance metrics recorded by the EBUS-TBNA simulator to evaluate subject performance based on a previously validated assessment of clinical EBUS-TBNA skill using the EBUS-TBNA simulator (7). Subjects were also given a post-training questionnaire to evaluate learner preferences regarding their respective training methods (Appendix B).

EBuS-TBnA hands-on simulation training sessions -day 2
On the second day of the course, all subjects participated in both EBUS-TBNA computer simulator sessions and wet lab simulation sessions, ensuring that everyone received significant experience with both learning methods. At the end of the course, all subjects (including the seven nonstudy participants, who were all respirologists who met the exclusion criteria) were given a questionnaire analyzing training method preferences (Appendix C). EBuS-TBnA simulator description: The EBUS-TBNA simulator used in the present study was the AccuTouch Flexible Bronchoscopy Simulator equipped with an EBUS-TBNA module (CAE Healthcare, Canada), described in detail previously (7,15,16). Wet lab description: Learners practiced localizing lymph nodes and blood vessels using actual EBUS-TBNA bronchoscopes on live anesthetized swine. The size and type of swine used ensured similar calibre airways to humans, and all animals had identifiable lymph nodes. Learners practiced EBUS-TBNA on the lymph nodes, including sample preparation as per protocol at the University of Calgary. An interventional respirologist or thoracic surgeon experienced with EBUS-TBNA proctored each station.

Measurement tools
The EBUS simulator software comes with built-in metrics that are recorded during each case (7,10,12). Two additional clinically relevant measurement metrics included were the percentage of lymph nodes correctly identified during lymph node ultrasound examination and the percentage of successful lymph node punctures. The percentage of lymph nodes correctly identified was defined as the lymph nodes verbally identified by the learner divided by the total number of lymph nodes for each case. The percentage of successful lymph node punctures was defined as the total number of successful attempts divided by the total number of attempts. Learners were required to perform three successful punctures per lymph node station, up to a maximum of five attempts.
A composite value, the 'EBUS Efficiency Performance Score', defined as the percentage of lymph nodes correctly identified on EBUS examination/total procedure time was also calculated. The EBUS Efficiency Performance Score is believed to be more representative of EBUS performance because it includes both values of procedural time and procedural accuracy, and has been previously validated (7,10).

Statistical analysis
Three performance metrics were selected a priori to be most relevant to the assessment of EBUS-TBNA skill (ie, primary outcome measures): total procedure time, percentage of lymph nodes correctly identified on lymph node examination and percentage of successful lymph node biopsies (successful lymph node biopsies/[failed lymph node biopsies + successful lymph node biopsies]). The results were analyzed between groups using the performance from the EBUS test case, with a standard t test for procedure time and EBUS efficiency score, and c 2 test for the percentage of lymph nodes correctly identified on lymph node examination and percentage of successful lymph node biopsies and questionnaire responses.
The study was limited by the small number of participants attending the course. Nevertheless, using a standard t test and including six subjects per group would allow the detection of a ±6 min difference in procedure time between groups (alpha of 0.05 and power of 0.80 with an SD of 3.5 min).

RESulTS
Twelve subjects were enrolled: six in the computer EBUS-TBNA simulator training group and six in the wet lab training group. All subjects successfully completed the study. Four subjects were excluded from the study because they had previously received more than 1 h of EBUS-TBNA training on a computer simulator or via wet lab simulation, and three subjects were excluded because they had performed more than 20 clinical cases. The demographic data of the study subjects are summarized in Table 1. The main results are presented in Table 2. The computer simulator group performed significantly better than the wet lab group in the percentage of lymph nodes correctly identified (70.4±16.7% versus 42.9±19.9%; P=0.002) and in EBUS-TBNA efficiency score (3.00±0.87 versus 1.73±0.96; P=0.037). There were no significant differences between groups for total procedure time and percentage of successful biopsies.
Near the end of subject testing, one of the computer EBUS-TBNA simulators experienced a minor malfunction. While testing two subjects (one from each study group) during the biopsy of the second of two lymph nodes in the test case, the computer did not recognize the learner needle input (needle malfunction). Learners were instructed to proceed through each step of needle usage (to ensure similar procedure times) and were given credit for three successful passes.
Poststudy questionnaire -day 1 All 12 subjects completed a questionnaire at the end of day 1. There were no significant differences between groups regarding overall satisfaction with either the wet lab or the computer simulation (computer simulator group: five of six very satisfied, one of six satisfied; wet lab training group: four of six very satisfied and two of six satisfied). Subjects in the wet lab training group appeared more likely to be very confident or confident (four of six) at being able to complete a simple EBUS-TBNA procedure than those in the computer simulator group (two of six), but this was not statistically significant. Subjects in the computer simulator group were more likely to feel very confident or confident (six of six) at being able to perform a systematic ultrasound examination of the lymph nodes than those in the wet lab training group (two of six), but this did not reach statistical significance. All subjects in both groups (six of six in both groups) were very confident or confident that they could successfully perform needle aspiration of an enlarged (>1.5 cm) mediastinal lymph node; however, only two of six subjects from each group stated that they would be confident performing a staging EBUS-TBNA procedure with needle aspiration of small (0.5 cm to 1.0 cm) mediastinal lymph nodes.
Subjects in the wet lab training group were more likely to feel very confident or confident at being able to operate the EBUS-TBNA bronchoscope and needle system than those in the computer simulator group (six of six versus four of six [P<0.05]).

Postcourse questionnaire -day 2
There was a total of 19 course participants. The final course questionnaire was completed by 18 (95%) of the course participants. All study subjects completed the final course questionnaire.
When asked which training method they preferred if only one was available, four (22%) course participants strongly preferred the wet lab, seven (39%) preferred the wet lab, five (28%) preferred the computer simulator and two (11%) strongly preferred the computer simulator. Interestingly, five of six (83%) subjects in the wet lab training study group either preferred or strongly preferred the wet lab, versus two of six (33%) subjects in the computer simulator group. While 13 (72%) of the participants strongly preferred or preferred the computer simulator when considering ease of learning, 18 (100%) participants strongly agreed (72%) or agreed (28%) that the use of computer EBUS-TBNA simulation and wet lab models offer complementary learning opportunities, with the majority of participants further emphasizing this viewpoint in the comments section of the questionnaire.

DiSCuSSion
The present study explored learner performance and preferences following hands-on EBUS-TBNA education with animal-based wet lab models versus computer simulators. From the performance perspective, two of the three primary outcome measures of the study (EBUS-TBNA procedure time and percentage of successful biopsies) were similar in both groups and correlated with an intermediate level of EBUS-TBNA skill as illustrated in a previous validation study (7). This suggests that training using the actual EBUS-TBNA equipment on a live swine model and training with a computer simulator are both useful methods of teaching basic EBUS-TBNA technique.
Previous studies evaluating the use of simulation education for both basic bronchoscopy (17) as well as EBUS-TBNA (10,12) demonstrated that the use of computer simulation leads to accelerated acquistion of technical skills without the need for practicing on patients. These same studies have shown that the skills learned on the computer simulators are transferable to real clinical bronchoscopy skills. In our study, learners in the computer simulation group performed better at systematic lymph node examination, with higher success rates for identifying mediastinal lymph nodes. This is not surprising given that  the airway and vascular structures present in the swine model are different than the human anatomy but are fairly accurately recreated by the computer simulators. Education programs relying heavily on wet lab models may need to explore additional methods of ensuring learners' understanding of mediastinal anatomy, particularly the relationships between airway landmarks, lymph node stations and vascular structures. Although all subjects in both groups appeared to be satisfied with their EBUS-TBNA learning experience, subjects in the wet lab group were more confident in their skills at being able to complete a simple clinical EBUS-TBNA procedure than those in the computer EBUS-TBNA simulator group. This could be explained by the wet lab group obtaining increased confidence because they were using the same EBUS-TBNA equipment that they would be using on patients and sampling real lymph nodes surrounded by real vascular structures. This explanation is further supported by the fact that subjects in the wet lab group demonstrated higher confidence at being able to operate the EBUS-TBNA bronchoscope and needle system than those in the computer simulator group. The wet lab model, by more closely reproducing a clinical setting, therefore, appears to improve confidence level over computer-trained individuals.
All subjects in both groups were confident or very confident that they could successfully perform needle aspiration of a large (>1.5 cm short axis) mediastinal lymph node in a clinical setting. This supports the utility of dedicated EBUS-TBNA CME courses to train experienced bronchoscopists to perform basic EBUS-TBNA procedures. The majority of subjects in both groups stated a lack of confidence in their ability to perform complete lung cancer staging procedures with aspiration of small (0.5 cm to 1.0 cm) mediastinal lymph nodes. Based on previously published data on EBUS-TBNA technical skill acquisition, this is not surprising because we would not expect learners to attain such technical skill in a two-day course (7)(8)(9)(10)12).
By the end of the two-day EBUS-TBNA CME course, all of the participants had equal opportunity to learn EBUS-TBNA using both wet lab and computer simulation. The postcourse questionnaire results revealed that there was no clearly preferred learning method, with some learners preferring the wet lab experience and some preferring the computer simulator. The course participants, however, unanimously and strongly responded (question 10 Appendix C) that both learning methods were complementary and should be offered together in future courses. The wet lab model appears to provide a degree of realism that is unmatched in other nonapprenticeship learning methods. Further studies are required to investigate the use of inanimate models of EBUS-TBNA simulation to determine how these models compare with the wet lab learning model.
The computer simulation group did have more opportunity to become familiar with the computer simulator; therefore, it is conceivable that using the EBUS-TBNA simulator to both train and assess EBUS-TBNA skill is a potential confounder. Three previous studies, however, support this approach. A validation study demonstrated that clinical EBUS-TBNA skills can be accurately measured using a computer EBUS-TBNA simulator (7). EBUS-TBNA simulation has been demonstrated to improve the rate of procedural skill acquisition in trainees when compared with clinical EBUS-TBNA experience (10). Yet another study demonstrated that skills learned on the simulator are, in fact, transferable to clinical EBUS-TBNA procedures (12). In addition, testing of course subjects during clinical procedures would have been a logistical impossibility and no other EBUS models have been validated or shown to correlate with clinical skills to date. As such, we believe that using a validated computer simulator-based assessment of EBUS-TBNA skill is an adequate surrogate to clinical performance.
A limitation to our study was the small sample size of each group. Confirmation of our findings with larger trials would be ideal. The similarities between groups suggest that the randomization process was effective; however, it is possible that undetected differences between the groups exist and that factors impacting skill acquisition and performance may not have been equally distributed between the two groups.
Our study compared two methods of teaching the technical skills of EBUS-TBNA, with no attempts to evaluate the cognitive aspects of EBUS-TBNA such as patient selection, result interpretation, etc. Our goal was not to assess competency for independent practice. In practice, a comprehensive program of simulation, as well as didactic and clinical training, may offer an optimal approach to achieving competency in EBUS-TBNA. Methods to evaluate competency in EBUS-TBNA that include the cognitive aspects of EBUS-TBNA as well as the technical skills will need to be developed and validated.
The present study identified some limitations associated with each learning method; however, some additional limitations should be noted. Wet lab simulation is associated with significant cost, ethical issues surrounding the use of animals for education as well as some inconvenience due to time and planning, which must occur to ensure that animals are used in an ethical and humane manner, and that appropriately trained animal caregivers are available. Computer EBUS-TBNA simulators are also very expensive (with a current cost of >US$100,000 per simulator), do not 100% effectively mimic the real EBUS-TBNA equipment (7,(10)(11)(12) and are prone to equipment malfunction. Given the limitations of current simulation methods, patient care bronchoscopy learning experiences remain an important and necessary part of training. Armed with the knowledge of the strengths and limitations of each method, the use of simulation in bronchoscopy education has the potential to signficantly reduce the number of patient care experiences required to achieve technical competency and reduce the burden of procedural learning on patients.

SuMMARy
The present study demonstrated that while both computer EBUS-TBNA simulation and wet lab simulation are effective methods of learning basic EBUS-TBNA skills, there are advantages and disadvantages to both learning methods that make them complementary learning modalities. The use of a computer EBUS-TBNA simulator is better than wet lab simulation using a live swine model at teaching human anatomy and the skill of correctly identifying human lymph node stations during an EBUS-TBNA procedure, while the wet lab model was associated with improved confidence in the performance of clinical cases. Lack of realism with the current computer EBUS-TBNA simulators and the lack of human anatomical correlations present in the wet lab model are major limitations of each method. We anticipate future improvements in computer simulation technology and as well as for inanimate models; however, at the present time, a combination of learning methods may be required for an optimal learning experience during EBUS-TBNA CME courses.
• Draw a diagram illustrating all 5 lymph node stations that you will examine, and a detailed relationship between the lymph node stations and all of the surrounding airways and landmark blood vessels (must repeat this drawing until they correctly identify all 5 stations including the correct relationship with the airways and the azygous vein, superior vena cava, inominate artery, aorta and left main pulmonary artery).