Virtual Reality, Haptic Simulators, and Virtual Environments

Fig. 16.1
Participants using virtual reality haptic simulators in a conference setting

These devices in general process a physical interface (e.g., laparoscopic instrument, laparoscopic camera, bronchoscope, or colonoscopes) that replicates the actual devices used in clinical practice, with respect to feel, heft, and controllers. The operator interacts with the virtual, computer-generated clinical environment (abdomen, tracheobronchial tree, colon) by engaging the physical device into a “mannequin”. Moving the device (advancing, withdrawing, turning left and right) produces accurate movement in the virtual environment. The positioning of the physical device in the virtual environment produces images on the display, optional force feedback (i.e., haptics, see below) and tissue deformity (movement, bleeding, or procedural success). Many of these devices have additional ports where other instruments can be deployed into the ­virtual environment to replicate interventions (cutting, needling, suturing, irrigation, or lavage). Several even possess performance matrices (procedure time, motion economy, structures visualized) and standardized feedback can be generated and followed over time and used to compare individual performance or to compare individuals to one another. Figure 16.2 demonstrates a typical surgical virtual reality simulator.


Fig. 16.2
LAP Mentor Express is a non-haptic laparoscopic training system provided at a lower cost. Available in either portable or optional tower including touch screen settings

Haptic feedback is defined as the combination of sensory input through the tactile receptors in the skin and the kinesthetic receptors in the muscles, tendons, and joints [7, 8]. Haptic surgical simulators have been created, most notably in laparoscopic surgery, but can now be found in non-laparoscopic surgical subspecialties such as ear, nose, and throat, orthopedic, and neurological surgery. Surgical simulators in each area are reviewed below as well as pertinent studies evaluating the effectiveness of haptic feedback in enhancing surgical training.

Laparoscopic Surgical Simulators

General Surgery

The need for simulation in general surgery arose mostly for the introduction and popularization of laparoscopic surgery in the 1980s. Despite the small incisions and proposed “minimal access,” it became quite clear it carried a higher complication rate and a steep learning curve for surgeons trying to adapt to this new technology [9]. Upon further investigations, it was found these deficiencies stemmed from psychomotor challenges related to the new cumbersome interaction with the videoscopic interface. Thus, a number of laparoscopic simulators were developed including but not limited to GI Mentor, ProMIS, LapSim Simulator, MIST-VR, and XiTact SA.

Within the last decade, the American College of Surgeons Residency Review Committee has mandated that all programs institute a skills laboratory curriculum with laparoscopic box trainers and virtual reality simulators as minimally acceptable equipment [8]. Haptic feedback distinguishes the box trainer from many of the virtual reality-based systems. However, more commonly, efforts are being made to equip the virtual reality simulators with adequate haptic feedback as exemplified by Procedicus, MIST, Lapsim, Reachin, and Virtual Endoscopic Surgery Trainer to create a more realistic feel to the virtual reality simulator.

The development and validation of a comprehensive fundamentals of laparoscopic surgery (FLS) was described by Peters et al. [10]. In this chapter, the importance of testing technical skills in laparoscopy was emphasized as a distinctly separate skill set that must be obtained by the practicing physician, and learning these skills can be accomplished outside of the operating theater [10]. The five tasks designed for evaluation are as follows: peg transfer, pattern cutting, ligating loop, suturing with an intracorporeal knot, and suturing with an extracorporeal knot. Figure 16.3a–g demonstrates the Simbionix Lap Mentor including the device (Fig. 16.3a), the instruments (Fig. 16.3b) and the screenshot portraying classic laparoscopic skills like peg transfers (Fig. 16.3c), and suturing (Fig. 16.3e). Completion is required for all trainees as of 2010 in order to sit for the American Board of Surgery exams.


Fig. 16.3
(ag) Device, instruments, and screenshots of the Lap Mentor (Photo courtesy of Simbionix)

Box trainers are used by several programs to enhance ­laparoscopic skills. Typically, these simulators use actual laparoscopic equipment including laparoscope, camera, light source, trocars, and authentic laparoscopic instruments. The instruments are placed into the training area via an opaque covering. The simulators usually require an experienced mentor and the trainee is evaluated on time to completion and number of errors. Studies show both improvement in laparoscopic skills and improved performance in the operative theater [11]. Although these simulators allow for haptic feedback while using authentic laparoscopic tools, the simulators lack the ability to simulate the actual surgery and are limited to fundamental skills.

Virtual reality simulators have risen in popularity secondary to success demonstrated in the aviation community and the demand for higher fidelity simulators capable of more than just simple task-oriented simulation. These simulators entail a virtual environment on a computer screen with three-dimensional graphics taken from actual MRI or CT scans. Instruments are created virtually that track the trainee’s movement and updated in real time. Haptic feedback is included in the higher end simulators but with an increased cost and represents varying levels of realism.

Seymour et al. [9] validated the use of virtual reality in improving operating room performance. Sixteen surgical residents at various levels of training were randomized into virtual reality trained group or a non-virtual reality trained group. The virtual reality group was trained on a MIST-VR laparoscopic trainer without haptic feedback. The results of the study showed that residents without virtual reality laparoscopic simulator training were nine times more likely to transiently fail in making progress, five times more likely to injure the gallbladder or burn nontarget tissue, and overall were six times more likely to commit errors.

The importance of haptic feedback in these simulators has been a subject of interest for some time. Panait et al. [8] examined the role of haptic feedback in laparoscopic simulation training by evaluating performance of medical students on two FLS tasks, peg transfer, and pattern cutting, on laparoscopic simulators with and without haptic feedback. In the simpler peg transfer, the study shows haptic feedback has no effect on performance; however, faster completion and a trend toward fewer errors were seen in the more complex pattern cutting task. The study concluded haptic feedback allows superior precision in more advanced surgical tasks.

In order to truly validate the importance of haptic feedback in laparoscopic simulators, it must be shown to enhance training when simulating actual surgical tasks. One study that examined this notion compared traditional box simulators with haptic feedback versus a VR laparoscopic simulator without haptic feedback. The study found participants significantly prefer haptic feedback when simulating an actual complex laparoscopic task [7]. Again, this study reiterates the finding that haptic feedback is preferred when simulating complex surgical tasks.

Alternatively, haptic feedback does not universally prove to be advantageous in the literature. Salkini et al. [12] investigate the role of haptic feedback on the Simbionix LapMentor II while performing three laparoscopic tasks in novice medical students. Similar to previous studies, the students were split into a haptic and a non-haptic group and asked to perform these tasks. No differences were noted between in speed, accuracy, or economy of movement between the two groups [12]. However, it should be noted that one of the tasks was clipping a tubular structure which was initially eliminated from FLS secondary to having no discriminatory value [10]. Overall, general surgery has been a pioneer in the creation and integration of haptic surgical simulation into the curriculum of surgical trainees.

Thoracic Surgery

Haptic simulators have also found a place in the training of thoracic surgeons. Solomon et al. [13] have reported the development of a haptic enable virtual reality cognitive simulator for video-assisted thoracoscopic lobectomy (VATS). The simulator was created through adaptation of anatomy explorer models and a previously developed simulation engine. Cognitively, this simulator allows for 13 anatomic identifications and 20 high yield lung cancer learning points. The procedural components to the simulator allow for placement of trocars, manipulation of the thoracoscope for exploration of the pertinent anatomy, stapling, and haptic-enabled gross dissection of the lung tissue to expose underlying hilar structures [13].

Unfortunately, the simulator does not allow haptic feedback for fine motor tasks such as dissecting around the hilar vessels, and validation for this interface is ongoing. Additionally, the significance of the gross haptic feedback currently enabled in the current system has not been evaluated in regard to impact subsequent improvement in the operative theater. Nonetheless, this project represents the progression of haptic simulators to simulate actual procedures as opposed to fundamental training tasks in yet another subspecialty of surgery.

Gynecological Surgery

The use of laparoscopic simulators in gynecological surgery was recently detailed [11]. Training systems using virtual reality for laparoscopic bilateral tubal ligation, hysteroscopic resection of myoma, endometrial ablation, and removal of a tubal ectopic pregnancy have all been described. However, these are at various stages of development and validation. The use of haptic feedback in these simulators is currently unclear. Figure 16.4a–d shows gynecological surgery simulators in use, including the DaVinci robot simulation (Fig. 16.4d).


Fig. 16.4
(ad) Gynecological surgery VR simulation (VirtaMed HystSim) (Photo courtesy of Simbionix)

Non-laparoscopic Surgical Simulators


Haptic simulators have also been created for the field of otolaryngology in order to train residents in endonasal techniques as well as temporal bone dissection. Tolsdorff et al. [14] reported a new virtual reality paranasal sinus surgery simulator. The simulator is a standard PC-based program with haptic feedback that presents a highly realistic environment to learn surgical anatomy and navigation in a complex anatomical space. Further studies are needed on this simulator to validate its usefulness in improving operative performance in residents [14]. Another endonasal simulator, the Endoscopic Sinus Surgical Simulator, has been studied and shown to have a positive effect on early operative performance of trainees [15].

Temporal bone dissection is another area of otolaryngology that has a haptic enable VR simulator currently studied for its use in self-directed learning. The CSIRO/University of Melbourne system is run on a standard desktop PC and utilizes a 3D model that the user interacts with through shutter glasses and haptic devices. This particular system was studied as a part of a randomized blinded control trial evaluating the role of this simulator in self-directed learning [16]. Two groups of medical students were randomly divided. One group received training via temporal bone models and video, whereas the other group underwent training on the VR simulator using self-directed curriculum. Afterward, both groups were given 1 h to complete a temporal bone dissection which was subsequently graded. Results indicated that the VR group showed better performance on all four outcome measures recorded in the study. Studies showing improved operative performance have yet to be performed with this model.


Simulators have been introduced into orthopedic surgical training in the form of knee and shoulder arthroscopic simulators. A systematic review of knee and shoulder arthroscopy was performed evaluating five studies that investigated knee arthroscopy and four studies for shoulder arthroscopy [17]. These studies were all found to have high levels of internal validity and consistency for computer simulation. In the nine studies evaluated, one study investigated the transfer of knee arthroscopic skills from the simulator to the operative theater [18]. Twenty junior orthopedic trainees were randomized to two groups with one group trained on the simulator with expert supervision while the other group observed diagnostic arthroscopy in the operative theater. Afterward, the participants were graded in the operative theater based on global assessment scale as well at the Orthopedic Competence Assessment Project score. Results show the simulator group outperformed the non-simulator trained group on both, concluding that these skills do in fact translate to performance in the operative theater. The role of haptic feedback was not evaluated in these studies. Figure 16.5a–c shows an orthopedic VR simulator, including the device, the hand controls, and a screenshot.


Fig. 16.5
(ac) Arthro Mentor (Photo courtesy of Simbionix)

Neurological Surgery

VR simulators in neurological surgery have been created for the areas of cranial skull base surgery, neuro-oncology, ventriculostomy placement, and placement of spinal hardware. In skull base surgery, Dextroscope was created initially as a tool to create patient-specific volumetric models that can be manipulated in 3D space with the use of handheld controllers. The Dextroscope was originally designed to assist in operative planning and to evaluate 3D anatomy prior to operative intervention [1921]. Wong et al. [22] created a procedural application for this model that allowed clipping of intracranial aneurysms [22]. To our knowledge, there are no current studies that have looked at enhancement of trainee performance on this simulator. Another simulator that is increasing in popularity in neurological surgery is the ImmersiveTouch. This system improves on the concept of Dextroscope by the addition of electromagnetic head-tracking goggles that allows a user-centered stereoscopic perspective and the addition of haptic feedback [23]. Using this simulator, trainees can perform a virtual ventriculostomy. In addition to this module, ImmersiveTouch also has a spine module that recreates the placement of pedicle screws for spinal fixation. Studies are currently ongoing regarding the transfer of these skills to the operative theater [21].

More recently, a haptic-enabled VR simulator has been created called NeuroTouch. The main components are a stereo vision system and bimanual haptic tool manipulators. The simulation software engine runs three processes for computing graphics, haptic feedback, and mechanics [24]. The goals of this training system are to safely remove the tumor without damaging normal tissue with an ultrasonic aspirator and to maintain hemostasis using bipolar cautery. The prototypes are currently being dispersed throughout training programs in Canada for beta testing and validation.

Procedural-based simulation is advancing rapidly and is on a collision course to intersect with other forms of simulation and visualization enabled by the rapid advancement of technology.

Nonsurgical Procedural Simulators

Endovascular Simulators

Endovascular interventions are another area of medicine where simulation has been utilized. Much of this discipline is becoming more minimally invasive with the creation of endovascular technologies. As we have seen in laparoscopic surgery, these minimally invasive techniques can possess a steep learning curve due to decreased tactile feedback and working in a 2D environment. Currently, several high-fidelity endovascular simulators are commercially available including Procedicus VIST, Angio Mentor, and Sim Suite. These simulators offer haptic, visual, and aural interfaces in order to create a lifelike environment to perform the procedures. In particular, coronary artery stenting and peripheral angioplasty are simulated and appear to be the most studied.

Three studies performed using Procedicus VIST simulator show simulator use improves procedure time, contrast volume used, and fluoroscopy time [2527]. Furthermore, the simulators were shown to have good construct validity given the ability to consistently distinguish novice from expert as well as show significant improvement with practice in novice users. In peripheral artery angioplasty, similar results were found in similar studies [28, 29]. However, Chaer et al. went one step further and performed the first randomized trial to show translation of endovascular simulation skills to the clinical environment [30]. The study concluded residents trained on the simulator scored higher on a global rating scale for performance than those without training. Thus, it is likely simulation training in vascular surgery will become increasingly integrated into resident training. Figure 16.6 shows VR endovascular simulators in use.


Fig. 16.6
VR endovascular simulation (Photo courtesy of CAE Healthcare © 2012 CAE Healthcare)


Bronchoscopy is performed by anesthesiologist, cardiothoracic and trauma surgeons, critical care physicians, and otolaryngologists. Training on these procedures varies across disciplines and institutions. Proficiency is currently based on a minimum number of procedures to perform at many facilities. Several virtual reality high-fidelity simulators, including Accutouch bronchoscopy simulator (CAE) (Fig. 16.7) and GI Bronch Mentor (Simbionix) (Fig. 16.7a, b), are currently available commercially. Several studies have documented improvement of skills with these simulators; for example, participants trained on the simulator are less likely to experience wall contact, red out, and complete the procedure more efficiently [31].


Fig. 16.7
(a, b) Accutouch endoscopy simulator (Photo courtesy of CAE Healthcare © 2012 CAE Healthcare)


Gastroenterologist also utilizes virtual reality simulators in order to train for endoscopic procedures. The bronchoscopy simulators mentioned above also possess endoscopy capabilities. There are two simulators currently on the market for use, Accutouch from CAE and GI Bronch Mentor. Both are virtual reality simulators equipped with haptic feedback (Figs. 16.7, 16.8a–c, and 16.9).


Fig. 16.8
(ac) Use of the GI Bronch Mentor (Photo courtesy of Simbionix)

multiple types of endoscopes are available depending on the procedure being simulated; both upper and lower GI endoscopy can be replicated. These simulators also involve upper and lower endoscopic procedures and are complete with haptic feedback. Comprehensive metrics are also included with these model [32]. Although studies have shown the GI Bronch Mentor to have good construct validity and improve skills when tested on a simulator, there are no studies that describe transfer of these skills to a clinical environment.


Fig. 16.9
A screenshot of the colonoscopy module from the Accutouch endoscopy simulator (Photo courtesy of CAE Healthcare © 2012 CAE Healthcare)

Virtual Platforms and Environments

Technological advancement is accelerating and so too are changes in healthcare. The purpose of healthcare education is to train workers who can competently care for their patients today while building a foundation for lifelong learning.

The need for lifelong learning in healthcare’s rapidly changing environment prompted the American Board of Medical Specialties (ABMS) and its 24 member boards in 2000, to promote Maintenance of Certification (MOC) to “evolve recertification programs towards continuous professional development” [33]. Continuous professional development requires objective proof that the workers’ practice is improving. These demands are forcing a shift in healthcare pedagogy toward interactive forms of learning formats capable of changing behavior.

Although the system targets competence for its workforce, many of us aspire to become experts in our field. According to Ericsson et al., the development of expertise requires 10,000 h of deliberate practice [34]. Deliberate practice is not time spent in patient care; instead, it requires time rehearsing, practicing, and reflecting on one’s actions—striving to improve with each and every repetition.

Deliberate practice is hard work, requiring motivation, endurance, and a grasp of domain-specific nuances of behavior. Unlike other domains where deliberate practice has been studied (e.g., music, chess playing), healthcare professionals cannot easily or ethically “practice” with live patients [35]. Through various forms of simulation, healthcare workers now have tools to aid in their pursuit of expertise.

Unfortunately, today’s physical simulators, because of their high cost, limitations on learner throughput, and need for highly trained facilitators, offer limited opportunities for deliberate practice—there simply are not enough simulators or facilitators to handle the sheer number of hours required.

What is needed is a robust, scalable, easily accessible, personal, interactive learning platform that enables learners to practice new skills, make mistakes, receive immediate feedback, and reflect on how they might improve—a scalable platform for deliberate practice. What is needed is the Ultimate Learning Platform.

The Ultimate Learning Platform

What might this Ultimate Learning Platform look like? In science we are often taught to visualize the end product first (e.g., a table or figure) and design our experiment backward from there. If we use the same approach for healthcare learning, the Ultimate Learning Platform would have the following features (Table 16.1).

Table 16.1
Features of the Ultimate Learning Platform: a comparison of mannequin-based learning (MBL), procedural simulation (procedural), single-player games-based learning (GBL single), and multiplayer games-based learning (GBL multi)

Features of the Ultimate Learning Platform



GBL (single)

GBL (multi)






Participate individually


Participate in teams


Peer instruction



Engages multiple senses





Sense of touch



Offers immediate feedback





Opportunity for reflection





Self-paced reflection





Individual remediation

Only gold members can continue reading. Log In or Register to continue

May 30, 2017 | Posted by in Uncategorized | Comments Off on Virtual Reality, Haptic Simulators, and Virtual Environments
Premium Wordpress Themes by UFO Themes