Evaluation of a Multimodal VR training platform for maxillofacial surgery

One of the most striking features of virtual reality systems is their ability to enrich training experience by allowing the developments of sophisticated feedbacks. This study focuses on the way to use modality management in virtual reality systems to accelerate training in a surgical task involving bone drilling. The study compares the efficiency of two punctual drilling training protocol implemented on the MFS SKILLS platform. The experiments were conducted with residents from 2 university hospitals. The results show that augmented simulation improves acquisition of specific aspects of the surgery in comparison with pure simulation.


Introduction
Contemporary developments in Virtual Reality (VR) technologies have emerged as a new tool to assist apprenticeship-based residency program.These systems provide an opportunity to reconsider didactic surgery with new eyes as they can provide routines for automation, precise monitoring and immediate feedback.They introduce new features previously difficult to achieve [1] [2].Compared with standard simulation training such as working with cadavers or mock-ups, VR allows for enriched representation of surgical procedures and brings anatomic accuracy and diversity [3] [4].Despite significant advances over the last decade, the adaptation of medical curricula to include alternative means of training is still at a very early stage, which is especially true for VR based training [5].The goal of this study was to evaluate the benefits of these innovative systems to a selection of tasks in bone surgery by evaluating the effect of multimodality management in the course of a training protocol.
The present research was conducted on the SKILLS maxillofacial surgery (MFS) demonstrator [6].This platform focused on Epker osteotomy.This surgery was chosen because of the important role played by haptic and tactile feedbacks in a situation where the operative field offers a very limited view of the surgery.One of the most important tasks is to learn how to drill properly the lower maxilla before splitting the bone.This drilling procedure must be perfectly controlled to avoid impairing the alveolar nerve that goes inside the maxilla, and therefore requires minimizing the penetration depth of the burr inside the spongy part of the bone.
In VR environment, the modalities brought to the student during practice (like auditory, tactile or haptic) can be fine tuned, selected and modified.Recent research in procedural learning and perception in VR suggests that playing with sensory cues might facilitate the acquisition of basic skills in surgeries involving strong haptic feedback [7].Indeed, a recent study, conducted in the scope of this research [8] to evaluate stiffness discrimination learning using visual and haptic modalities, challenges the notion that long-term unisensory learning mechanisms operate optimally under multisensory training conditions.Information feedback during training was shown to positively affect decision time, but has not been shown to be effective in improving the accuracy of discrimination.The tendency to improve timing of performance and decision was consistent with findings in a complementary vibro-auditory discrimination experiment [9].This study focuses on the way to use VR system to accelerate learning of drilling abilities.In order to study the effect of a multimodal approach, two different training protocols were implemented and compared.One is a direct simulation of the surgery, while the other features a specific uni-to multisensory approach in the course of training.

Setup and experimental protocol
The MFS demonstrator is a multimodal training platform featuring high-resolution force and tactile feedback, 3D fish tank rendering, sound and vibrotactile synthesizers.Detailed description of the system may be found in [6,10,11,12].Modalities were implemented in the simulation using results from multimodal capturing campaigns performed in an anatomic laboratory by expert surgeons on cadaver heads.During these sessions, position and orientation of the tool were recorded as well as the force and torque applied on the bone by the drill.Sound and vibration of the tool were also recorded.Data were used to define, develop and calibrate the modalities involved during the drilling [13].The focus was put on the critical step of the task, when the burr leaves the hard part (the cortical) and enters the soft part of the bone (the spongious), a task analysis studies suggested that the most critical step in Epker surgery is when the surgeon drills the first holes in the bone.In this task expert surgeons minimize both the penetration depth in the spongious and the drilling duration [12].These parameters were selected as the main performance criterion of the training protocols.
The exercises implemented in the training protocols consisted of either doing a punctual drilling or drilling along a line.The protocols can be followed by both junior and senior residents as no specific anatomical reference is necessary.Two different approaches were considered to design the exercises.In protocol 1 simulated and trainees practice on a physiologically realistic jaw as if they were in the operating room.In protocol 2 the simulation only provide key element at each step of the training protocol allowing the trainee to concentrate on specific sub-skills.The trainee progresses from drilling on simple to more complex and realistic shapes.Furthermore, the order of the experience with specific sensory aspect was predefined in the first 3 exercises: (i) Haptic only, (ii) Haptic + Sound + Vibration, and finally (iii) All modalities.
The two protocols were segmented into 9 exercises performed in 3 training sessions of around 1 hour each spanning over 20 days.During the course of the training, each trainee drilled at least 300 holes and 20 lines.
During the experiment the platform was moved to two university hospitals to test the relative efficiency of the two protocols.In order to monitor performances before and after the training, a pre-test and a post-test, consisting of 5 punctual drilling in simulation were added and a confidence evaluation was done after pre and post tests.Novice surgeons seem indeed to lack self confidence.As a consequen drill not to impair the nerve.Self confidence should therefore be a good indicator of the progression of be estimated on 10 point Lickert Scale.For every exercise, the trainee was instructed that he must limit as much as possible the penetration of the drill into the spongy bone.He was also informed that he has to stop the progression of the drill as soon as he feels a change in sensation and that he should proceed efficiently by minimizing the duration of each drill.Performance feedback was delivered at the end of each exercise presenting the average duration of the drills, the forces applied on the bone and the trajectory of the drill where the penetration depth can be clearly seen.Expert performances levels were indicated for comparison, to promote motivation.An overall of 20 residents followed the protocol, 10 in each group.In addition, performance of 4 experts in the field of maxillofacial surgery were recorded and compar performances.

Results
Expert surgeons achieved during the pretest the highest score without any prior training on the platform.Their performances were comparable with what was observed in anatomic laboratory in the task analysis campaign.There penetration depth is typically lower than 3mm and each drilling task lasts around 3s.
A comparison of performances between pre and posttest was carried out with repeated-measures ANOVA analysis for the two groups of residents following the two protocols.During the experiment, cortical depths were randomized over repetitions.For each pre and post test, we calculate global mean penetration on 5 drills.For both protocols penetration depth was lower at post test (m=4.12mm,SD=2.19) than at pre test (m= 5.74mm, SD=3.44) indicating that training on the platform enhances virtual drilling performance in terms of minimization of penetration depth (see Figure 1).At post test, penetration depth is shorter for the skills group (m=3.54mm,SD=1.74) than for the simulation group (m=4.76mm,SD=2.56), but this result is not statistically significant.

00011-p.2
The impact of training on post test performance is also dependant on the expertise level of the trainee.For novice participants, the impact of training is about the same for both training.Penetration depth at pre test for the Skills group is m=5.501mm(SD=1.23)and m=3.957mm (SD=0.874)at post-Test.For the Simulation group, mean penetration depth at pre Test m=8.45mm (SD=1.236)and 5.45mm (SD=0.87)at post test.For more advanced participants, penetration depth at pre test for the Skills group is m=3.70mm(SD=1.51)and m= 2.93 (SD=1.070)at post-Test.For the Simulation group, mean penetration depth at pre Test m=3.56mm (SD=1.74)and 3.28mm (SD=1.23)at post test.Also for junior residents self confidence improves from 3.83 to 5.83 for Skills protocol and from 4.33 to 7.33 for the simulation group.For advanced participants, self confidence keeps constant between pre-test and post-test.Significant differences between the two protocols were found with regard to the duration of drilling.Mean drilling duration was shortened with training for both groups (see Figure 2).Mean duration of simulation group was reduced from m=4.8s (SD=1.95) at pre test to m=2.8s (SD=0.82)at Post Test.The training protocol also had a significant effect on duration although the participants of the skills protocol presented at pre test, higher drilling times (m=5.609s,SD= 2.106) as compared to the Simulation group (m=4.03s,SD=1.45), this difference disappeared at post test: m_skills=2.71s(SD=0.42)and m_simu=2.89(SD=1.14).The interaction between the type of training protocol and expertise was significant (F*= 5.95; p=0.01)Mean penetration of the drill into the spongy bone is one of the main relevant metric of the task for the Epker surgery.MFS surgeons are requested to limit the plunge depth of the drill into the spongy bone to preserve the integrity of the alveolar nerve, while insuring that the drill is complete to provide a mastered fracture of the bone.The results show that plunge depth was improved with the training on the virtual platform.After 300 drills, the participants managed to reach appropriate plunge depth and drill duration relatively to expert performance.The results fail to demonstrate significant differences between the training protocols for penetration depth.The absence of significant difference between the participants from the Skills group and the simulation group may reflect the absence of impact of nature of the training (either Skills or simulation) on final plunge depth.The impact of the type of training on final plunge depth may be masked after 300 repetitions performed over 3 weeks.The analysis should focus on the early training period to study more thorough effects of the impact of training on plunge depth performance.This lack of effect may also be explained by the higher initial performance level of the Skills group.Participants were randomly assigned to specific protocol.The results showed that within these criteria, participants of the skills group were more inclined to proceed to drills with minimum penetration into the spongy bone.We can hypothesize that performance enhancement is more difficult when baseline performance is already reaching high levels, thus masking the impact of training on mean plunge depth.
Duration of a surgical task is regularly presented as significant variable of surgical expertise.Expert surgeons present shorter durations than less expert surgeons due to optimization of movements.Expert gestures are highly controlled and focused to the The International Conference SKILLS 2011 00011-p.3purpose of the surgery, whereas non expert surgeons tend to change their control strategies during the task, thus augmenting the number and the course of movements and as a consequence augments duration task.The results of the present experiment are in agreement with this general notion.Training on the platform reduces mean drilling time.

Conclusion
This experiment shows that repeated practice on the platform improves both the plunge depth of the drill and drilling time.The results fail to reveal strong differences within the training protocols for penetration depth of the tool.However a significant difference is seen between the two groups regarding the efficiency of the drill, measured by the drilling duration, and selfconfidence was significantly improved for junior residents in the Skills group.Many data gathered during the experiment still have to be processed, analysed and interpreted.We hypothesize that skills training protocol would favour fine force control as the training focuses on compliance during the early training phase and should favour compliance control of the drill over the other training protocol (i.e.simulation group).This variable is under process and will be included in future publications.

Figure 1 :
Figure 1 : Mean plunge depth at Pre Test and Post test for Skills participants and simulation participants.A reference expert value is introduced in the figure.

Figure 2 :
Figure 2 : Mean drilling duration (in s.) at Pre Test and Post test for Skills participants and simulation participants.A reference expert value is introduced in the figure