Training of Perceptual Motor Skills in Multimodal Virtual Environments

Multimodal, immersive, virtual reality (VR) techniques open new perspectives for perceptualmotor skill trainers. They also introduce new risks and dangers. This paper describes the benefits and pitfalls of multimodal training and the cognitive building blocks of a multimodal, VR training simulators.


Introduction
With the growing complexity of systems and their operation environments, the required duration of training and the increased costs of errors, on the job training became difficult or impossible and alternative simulation platforms have been developed to enable training.Flying, driving, space operation, surgery, power plant and process control are salient examples for tasks for which training simulators were developed.Early simulators were physical and mechanical analogues of their represented systems.With the advance of computer technology simulators become more and more hybrid and system dynamic, visual field and audition have been increasingly driven and generated by computers.Contemporary developments in sensors and display capabilities as well as computation speed and storage capacity led the way to the development of multimodal virtual environments.In these environments, the operator is immersed, experience multimodal sensations and interacts with virtual objects including other humans [1].Vision and audition have been in the study and de-sign of simulators from their inception.The new and important addition is the inclusion of haptics: the ability to feel and exercise force, touch,and kinematics.Haptic technology is developing rapidly and haptic interfaces are now been incorporated in many virtual worlds.
At the outset of this discussion it is important to emphasize that topics of training and transfer of skills have not been in the focus of interest for the present research and development of virtual environments.When studying human behavior, the interest has been in the fundamentals of Virtualization: the process by which a human viewer interprets a patterned sensory impression to be an extended object in an environment other than that in which it physically exists Presence: the sense of being in a VE rather than the place in body is actually located.Immersion: A person is immersed in an environment that is realized through computer-controlled display systems [2].
For the training of skills, Presence and Immersion are related but not the prime focus concern.The value of a training system is judged by relevance, its ability to provide relevant experience; by the provision of facilitation and guidance to the acquisition of the designated skill; by the transfer from VR training to performance in the real world.Relevance, facilitation and transferability are therefore the key constructs and the crucial criteria for the value of a training system, which are very different from presence and immersion.

Training in Multimodal Environments
Multimodal displays support flexible efficient communication, they are easy to learn, can be used in challenging situations, and people enjoy using them [3].Multimodal interfaces succeed in creating a stronger sense of presence by better mimicking reality [4].The sensorial richness of multimodal environments translates into a more complete and coherent experience of the virtual world and therefore the sense of being present in the virtual realm is stronger.An important advantage of the VR technology is the ease of providing augmented sensory feedback, visual guidance, auditory directors or augmented haptic cues.

Developing training in multimodal environment
To illustrate the application of these principles and concerns in the development of training platforms, we briefly describe platforms developed within the European Community 6 th framework project "Skills", Transfer of Skills".The main building blocks of a training platform and a training program are: 1.A clear specification of the task to be learned, the skills to be acquired, the objectives of training, and the designated criteria of graduation.
2. Design of task scenarios, task versions and difficulty manipulation that best represent typical and key requirements of the task.[5,6].
3. Identification of key response and performance measures as well as progress criteria to evaluate trainee progress on relevant aspects of task performance and enhanced competence.
4. Definition of desired feedback indices and knowledge of results information to be given to trainees, as well as their frequency and mode of presentation.

Transfer of training.
The relevance of the training experience in the learning environment determines the level of transfer from training to actual task performance.
Topics 2-4 have been instantiated through the development of accelerators and training protocols.The term accelerator is used to refer to variables that are introduced and implemented to facilitate, assist and improve learning.The term training protocol is employed to describe training schedule, duration, selected tasks scenarios, difficulty manipulations and their order of presentation.Six training platforms have been developed and evaluated: Rowing (ROW), Juggling (JUG), Maxillo-Facial Surgery (MFS), Upper Limb Rehabilitation (ULR), Industrial Maintenance and Assembly (IMA), Programming by Demonstration (PBD) and.Table 1 presents a brief summary of the platform focus and their associated accelerators.In assessing the value of the developed platforms for skill acquisition and the best ways of applying them in training there are four basic evaluation aspects that need to be examined: 1) A comparative evaluation of the differential experience of performing the same tasks on the VR platform and in the real world; 2) Evaluation of the contribution of accelerators; 3) Assessment of training protocols that will maximize learning and skill acquisition on a platform; 4) Transfer of training studies.
Transfer of training studies with the training platforms, have all shown a variety of benefits to the real life task performance.Moreover, important principles have been developed for the combined use in skill training of haptics, vision and audition.In summary, it has been clearly shown that from a human performance skill acquisition perspective, the new multimodal VR technologies offer new and exciting potential for the development of simulators and the training of complex skills, as well as create risks and dangers to be avoided [7].

Table 1 -
Six Multimodal VR training platforms and designed accelerators