Error Sonification of a Complex Motor Task

Visual information is mainly used to master complex motor tasks. Thus, additional information providing augmented feedback should be displayed in other modalities than vision, e.g. hearing. The present work evaluated the potential of error sonification to enhance learning of a rowing-type motor task. In contrast to a control group receiving self-controlled terminal feedback, the experimental group could not significantly reduce spatial errors. Thus, motor learning was not enhanced by error sonification, although during the training the participant could benefit from it. It seems that the motor task was too slow, resulting in immediate corrections of the movement rather than in an internal representation of the general characteristics of the motor task. Therefore, further studies should elaborate the impact of error sonification when general characteristics of the motor tasks are already known.


Introduction
To efficiently learn a simple motor task, augmented feedback should rarely be provided to prevent reliance on it.In contrast, learning of a complex motor task is suggested to be facilitated by concurrently presented augmented feedback, at least in early learning stages [1].
As most complex motor tasks, especially in sports, are cognitively mastered in response to information that is visually perceived, additional provision of visual concurrent f capabilities to process all available information.Reallocation of information to modalities other than vision may overcome the performance loss coming along with limited visual perception and cognitive processing.This assumption is based on the multiple resource theory stating that distribution of information to different modalities is superior to a unimodal presentation [2].Justification is given, for instance, by the fact that visual-spatial information and auditory verbal information are maintained in different areas of the working memory, which extends its processing capacities when multimodal, e.g.visual and auditory information, is provided simultaneously [3].
Auditory information may not only reallocate perceptual and cognitive workload but also reduce distraction [4] since, unlike visual perception, auditory perception requires neither a head orientation to the display itself nor an attentional focus [5].
Alarm-type auditory feedback has already successfully been applied to facilitate learning of sportive tasks, e.g.[6][7][8].In addition, movement variables have also been sonified to continuously represent the current performance, e.g.[9][10][11].Thereby, concurrent sonification of the own movement is efficient to facilitate motor learning if a precise imagined movement is present and linked with sonified, optimal movement variables.Motor learning may be even more facilitated if the current deviation between own movement and optimal movement is acoustically presented, i.e. the movement error is sonified.Therefore, we have started to elaborate the impact of error sonification on learning of complex motor tasks.
As a first step, we have demonstrated that naïve subjects could almost immediately use a multidimensional error sonification to reduce the deviation to a target oar in a rowing-type movement [12].However, motor learning was not assessed.Thus, the goal of the present work was to assess learning of a rowing-type movement enhanced by concurrent, multidimensional error sonification.

Participants
The experimental group consisted of nine healthy participants (2 females, 7 males, aged 23 to 41 years).Further nine participants (3 females, 6 males, aged 24 to 30 years) served as a control group.All participants declared themselves to be non-rowers, to have normal hearing, and normal (or corrected to normal) vision.They signed an agreement that followed the guidelines of the local ethics commission which had approved the experiment.

Apparatus and task
The participant was seated in a real but trimmed rowing boat.A trimmed sweep rowing oar was moved with both hands and arms, but without moving the legs.Only the inertia of the oar was felt, water resistance was not simulated.To virtually display the oar blade on a large screen, oar motion was captured in real-time by an opto-electrical motion tracking system (Qualisys, Goteborg, Sweden).Kinematic data was also used to assess learning.
The rowing-type target movement included three oar angles (horizontal range of outer hand movement: m; two oar blade rotation of 90° from steep to flat and vice versa resulting in two valid oar blade orientations).Task complexity was further enhanced by a rowing-type angular velocity profile: During the drive phase, angular velocity was two times higher than during the recovery phase.One cycle lasted 6 s.

Procedure
Each participant was invited on four consecutive days (days 1-4) and one week after day 4, i.e. day 11.On day 1, participants familiarized themselves with the setup.The experimental group was introduced to the auditory feedback displayed via headphones: at first, to the error sonification applied to each oar angle.Secondly, the overall, three-dimensional error sonification on all oar angles was exemplified for 120 s: The experimental group had to match their movement to a circular movement with constant velocity which was not visually displayed.During familiarization with the setup, the control group matched a virtual target oar moving on a visually displayed circle with their own virtual oar.Both groups were then instructed to memorize the target movement which was visually displayed for 60 s (while they were not allowed to move their oar).In an immediately following baseline test lasting 180 s, they had to repeat the movement without seeing the target movement.After this baseline test, three trainings were conducted, each followed by 30 s of catch trials.The durations of the three consecutive trainings varied (150 s, 180 s, or 210 s) in order to avoid estimation when catch trials started.Days 2 and 3 started with a retention test, similar to the baseline test on day 1.On day 4 and 11, a 180 s retention test was performed.

Feedback
The experimental group received auditory feedback.Deviations to the horizontal oar angle were mapped to stereobalance, deviations to the vertical oar angle to pitch.A direction-indicative polarity was chosen, e.g. a sound from the right indicate to move to the right and a high pitch indicated to lift the oar blade.A wrong oar blade orientation was mapped to a raspy saw tooth wave.The total horizontal and vertical deviation was additionally mapped to volume (the better the quieter), for further details on the mapping see [12].
The control group could request terminal feedback anytime they had completed at least three cycles, but not during catch trials.The feedback, i.e. the own movement of the last 18 s, was visually displayed on the screen exactly at the position where the movement happened.The target trajectory was simultaneously displayed starting its movement closest to the the participant could watch the trajectories for maximally 10s.

Data analysis
Kinematic data was analyzed with Matlab (MathWorks, Natick, Massachusetts, USA).Data was low pass filtered by fitting a smoothing spline to it.The first cycle and last two cycles of the baseline and retention runs as well as cycles with missing data caused by motion capture errors were not considered in the further analysis.
Movements of the participants were compared to the reference movement.Data was cut into cycles at the minimal horizontal angle.To calculate spatial and temporal errors, the horizontal and vertical angles were used for a spatial and temporal analysis of the trajectory [13].In baseline and retention tests, comparison to the absolute position of the target trajectory.In the present work, it was rather focused on relative errors than on absolute errors, i.e. shift errors.Therefore, each trajectory was moved to the position where the spatial error was minimal before the relative spatial and temporal error was calculated.
To calculate timing errors of blade rotations, each completed cycle was taken as 100%.The difference the reference cycle were calculated in percent.
To asses learning, statistical analysis was done with the data of the baseline and retention tests by a Friedman test followed by a multiple comparison with post-hoc Bonferroni correction (p = .05).

Results
No significant group differences were found at the baseline measurement in terms of spatial error, temporal error, and timing of the blade rotation.
Eight out of nine participant corrected their movements based on the auditory feedback during the trainings, whereas one participant was not able to do so during all days.The experimental group showed jerky movements during the training phase (exemplified in Fig. 1) which were not present in the control group.
From baseline to each of the retention tests on day 2, 3, 4, and 11, spatial errors were not significantly reduced by the experimental group receiving auditory feedback but by the control group receiving selfcontrolled terminal feed (see also Fig. 2).In comparison to the baseline test, the control group could also significantly reduce the temporal error Both groups could not significantly reduce the temporal error.Only the control group was able to significantly improve the timing of the oar blade rotation.

Discussion and Outlook
The experimental group was able use the auditory feedback to reduce spatial and temporal errors during the training, in particular from day 2 onwards.However, they were not able to improve their performance in the retention tests, thus, learning of the rowing-type task was not facilitated by auditory feedback in contrast to self-controlled terminal feedback (Fig 2).It seems that participants used the feedback to immediately correct their current movement which was possible because the target movement was rather slow.However, these immediate corrections hamper the development of an internal representation of the whole target movement.The auditory feedback remained unfamiliar at least during the first half of all training sessions which suggests that auditory feedback was very demanding and required lots of attention.Thus, in retention tests, participants trusted on the memorized visual demonstration of the target movement rather than on their own trainings which was confirmed by a subsequently done questionnaire.
If the movement itself had been faster (a real rowing stroke is about three times faster), immediate accurate corrections might have been impossible but corrections on trajectory characteristics might have been enabled in subsequent movement cycles.In other words, error sonification may accelerate learning of specific details of a movement trajectory if the athlete is already familiar with its general shape and timing.Learning might also be accelerated if users will get more familiar with the auditory feedback, which has to be investigated in further studies.Further studies should also elaborate the efficiency to learn a complex motor task when multimodal concurrent feedback is provided.However, applicability of such multimodal feedback strategies, e.g.audiovisual or visualhaptic, might be limited to sport simulators in virtual environments.Thus, unimodal auditory feedback should further be developed to ensure that comprehensive feedback during real sports could also once be provided without overloading perceptual and cognitive resources.