A Social Cognitive Neuroscience Stance on Human-Robot Interactions

Robotic devices, thanks to the controlled variations in their appearance and behaviors, provide useful tools to test hypotheses pertaining to social interactions. These agents were used to investigate one theoretical framework, resonance, which is defined, at the behavioral and neural levels, as an overlap between firstand thirdperson representations of mental states such as motor intentions or emotions. Behaviorally, we found a reduced, but significant, resonance towards a humanoid robot displaying biological motion, compared to a human. Using neuroimaging, we've reported that while perceptual processes in the human occipital and temporal lobe are more strongly engaged when perceiving a humanoid robot than a human action, activity in areas involved in motor resonance depends on attentional modulation for artificial agent more strongly than for human agents. Altogether, these studies using artificial agents offer valuable insights into the interaction of bottom-up and top-down processes in the perception of artificial agents.


Introduction
Artificial anthropomorphic agents such as humanoid robots are increasingly present in our societies.For these robots to interact optimally with humans, it is important to understand humans' automatic and unconscious reactions to these agents.Studies have addressed the issue of the form and functionalities a humanoid robot should have in order to be socially accepted.Both types of approaches have mostly relied on introspective judgments or implicit assumptions, such as the need for anthropomorphism, which may bias their conclusions [1].
I believe the issue of optimizing robotic designs should be investigated using robust experimental paradigms firmly grounded in a theoretical framework.I will present how such a strategy has been implemented using the "resonance" framework, which offers plausible explanations of implicit aspects of social interactions between humans in everyday life.In a first part, I will introduce the theoretical framework of social resonance.In the second part, I will review works on the effect of artificial agent's anthropomorphism on markers of resonance.

The concept of resonance
Theories of social behaviors using concepts of resonance have flourished in the scientific literature following the finding that the same neural structures respond both when executing an action and when observing another individual executing the same action.Already, in language, the motor theory of speech perception claimed that the object of speech perception are not sounds, but the phonetic gestures of the speaker [2].We will refer to these processes as resonance, which is defined, at the behavioral and neural levels, as the overlap of first-and third-person representations of hidden mental states.

Neurophysiology of resonance
Given the function of macaque's brain area F5 in the execution of goal-directed actions, the discovery of mirror neurons, "a particular subset of F5 neurons [which] discharge[s] when the monkey observes meaningful hand movements made by the experimenter" [3], offered the first physiological demonstration of resonance in the motor domain.Human neurophysiological data confirmed that premotor cortices are also active during observation of actions in the absence of any action execution.An interesting trend in human cognitive research is that this resonance generalizes to a number of other domains of cognition.For example, an fMRI study found an overlap of activity in the somatosensory cortex between being touched and observing someone being touched [4], observation of the expression of disgust activates a region of the insula also activated during the feeling of disgust [5]… Taken together, these findings led to the hypothesis that resonance between oneself and other selves is a phenomenon that underlies a number of social behaviors Motor resonance is evidenced by behaviors [6] like action contagion (contagion of yawning for example), motor priming (the facilitation of the execution of an action by seeing it done) and motor interference (the hindering effect of observing incompatible actions during execution of actions).The chameleon effect describes the unconscious reproduction of "postures, mannerisms, facial expressions and other behaviors of one's interacting partner" [7].This mimicry has been described as a source of empathy [8].
Functions classically attributed to motor resonance include action perception and understanding and imitation [9], important in social transfer of motor skills.More generally, resonance offers a parsimonious explanation of the sense of bonding with (similar) conspecifics.It should thus be taken into account in robotics, not only to design human-like social learning abilities in robots, but also to optimize robots design with regards to acceptability.

Mechanisms of resonance
The ontogenic origin of motor resonance is debated, and it has been argued that it could emerge as a result of self-observation.In a computational framework, the hypothesis is that the synchrony between motor commands sent to a visible effector like the hand and the visual feedback, like the experience of a baby during motor babbling with his hand, can induce Hebbian-like acquisition of sensory-motor associations forming a motor resonance network.The activation of such a network when actions from another person are perceived would result in automatic and involuntary production of the associated motor output.
We tested this proposal on a robotic experimental platform [10].We combined a 16 degrees of freedom robotic hand and a minimal visual retina using an associative network based on Hebbian learning.This visuomotor association network, trained by the selfobservation of hand postures, was able to reproduce gestures shown with another hand, but also to generalize to new hand postures the network hadn't experienced during training.We thus suggested that behaviors as complex as learning by imitation could be bootstrapped by self-observation, in robots too [10].
Even more interesting is the putative link between motor resonance and forward modeling in action control used to describe the properties of mirror systems [11].To put it is simply, the sensorimotor associations argued to support resonance would be byproducts of networks involved in motor control.When a given action is mastered, its sensory, included visual, consequences can be accurately anticipated (forward modeling), and processed without conscience.In this view, resonance is the activation of such an internal model not from its motor side like in action execution, but from its sensory side.Altogether, apprehending motor resonance as a byproduct of the maturation of motor control systems offers a parsimonious and coherent framework to root high-level social interactions into low-level motor control.Motor priming with a robotic hand Motor priming can be conceptually conceived as a form of "automatic imitation" consequential of motor resonance.Observing an action facilitates ("primes") the execution of the same action.This effect was investigated with two actions, hand opening and hand closing, in response to the observation of a hand opening and closing, with the hand being either a realistic human hand or a simple robotic hand having the appearance of an articulated claw with two opposite fingers [12].Volunteers in the experiment were required to make a prespecified response (to open or to close their right hand) as soon as a stimulus appears on the screen.Results showed an increased response time in incongruent compared to congruent conditions, in response to both human and robotic hand, suggesting that the motor priming effect generalized to robotic stimuli [12].Yet the size of the effect, (time difference between response to incongruent and congruent stimuli), was larger for human (~30 ms) that for robotic stimuli (~15 ms).

Resonance with humanoids
A follow-up experiment tested whether the effect is better explained by a bottom-up process due to the overall shape or a top-down process caused by the knowledge of the intentionality of humans compared to robotic devices [13].Human hands were modified by the addition of a metal and wire wrist, and were perceived as less intentional than the original hands.No significant differences were found between the priming effect of the original and of the robotized human hand, in favor of the bottom-up hypothesis that the overall hand shape, and not its description as a human or robotic hand, affects the priming effect.

3.1.2
Motor interference with humanoid DB Another series of experiments [14,15], was initiated by Kilner et al.'s [16] study of motor interference.In contrast to observing a human, they did not find any interference effect when facing an industrial robotic arm moving at a constant velocity, suggesting that motor interference is specific to interactions between human agents.We investigated this question with the humanoid robot DB.In our experiment, subjects performed rhythmic arm movements while observing either a human agent or the humanoid robot DB performing either congruent or incongruent BIO Web of Conferences movements.In a first experiment, trajectories were derived from captured human motion of the same movements performed by the human control for the experiment [14].Volunteers were instructed to make rhythmic arm movements from either the top-right to the bottom-left or from the top-left to the bottom-right, while standing approximately 2 m away from either a humanoid robot or another human that was performing similar arm movements.The movements produced by the other agent (human or DB) were either spatially congruent or incongruent.During each 30-second trial, the kinematics of the endpoint of the subject's right index finger was recorded with a motion capture device.
The variance of the executed movements was used as a measure of motor interference caused by the observed action.We found that in contrast to the industrial robotic arm, the humanoid robot caused a significant interference effect, such that the ratio between the variance in the incongruent and in the congruent conditions increased from the industrial robotic arm (r=1, no effect; [16]), the humanoid robot (r~1.5, [14]) and the human (r~2, [14,16]).
In a follow-up experiment, we investigated the effect of the movement kinematics on the interference.The humanoid robot moved either with a biological motion based, as previously, on recorded trajectories, or with an artificial motion implemented by a 1-DOF sinusoidal movement of the elbow.We found a that the increase in incongruent conditions was only significant when the robot movements followed biological motion [15].A similar trend for artificial motion was not significant.A final experiment assessed whether seeing the full body or only body parts of the other agent influences motor resonance [1].The effect of interference could be due merely to the anthropomorphic appearance of the agent, or it could be influenced by the knowledge we have about the nature of the other agent.To test whether appearance was the main factor, we covered the body and face of both agents, the human and the humanoid robot, with a black cloth leaving just the moving arm visible, and compared the results of the interference paradigm between covered and uncovered agents.Results indicate that the variance is increased only when the body is visible.Behavioral experiments thus provide results in favor of a bottom-up effect of anthropomorphism on behavioral markers of resonance.

3.2.1
Neuroimaging of grasping movements Motor resonance has been extensively studied with neuroimaging in humans, and it is possible to adapt similar approaches to the perception of anthropomorphic robots.Neuroimaging experiments comparing the observation of humans versus robots have so far yielded mixed results.In a PET study, subjects were presented with grasping action performed by a human or by a robotic arm.The authors report that the left ventral premotor activity found in previous experiments of action observation responded to human, but not robot, actions [17].However, results of an fMRI study indicate that a robotic arm and hand elicits motor resonance, in the form of increased activity in regions activated by the execution of actions during the observation of object-directed actions compared to simple movements [18].How can we reconcile these two sets of results?One possibility, the difference techniques used in these experiments, PET and fMRI, cannot explain the dramatic reversal of the results.Another possibility derives from differences in anthropomorphism of the robotic arm and hand used by the two groups, But one major difference between the two studies comes from the experimental instructions.In the first report, "subjects were instructed to carefully observe the human (experimenter) or the robot model", while in the second, "subjects were instructed to watch the movies carefully, paying particular attention to the relationship between the agents and the objects."I suggest that these differences could explain discrepancies in the results.

3.2.2
Neuroimaging of facial actions Using fMRI [19], we recorded brain activity when participants (n=13) observed short video clips of human and humanoid robot WE-4RII facial expressions of emotions (Joy, Anger, Disgust), while rating the emotion (explicit processing of emotion) or the movement (implicit task).The robot stimuli were rated as significantly less emotional than the humans'.We identified regions involved in motor resonance by computing the main effect of action observation.We were particularly interested in activity in the left ventral premotor cortex.There was a significant interaction (Figure) between the subjects' task (implicit or explicit) and the agent used to display the stimulus (human or robot).Based on our hypothesis on the ontogenic origin of resonance, I propose that bottomup processes would be automatic when perceiving human stimuli, and show little modulation by the task.In contrast, robot stimuli cannot be processed automatically because the system has no internal model of robots' actions.When the task is to explicitly judge emotion, the large increase of activity in the left inferior frontal cortex in response to robot stimuli results from forcing the perceptual system to process these stimuli as goal-directed, anthropomorphic, actions.The interaction between task and agent would The International Conference SKILLS 2011 thus derive from an interaction between bottom-up processes, influenced by the nature of the agent (automatic for human, not for robot), and top-down processes, depending on the object of attention.If this interpretation is correct, motor resonance towards artificial agents would be enhanced when the agents' movements are explicitly processed as actions.

Conclusions
I presented behavioral and fMRI experiments investigating how motor resonance, a plausible foundation for higher-order social cognition, differs between natural and artificial agents.On the basis of both results and the proposed ontogeny of motor resonance, I propose that while the anthropomorphic shape of humanoid robots is sufficient to trigger a reduced resonance, compared to real humans, it is strongly affected by top-down modulation such as the orientation of attention towards actions' goals.This finding has implication for the design of socially acceptable humanoid platform, irrespective of their final use.

Figure
Figure: graphs presenting brain activity in the left ventral premotor cortex in response to human (white) and robot (grey) agents depending on the Task (error bar represent SEM).