Perception and action is linked via the environment. The environment consists of a physical and a social part, and most studies addressing human perception and action have focused on the interaction with the physical world. However, for the proper functioning of an individual, communication and interaction with its social environment is highly important. There is little doubt that perception aids interaction and communication and therefore is essential for an individual in its social context. Furthermore, it is assumed that the human perceptual and action system phylogenetically developed also due to the demands present in social interaction and exchange with others.

Some “special” skills are necessary in social interaction, which are not necessarily required for the interaction with the physical world. Humans need to recognize the actions of others and their goals. They have to perceive inner states like emotions, intentions and beliefs and predict the actions and the course of ongoing movements. Finally, they have to coordinate attention and action when it comes to close interaction. It seems reasonable to assume that the perceptual system has specialized processes dedicated to the processing of social information because of the frequency and abundance with which humans have to deal with social information.

In this area we investigate a) coordination issues between individuals, b) perceptual influences by the mere presence of others, and c) potential specialized perceptual processes in the perception and action during human interaction.

◘ Multi User Interaction and Control
◘ Accessing social information during joint action coordination
◘ The role of perceptual processes in the recognition of social interaction
◘ Motion Perception in Virtual Environments

The role of perceptual processes in the recognition of social interaction

Humans typically interact with two major classes of visual stimuli in everyday life: objects and other humans. While we have a fair understanding of how objects are recognized, little is known about the recognition of social interactions (recognizing person-directed actions). Classically the recognition of social interactions was assumed to rely mainly on higher-level cognitive processes (e.g. inference), whereas more recent evidence suggests that lower-level perceptual processes may play an important role [1]. Of particular importance are cortical units that directly link the execution of a particular action and perception of the same action (i.e. mirror neurons or action-perception units) [1]. However there is little evidence that such units exist in humans. Moreover, relatively little is known about the involvement of other perceptual processes in social interaction recognition.

Our aim is to examine the involvement of perceptual processes, particularly those of an action-perception unit, in the recognition of social interactions.

We investigated the existence of an action-perception unit behaviorally and with fMRI in a human-object interaction task. We reasoned that the repeated execution of an action should lead to an adaptation effect in the action-perception unit which, in turn, should affect the perception of that same action (see also Figure 1ab). In a different set of experiments, we assessed the efficiency of social interaction recognition when the contribution of the action-perception unit was minimized. Because we know that object recognition is efficient [2], we compared the speed of recognition when viewing static images of social interactions and static images of objects.

Figure 1a

Figure 1b
Figure 1ab: Experimental design (1a) and representative results (1b) of the action adaptation-perceptual transfer
experiments. 1a: Participants did one of two conditions: they repeatedly grabbed a cylinder in the grab condition
(top panel in 1a) or did a ‘fake grab’ next to the cylinder in the non-grab condition (red) (bottom panel in 1a).
Following this action adaptation, participants saw a movie of variable length of a person either grabbing or fake
grabbing a cylinder (green) and answered whether they saw a grab movement (gray). These phases were repeated
several times. 1b shows participants’ performance (d prime) at each movie length to discriminate between these
two movies. Participants showed the same discrimination performance regardless of whether they adapted to a grab
or non-grab action. Bars indicate standard error from the mean.

We failed to find an action adaptation-perception transfer under a variety of experimental conditions. Moreover, we found that social interaction recognition is efficient even when the contribution of an action-perception unit is minimized. Specifically, the speed of identifying social interactions is not statistically different from identifying objects (Figure 2).

Figure 2: Representative results from the experiments investigating the speed of object and social interaction
recognition. Mean presentation time thresholds for recognizing static images of objects and social interactions
with 80% accuracy in three different recognition tasks (detection, categorization, and identification). Only the
mean categorization thresholds differed significantly between social interaction and object recognition. Bars
indicate standard error from the mean.

We might not have found action adaptation-perception transfer because an action-perception unit a) generally does not show adaptation effects, or b) is tuned to human-human interactions .The latter possibility will be explored in future experiments. Our second line of experiments, however, suggests that social interaction recognition is efficient even when an action-perception unit is hardly employed. Our results therefore challenge the view that an action-perception unit is critical for social interaction recognition. Future experiments will examine the perceptual processes mediating social interaction recognition behaviorally with virtual environments (see also Streuber, S.) and with imaging techniques.

REFERENCE
◘ Sebanz, N., Bekkering, H., Knoblich, G: Joint action: bodies and minds moving together., Trends in Cognitive Science, 10:70-76.

◘ Thorpe, S. , Fize, D., Marlot, C. : Speed of processing in the human visual system., Nature, 381, 520-522, 1996