Perception and action is linked via the environment. The environment consists of a physical and a social part, and most studies addressing human perception and action have focused on the interaction with the physical world. However, for the proper functioning of an individual, communication and interaction with its social environment is highly important. There is little doubt that perception aids interaction and communication and therefore is essential for an individual in its social context. Furthermore, it is assumed that the human perceptual and action system phylogenetically developed also due to the demands present in social interaction and exchange with others.

Some “special” skills are necessary in social interaction, which are not necessarily required for the interaction with the physical world. Humans need to recognize the actions of others and their goals. They have to perceive inner states like emotions, intentions and beliefs and predict the actions and the course of ongoing movements. Finally, they have to coordinate attention and action when it comes to close interaction. It seems reasonable to assume that the perceptual system has specialized processes dedicated to the processing of social information because of the frequency and abundance with which humans have to deal with social information.

In this area we investigate a) coordination issues between individuals, b) perceptual influences by the mere presence of others, and c) potential specialized perceptual processes in the perception and action during human interaction.

◘ Multi User Interaction and Control
◘ Accessing social information during joint action coordination
◘ The role of perceptual processes in the recognition of social interaction
◘ Motion Perception in Virtual Environments

Accessing social information during joint action coordination

Understanding human-human interaction is an emerging research field. However, up until now, there has been relatively little empirical evidence regarding the perceptual and cognitive mechanisms involved in such interactions. One interesting aspect of human interaction is the role of sensory feedback during the performance of coordinated tasks. Classically the process of human interaction was assumed to rely mainly on higher-level cognitive processes (e.g. inference) whereas more recent evidence suggests that lower-level perceptual processes may play an important role e.g. [1]. In particular, several pieces of evidence suggest a direct link between an action (e.g. grasping) and the perception of the same action [2]. These findings suggest that information about the interaction partner’s body might play an important role in human interaction.

Our goal is to investigate the importance of different sources of sensory information on joint-action performance under conditions that allow good external validity. In particular we are using Virtual Reality (VR) technology which allows for high level of experimental control over visual information and precise measurements, while at the same time providing a high degree of realism and enabling natural behavior.

In a dyadic stretcher-carrying task, we investigated the contributions of different sources of information during joint-action coordination. Using VR allowed us to manipulate the visual information about the task and about the interaction partner. In all conditions subjects physically carried a stretcher through a computer generated maze, which was visible in all conditions. In the three testing conditions subjects either perceived no additional information about either the stretcher or the joint-action partner (invisible condition), visual information about the interaction partner (partner condition) or visual information about the stretcher (stretcher condition). To investigate the effect of these different sources of visual information on joint-action performance we compared participants’ walking trajectories in the abovementioned conditions to their walking trajectories when participants perceived visual information about both the stretcher and the interaction partner (control condition).

The walking trajectories in the experimental conditions and control condition were not significantly different. Hence, visual information about the stretcher and the interaction partner did not affect joint-action behavior.

Collaborative Virtual Environment in which two participants carry a stretcher. Left: Subjects
equipped with laptops, head mounted displays and tracking helmets. Right: Visual stimulus from
a third person perspective driving simulation: gradually reduce contrast (according to visual
depth) induces lower produced speed when compared to normal visibility (clear) and uniformly
reduced contrast.

REFERENCE
◘ Sebanz, N., Bekkering, H., Knoblich, G: Joint action: bodies and minds moving together., Trends in Cognitive Science, 10:70-76.

◘ Rizzolatti, G. and Craighero, L : The Mirror-Neuron System., ANNUAL REVIEW OF NEUROSCIENCE, 27:169-192, 2004

◘ Streuber, S., Chatziastros, A., Mohler,B. and Bülthoff, H.:: Joint and individual walking in an immersive collaborative virtual environment., APGV 2008