Traditionally, perceptual research has been compartmentalized into distinct and isolated categories according to individual modalities (i.e., visual, auditory, haptic, proprioceptive, vestibular, etc.).  This modular approach has treated individual sensory and motor processing as involving largely independent systems.  However, recently investigators are recognizing the importance of understanding cross-modal interactions and how they relate to perception and behaviour.

Much of the multisensory research up until this point has focused upon tasks involving discrete stimulus presentations in near body space. Such cue interactions of interest have included: visual-auditory integration, visual-proprioceptive integration, or visual-haptic integration. It is important, however, to investigate multisensory integration from the perspective of large-scale self-motion through action space. Unlike traditional approaches to examining the integration of two specific cues at a particular instance in time, navigating through one’s environment requires the dynamic integration of several cues across space and over time (i.e., visual flow, lower-limb proprioception, and vestibular information).  Understanding the principles underlying multimodal integration in this context of unfolding cue dynamics is very important as it provides insight into an important category of multisensory processing.

◘ Multi-sensory integration in the estimation of distance traveled
◘ Contribution of inertial information to the perception of walking speed
◘ Perception of visual speed while walking
◘ Adaptive treadmill control
◘ Vestibular perception is slow
◘ Perceived object stability
◘ Shape from shading

Shape from shading

In environments where orientation is ambiguous, the visual system uses prior knowledge about lighting coming from above to recognize objects, reorient the body, and determine which way is up (where is the sun?). It has been shown that when observers are tilted to the side relative to gravity, the orientation of the light-from-above prior will change in a direction between the orientation of the body, gravity and the visual surround. Here we investigate the contribution of ocular torsion in this change of the light-from-above prior has been acknowledged but not specifically addressed. Our initial investigation tested the hypothesis that when lighting direction is the only available visual orientation cue, change in orientation of the light-from-above prior is accounted for by ocular torsion. In this experiment observers made convex-concave judgments of a central shaded disk, flanked by three similarly- and three oppositely-shaded disks. Lighting was tested every 15° in roll in the fronto-parallel plane. Using the KUKA motion simulator to move observers into different orientations, observers were tested when upright, supine, and tilted every 30° in role relative to gravity. Our results show that change of the light-from-above prior is well predicted from a sum of two sines; one consistent with predicted ocular torsion, the other consistent with an additional component varying with twice the frequency of body tilt. The next phase of our investigations will address the nature of this second component as well as assess the relative contribution of additional lighting cues added to the surrounding environment.

REFERENCE
◘ Barnett-Cowan, M., M. O. Ernst and H. H. Bülthoff: "Where is the sun?" The sun is 'up' in the eye of the beholder. European Conference on Visual Perception 2010