Bayesian integration of visual and vestibular information

Self-motion through an environment involves a composite of signals, including visual and vestibular cues. It has been shown that visual-auditory cues and visual-haptic cues combine in a statistically optimal fashion [1] but very little is currently known about visual-vestibular cue integration.
Our goal - we investigated the relative weights of visual and vestibular cues during self-motion. Further, we tested the limits of this cue integration by creating spatial [2] and temporal offsets between the two cues, providing different acceleration profiles to the two cues [3], and by changing the immersive nature of the visual input (i.e. 2D vs. 3D) [4].
Participants performed a 2-interval forced choice task and were asked in which of two movement intervals did they moved more to the right (see Figure 1). From these responses we plotted psychometric functions from which we extracted the participants’ variance (reliability). We predicted the combined cue responses and the weights of the individual cues from the responses in the unimodal conditions [1]. To observe the weights assigned to each modality and to assess the limits of cue integration, we introduced either a spatial, temporal or motion profile discrepancy between the visual and vestibular cues in the standard visual-vestibular heading.
Our results show that visual-vestibular cue combination is robust, such that when there are conflicts between the visual and vestibular cues; participants exhibit a statistically optimal reduction of variance. Furthermore, we found that the unimodal cues did not predict the weights in the combined cue but that there is a prior which lends more weight to the vestibular cue. Finally, we found visual-vestibular cue integration breaks down when the visuals are not presented in stereo.


Butler JS Person, Smith ST Person, Campos JL Person and Bülthoff HH Person (September-2010) Bayesian integration of visual and vestibular signals for heading Journal of Vision 10(11) 1-13.
CiteID: 6783
Butler JS Person, Campos JL Person and Bülthoff HH Person (August-2008): The robust nature of visual -vestibular combination for heading, 31st European Conference on Visual Perception, Utrecht, Netherlands, Perception, 37(ECVP Abstract Supplement) 40.
CiteID: 5133
Butler JS Person, Bülthoff HH Person and Smith ST Person (July-2008) Abstract Talk: The role of stereo vision in visual and vestibular cue integration, 9th International Multisensory Research Forum (IMRF 2008), Hamburg, Germany 179.
CiteID: 5159
Ernst MO Person and Banks MS Person (January-2002) Humans Integrate Visual and Haptic Information in a Statistically Optimal Fashion Nature 415(6870) 429-433.
pdfCiteID: 1005

Export as:
BibTeX, XML, Pubman, Edoc
Last updated: Friday, 05.10.2012