KakaoTalk_Photo_2021-10-29-17-11-38_edited.jpg

MULTISENSORY PERCEPTION

RESEARCH

Human perceptual experience at any moment is a conglomeration of different sensory information. While you’re browsing this website, you may hear some music and you may feel the surface of the computer keyboard, or a hot mug filled with coffee. Experimental results so far have shown that what we hear or how we feel can affect what we see, or vice versa. However, we all know that we have five distinct senses. Also, neuroscience has taught that we have separate neural pathways for each of those senses. This seemingly contradictory findings – integration of different sensory experiences and correspondence of seemingly unrelated features between sensory modalities, despite distinct processing routes for individual senses –is one of the most important questions in recent perception research.

PUBLISHED WORKS

We have been studying multisensory perception with both psychophysical and neuroscientific methods. In particular, we have been exploring interactions between multiple sensory processes.

Published works include, 

  • Kwak, Y., Nam, H., Kim, H.-W., & Kim, C-Y. (2020) Cross-modal correspondence between speech sound and visual shape influencing perceptual representation of shape: the role of articulation and pitch. Multisensory Research, 33(6), 569-598. 

  • Park, M., Blake, R., Kim, Y., & Kim, C-Y. (2019). Congruent audio-visual stimulation during adaptation modulates the subsequently experienced visual motion aftereffect. Scientific reports, 9(1), 1-11. 

  • Kim, H-W., Nam, H., & Kim, C-Y. (2018) [i] is Lighter and More Greenish than [o]: Intrinsic Association between Vowel Sounds and Colors. Multisensory Research, 31, 419-437.  

  • Kim, S., Blake, R., Lee, M., & Kim, C-Y. (2017) Audio-visual interactions uniquely contribute to resolution of visual conflict in people processing absolute pitch. PLoS One,12(4). 

NEW STUDIES

  • Audio-visual interactions during motion adaptation modulates the perceived duration of the motion aftereffect and the brain activity 

  • Visual perception affected by prior information on audiovisual correspondence outside of awareness 

  • Microsaccades and pupil dynamics during audiovisual spatial integration 

  • Perceptual deficits in audiovisual temporal and spatial integration in schizophrenia 

RESEARCH GRANTS

Our research on multisensory perception has been supported by research grants including 

  • "Harmony of the Senses: Generalization, Extension, and Application of Multisensory Interaction Studies (Years 2016-2019, NRF-2016R1A2B4011267)" 

  • “Coexistence of the senses – multisensory integration, synesthesia, and its neural concomitants (Years 2013-2016, NRF-2013R1A1A1010923)" 

  • “Union of the Senses and underlying neural mechanisms (Years 2009-2011, KRF-2009-332-H00011)”