A theoretical framework for transfer of knowledge across modalities in artificial and biological systems

Learning from sensory patterns associated with different kinds of sensors is paramount for biological systems, as it permits them to cope with complex environments where events rarely appear twice in the same way. In this paper we want to investigate how perceptual categories formed in one modality can be transferred to another modality in biological and artificial systems. We first present a study on Mongolian gerbils that show clear evidence of transfer of knowledge for a perceptual category from the auditory modality to the visual modality. We then introduce an algorithm that mimics the behavior of the rodents within the online learning framework. Experiments on simulated data produced promising results, showing the pertinence of our approach.