Humans process knowledge, simultaneously across multiple modes of data inputs. The way we learn, our experiences are all multimodal. We don't have just vision, just audio and just text.
These foundational principles of learning were articulated by the father of modern education John Amos Comenius, in his work, "Orbis Sensualium Pictus", dating back to 1658.
"All things that are naturally connected ought to be taught in combination"
Contrary to those principles, in the past, our approach to Machine Learning was often focused on specialised models tailored to process a single modality. For…