Multi-modal
Multi-modal refers to AI systems capable of processing and integrating multiple types of information inputs at the same time, such as text, images, audio, and video.
How it works
It processes the data of each modality through separate neural network modules, then fuses this information for unified understanding and generation. This allows AI to perceive the world more fully—much like humans using their eyes and ears at the same time—and is a key step toward more general, more capable artificial intelligence.
Examples
- 1.An AI assistant can 'see' the image you uploaded and 'listen' to your voice command at the same time to answer a question
- 2.Self-driving cars combine camera footage, radar data, and map information to make driving decisions
- 3.A generative AI model can create an image matching a text description