Multimodal AI refers to models that can process and generate multiple types of data — text, images, audio, and sometimes video — within a single system. GPT-4V can analyze images and answer questions about them; Gemini processes text, images, and audio together. This breaks down silos between data types: a factory can show a camera image to a multimodal model and ask in text what defect it shows, getting a text answer referencing the visual.
LLMs & Models
Multimodal AI
Related terms
Learn to use these concepts in practice.
Join the 212AY Academy →