Reference · Glossary
Multimodal
Last updated
Document Q&A over screenshots, image captioning, voice + text agents, video understanding, or any job where the evidence is not text-only.
#When to use
Document Q&A over screenshots, image captioning, voice + text agents, video understanding, or any job where the evidence is not text-only.
#When not to
Text-only tasks where a smaller language model is cheaper and clearer. Extra modalities add cost, privacy surface, and new failure modes.
#Quality checklist
- State which modalities are authoritative inputs
- Verify OCR/transcript steps separately from reasoning
- Watch for models inventing details not in the image/audio
- Check retention for uploaded media
#Example
Input: photo of a whiteboard + question "list action items"
Require: quote only what is visible; mark unclear handwriting#Learn next
- Lesson: `multimodal-ai`
- How-to: Call a vision API