Reference · API

Gemini multimodal generateContent

Last updated

Send **text, images, audio, or video** in one request; get text (or structured) output from Google Gemini models.

Send **text, images, audio, or video** in one request; get text (or structured) output from Google Gemini models.

#When to use

  • Describe or classify images in product flows
  • OCR + reasoning over screenshots or PDFs
  • Video/audio understanding without building separate pipelines
  • For text-only chat, the same endpoint works — multimodal parts are optional.

    #Endpoint

    POST https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent

    #Request shape (sketch)

    {
      "contents": [{
        "role": "user",
        "parts": [
          {"text": "What is in this image?"},
          {"inline_data": {"mime_type": "image/jpeg", "data": "<base64>"}}
        ]
      }]
    }

    #Python sketch

    import google.generativeai as genai
    genai.configure(api_key="...")
    model = genai.GenerativeModel("gemini-2.0-flash")
    response = model.generate_content([
        "Summarize this receipt.",
        {"mime_type": "image/png", "data": open("receipt.png", "rb").read()},
    ])
    print(response.text)

    #Common pitfalls

    | Pitfall | Fix |

    |---------|-----|

    | Huge inline base64 payloads | Resize images; use file API for large media |

    | Wrong MIME type | Match bytes to declared `mime_type` |

    | Safety blocks on user uploads | Handle empty/blocked responses gracefully |

    | Unpinned model name in prod | Lock model version in config |