Images

Send images as multimodal input to Gemini

Gemini models are multimodal: you can include images alongside text for captioning, classification, visual question answering, and more. Add image steps to the input array. Images may be supplied as inline base64 data or referenced by URL.

The Interactions API is best supported on the Enterprise plan. Use https://enterprise.blackbox.ai as the base URL.

Inline Base64

Encode the image and pass it as data with the correct mime_type. PNG and JPEG are supported.

import os
import base64
import requests

with open('chart.png', 'rb') as f:
    b64 = base64.b64encode(f.read()).decode()

response = requests.post(
    'https://enterprise.blackbox.ai/v1/interactions',
    headers={
        'Content-Type': 'application/json',
        'Authorization': f"Bearer {os.environ['BLACKBOX_API_KEY']}",
    },
    json={
        'model': 'gemini-3.8-flash',
        'store': False,
        'input': [
            {'type': 'text', 'text': 'Caption this image.'},
            {'type': 'image', 'data': b64, 'mime_type': 'image/png'},
        ],
    },
)
print(response.json())

B64=$(base64 -i chart.png)
curl https://enterprise.blackbox.ai/v1/interactions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BLACKBOX_API_KEY" \
  -d '{
    "model": "gemini-3.8-flash",
    "store": false,
    "input": [
      { "type": "text", "text": "Caption this image." },
      { "type": "image", "data": "'"$B64"'", "mime_type": "image/png" }
    ]
  }'

Inline data counts toward the total request size. Keep text, system instructions, and inline image bytes under 20 MB combined. For larger or reused images, pass a URL instead.

Image by URL

Reference a publicly accessible image with uri. The URL must exist and be crawlable (images blocked by robots.txt or returning 404 are rejected).

import os
import requests

response = requests.post(
    'https://enterprise.blackbox.ai/v1/interactions',
    headers={
        'Content-Type': 'application/json',
        'Authorization': f"Bearer {os.environ['BLACKBOX_API_KEY']}",
    },
    json={
        'model': 'gemini-3.8-flash',
        'store': False,
        'input': [
            {'type': 'text', 'text': 'Describe this image in five words.'},
            {'type': 'image',
             'uri': 'https://upload.wikimedia.org/wikipedia/commons/3/3f/JPEG_example_flower.jpg',
             'mime_type': 'image/jpeg'},
        ],
    },
)
print(response.json())

curl https://enterprise.blackbox.ai/v1/interactions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BLACKBOX_API_KEY" \
  -d '{
    "model": "gemini-3.8-flash",
    "store": false,
    "input": [
      { "type": "text", "text": "Describe this image in five words." },
      { "type": "image", "uri": "https://example.com/flower.jpg", "mime_type": "image/jpeg" }
    ]
  }'

Multiple Images

Include several image steps in one request to compare or reason across images:

{
  "model": "gemini-3.8-flash",
  "store": false,
  "input": [
    { "type": "text", "text": "What is different between these two images?" },
    { "type": "image", "uri": "https://example.com/image1.jpg", "mime_type": "image/jpeg" },
    { "type": "image", "uri": "https://example.com/image2.jpg", "mime_type": "image/jpeg" }
  ]
}

Request Parameters

input[].typestringrequired

Use "image" for an image step, alongside "text" steps.

input[].datastring

Base64-encoded image bytes. Mutually exclusive with uri.

input[].uristring

URL of a publicly accessible image. Mutually exclusive with data.

input[].mime_typestringrequired

The image MIME type, e.g. image/png or image/jpeg.

Next Steps

Text Generation

Combine images with multi-turn prompts

Reasoning

Reason over visual input